Screen projection method, device, electronic device and computer-readable storage medium

Abstract

The present disclosure provides a screen projection method, device, electronic device and computer-readable storage medium, which relate to the field of vehicle networking technology, and in particular to the field of vehicle-machine interconnection technology. The specific implementation scheme is: in response to a screen projection instruction for a first application, the first page when the first application is running is added to the second page when the second application is running to obtain a page to be projected, wherein the second application has a screen projection function; and the page to be projected is projected to a second terminal device. Based on this scheme, by merging the page when the first application is running with the page when the second application is running, and performing screen projection display based on the screen projection function of the second application, screen projection in the vehicle-machine interconnection scenario can be effectively realized, meeting the user's usage needs.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet of Vehicles, and in particular to the field of vehicle-machine interconnection technology. Specifically, the present disclosure relates to a screen projection method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the rapid development of terminal technology, in-vehicle terminals are becoming more and more popular. In-vehicle terminals are generally equipped with display screens for content display.

[0003] When using an in-vehicle terminal, users may need to project the display content of a mobile terminal (such as a mobile phone) to the in-vehicle terminal in real time for display, allowing the user to view the display content on the mobile terminal through the in-vehicle terminal. Therefore, how to effectively project the display content of a mobile terminal (such as a mobile phone) to the in-vehicle terminal in real time has become a major technical issue in the field of vehicle-to-machine interconnection. Summary of the Invention

[0004] In order to solve at least one of the above-mentioned defects, the present disclosure provides a screen projection method, device, electronic device and computer-readable storage medium.

[0005] According to a first aspect of the present disclosure, a screen projection method is provided, the method comprising:

[0006] In response to a screen projection instruction for a first application, adding a first page when the first application is running to a second page when a second application is running to obtain a page to be projected, wherein the second application has a screen projection function;

[0007] Project the page to be projected to the second terminal device.

[0008] According to a second aspect of the present disclosure, a screen projection device is provided, the device comprising:

[0009] a screen-to-be-cast page generation module, configured to, in response to a screen-casting instruction for a first application, add a first page when the first application is running to a second page when the second application is running to obtain a screen-to-be-cast page, wherein the second application has a screen-casting function;

[0010] The screen projection module is used to project the page to be projected to the second terminal device.

[0011] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0012] at least one processor; and

[0013] A memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the above-mentioned at least one processor, and the instructions are executed by the above-mentioned at least one processor so that the above-mentioned at least one processor can execute the above-mentioned screen projection method.

[0015] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned screen projection method.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above-mentioned screen projection method when executed by a processor.

[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0019] Figure 1 This is a flowchart of a screen projection method provided by an embodiment of the present disclosure;

[0020] Figure 2 This is a schematic diagram of the page to be projected obtained after page fusion provided by the embodiment of the present disclosure and the display effect after the page to be projected is projected;

[0021] Figure 3 This is a flowchart of a specific implementation of the screen projection method provided in the embodiments of the present disclosure;

[0022] Figure 4 This is a structural diagram of a screen projection device provided according to the present disclosure;

[0023] Figure 5 It is a block diagram of an electronic device used to implement the screen projection method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1A schematic diagram of a screen projection method provided by an embodiment of the present disclosure is shown, which can be applied to a first terminal device, such as Figure 1 As shown in , the method may mainly include:

[0026] Step S110: In response to a screen projection instruction for the first application, adding a first page when the first application is running to a second page when the second application is running to obtain a page to be projected, wherein the second application has a screen projection function;

[0027] Step S120: Project the page to be projected to the second terminal device.

[0028] The first terminal device may be a user's mobile terminal, such as a mobile phone, a tablet computer, etc. The second terminal device may be a vehicle-mounted terminal (ie, a vehicle-mounted system), etc.

[0029] The second application has a screen projection function. In the vehicle-machine interconnection scenario, the second application can be an application based on the MirrorLink screen projection protocol. The first terminal device and the second terminal device can establish a connection via USB, Bluetooth, or WiFi.

[0030] When a screen projection instruction for a first application is detected, the first page of the first application running can be added to the second page of the second application running to obtain a screen projection page. The screen projection page is a fusion of the first and second pages, and the screen projection page contains the content of the first page. The screen projection page can be projected to the second terminal device for display based on the screen projection function of the second application.

[0031] The method provided in the embodiment of the present disclosure projects the first page of the first application when it is running to the second page of the second application when it is running, in response to a screen projection instruction for the first application, to obtain a page to be projected, thereby projecting the page to be projected to the second terminal device based on the screen projection function of the second application. Based on this solution, by fusing the page of the first application when it is running with the page of the second application when it is running, and projecting the screen based on the screen projection function of the second application, screen projection in the vehicle-machine interconnection scenario can be effectively realized, meeting the user's usage needs.

[0032] The method provided by the embodiment of the present disclosure can realize screen projection of the runtime page of the first application that does not have the screen projection function based on the screen projection function of the second application, thereby improving the screen projection function of the terminal device and meeting the user's screen projection needs for multiple applications.

[0033] In the embodiment of the present disclosure, a display (Presentation) component can be loaded to display the second page when the second application is running. A surface view (SurfaceView) component is added to the Presentation component. After obtaining the screen recording permission, a surface (Surface) object is obtained from the SurfaceView component, and the Surface object is assigned to the media projection (MediaProjection) object. The first page of the first application at runtime is captured through the MediaProjection object, so that the first page is projected to the SurfaceView component, thereby adding the first page of the first application at runtime to the second page of the second application at runtime. At this time, the Presentation component is used to display the page to be projected.

[0034] In the embodiment of the present disclosure, the view of the Presentation component (i.e., the page to be projected) can be encoded through a media codec (Mediacodec) object to achieve the projection of the page to be projected to the second terminal device.

[0035] In an optional embodiment of the present disclosure, the above method further includes:

[0036] In response to a user's first page switching instruction to the first page in the page to be projected, the first page in the page to be projected is removed.

[0037] Taking the vehicle-machine interconnection scenario as an example, in the related art, when an application with a screen projection function on a mobile terminal (equivalent to the second application in this example) projects the page of another application (equivalent to the first application in this example) to the vehicle terminal, it generally uses the system's native screen recording mechanism to record the page of the first application displayed on the first terminal device, and then performs video encoding at the business layer before projecting it to the vehicle terminal. In this way, the page of the first application will be fully displayed on the vehicle terminal. If the page of the second application needs to be displayed on the vehicle terminal, it is necessary to switch the video stream, which is relatively inconvenient.

[0038] In an embodiment of the present disclosure, when the first page of the first application is being projected to the second terminal device for display, the page to be projected includes the first page. At this time, if it is necessary to project the second page of the second application when it is being run to the second terminal device for display, the first page included in the page to be projected can be removed. At this time, the page to be projected only includes the second page, thereby realizing the projection of the second page of the second application when it is being run to the second terminal device for display.

[0039] In the disclosed embodiment, a page switching control can be set in the page to be projected, and the user can switch between the first page and the second page by clicking the page switching control. Specifically, when the page to be projected includes the first page, if it is detected that the user has clicked the page switching control, a first page switching instruction can be generated, and in response to the first page switching instruction, the first page in the page to be projected is removed.

[0040] In an optional manner of the present disclosure, after removing the first page from the pages to be projected, the method further includes:

[0041] In response to the user's second page switching instruction to the second page in the page to be projected, the first page is moved into the page to be projected.

[0042] In an embodiment of the present disclosure, when the second page of the second application is being projected to the second terminal device for display, the first page is not included in the page to be projected. At this time, if it is necessary to project the first page of the first application when it is being run to the second terminal device for display, the first page can be moved into the page to be projected so that the first page is included in the page to be projected, thereby realizing the projection of the first page of the first application when it is being run to the second terminal device for display.

[0043] When the first page is not included in the page to be projected, if it is detected that the user has clicked the page switching control, a second page switching instruction can be generated, and the first page can be moved into the page to be projected in response to the second page switching instruction.

[0044] The solution provided by the embodiment of the present disclosure obtains the page to be projected by adding the first page when the first application is running to the second page when the second application is running, and projects the page to be projected. When it is necessary to switch between the first and second projected pages, the first page can be removed from or moved into the page to be projected, thereby achieving switching between the first and second pages in the content of the projected page without the need to switch video streams, which is more convenient.

[0045] In an optional manner of the present disclosure, the size of the first page is smaller than the page to be projected.

[0046] In the embodiment of the present disclosure, the size of the first page can be configured to be smaller than the size of the page to be projected, so that part of the content of the second page can also be displayed in the page to be projected. This part can add some controls for the second application, such as controls for exiting the projection or controls for voice control, so that when the user projects the page of the first application, there is no need to switch the projection content to the page of the second application, and the user can operate the second application, which is convenient for the user.

[0047] In an optional embodiment of the present disclosure, the above method further includes:

[0048] In response to a page adjustment instruction from the user for the first page in the screen to be projected, page adjustment is performed on the first page in the screen to be projected.

[0049] In an embodiment of the present disclosure, the first page in the to-be-projected screen may be adjusted to meet the user's different demands for displaying the first page during projection.

[0050] In an optional manner of the present disclosure, in response to a user's page adjustment instruction for a first page in the screen to be projected, performing page adjustment on the first page in the screen to be projected includes any of the following:

[0051] In response to a user's instruction to adjust the size of a first page in the pages to be projected, adjusting the size of the first page in the pages to be projected;

[0052] In response to a user's instruction to adjust the position of a first page in the pages to be projected, the position of the first page in the pages to be projected is adjusted.

[0053] In the disclosed embodiment, adjusting the first page may include adjusting the size and position of the first page in the page to be projected. Users can adjust the size and position of the first page in the page to be projected based on actual needs. For example, adjusting the size of the first page allows for full display of part of the content on the second page. Another example is flipping the first page 90° to accommodate the user's viewing habits.

[0054] Specifically, a page adjustment control can be set in the page to be projected, and the user can adjust the page by clicking the first page of the page adjustment control. For example, after the user clicks the page adjustment control, the user can zoom, rotate, and drag the first page. A page size adjustment instruction is generated based on the user's zoom operation, and the size of the first page in the page to be projected is adjusted in response to the page size adjustment instruction; a page position adjustment instruction is generated based on the user's rotation and dragging operations, and the size of the first page in the page to be projected is adjusted in response to the page position adjustment instruction.

[0055] In an optional embodiment of the present disclosure, generating a screen projection instruction for a first application includes:

[0056] presenting the candidate application list to the user via the second application;

[0057] Based on the user's selection operation on the candidate application list, a first application is determined from the candidate application list, and a screen projection instruction for the first application is generated.

[0058] In an embodiment of the present disclosure, a candidate application list may be displayed in the second application. The candidate application list includes controls corresponding to the candidate applications, and the candidate applications are applications that are allowed to perform screen projection.

[0059] The user can select the candidate application by clicking on the control corresponding to the candidate application. Based on the user's selection operation, the first application can be determined from the candidate applications and a screen projection instruction for the first application can be generated.

[0060] In the embodiment of the present disclosure, after the user completes the selection operation of the first application, the first mobile terminal can jump from the interface of the second application to the interface of the first application, providing a basis for subsequent screen recording operations.

[0061] In an optional embodiment of the present disclosure, the above method further includes:

[0062] Updates are made to the candidate application list.

[0063] In the embodiment of the present disclosure, the candidate application list can be updated according to actual needs.

[0064] In an optional manner of the present disclosure, updating the candidate application list includes any of the following:

[0065] In response to a user's editing operation on the candidate application list, updating the candidate application list;

[0066] Receive update information of the candidate application list sent by the server, and update the candidate application list based on the update information.

[0067] In the embodiment of the present disclosure, the candidate application list can be updated in response to a user's editing operation. Specifically, the user can edit the candidate application list according to actual needs, such as adding or deleting candidate applications in the candidate application list.

[0068] In the embodiment of the present disclosure, the candidate application list may also be updated based on update information obtained from the server. Specifically, the server may generate update information when the candidate application list is updated and send it to the first terminal device.

[0069] In an optional manner of the present disclosure, adding a first page when a first application is running to a second page when a second application is running to obtain a page to be projected includes:

[0070] Display a prompt interface to the user to ask whether to enable screen recording permission;

[0071] In response to the user's operation of enabling screen recording permission in the prompt interface, the first page when the first application is running is added to the second page when the second application is running to obtain the page to be projected.

[0072] In the disclosed embodiment, the MediaProjection object requires the user to enable screen recording permission before it can capture the first page of the first application during runtime. Therefore, the user can be asked whether to enable screen recording permission. After detecting that the user has enabled screen recording permission, subsequent operations can be performed.

[0073] As an example, the prompt interface may display the words "Do you want to enable screen recording permission?" and provide controls corresponding to the operation of enabling screen recording permission and controls corresponding to the operation of refusing to enable screen recording permission.

[0074] In an optional method of the present disclosure, projecting the page to be projected to the second terminal device includes:

[0075] Generate the video stream of the page to be projected;

[0076] The video stream is sent to the second terminal device so that the second terminal device plays the video stream.

[0077] In the embodiment of the present disclosure, the view of the Presentation component (i.e., the page to be projected) can be encoded through the Mediacodec object to generate a video stream of the page to be projected, and then the video stream can be sent to the second terminal device so that the second terminal device plays the video stream to realize the projection function.

[0078] As an example, Figure 2 The figure shows a schematic diagram of the page to be projected obtained after page fusion in the embodiment of the present disclosure and the display effect after the page to be projected is projected.

[0079] Among them, A is the second page when the second application is running, in which the SurfaveView component is added. B is the mobile phone screen, that is, the first page when the first program is running. The Surface object is assigned to the MediaProjection object, and the first page of the first application at runtime is captured through the MediaProjection object, and the data of the first page is given to the Surface object for display, that is, the first page is projected to the SurfaceView component. C is the page to be projected after projecting the first page to the second page. D is the page of the second terminal device. E is the page to be projected that is projected to the second terminal device, that is, the projection page finally displayed to the user.

[0080] As an example, Figure 3A flow chart of a specific implementation of the screen projection method provided in an embodiment of the present disclosure is shown in FIG. In this example, the second application is a Carelife application.

[0081] like Figure 3 As shown in , Carelife on the mobile phone side is the second application running on the first terminal device, and Carelife on the vehicle side is the program corresponding to the second application running on the second terminal device.

[0082] 1. Carelife connection is successful, that is, the first terminal device establishes communication with the second terminal device.

[0083] 2. Open the third-party application page and load the mobile application whitelist, that is, display the candidate application list to the user through the second application.

[0084] 3. Select an application and load the SurfaceView component, that is, based on the user's selection operation on the candidate application list, determine the first application from the candidate application list, and then load the SurfaceView component.

[0085] 4. Get the Surface object through SurfaceView and assign it to the MediaProjection object, that is, get the Surface object from the SurfaceView component and assign the Surface object to the MediaProjection object.

[0086] 5. Apply for global screen recording permission from the Android system of the mobile phone, that is, apply to the user to enable the system's screen recording permission.

[0087] 6. Add the SurfaceView component to the Carelife projection page Presentation, that is, add the SurfaceView component to the Presentation component. The Presentation component is used to display the second page when the second application is running.

[0088] 7. Carelife uses Mediacodec to encode the Presentation view, realizing the integration of Carelife's own view and third-party application view. Figure 1 Projecting to the mobile phone, that is, encoding the view of the Presentation component (that is, the page to be projected) through the Mediacodec object, generating a video stream of the page to be projected, and then sending the video stream to the second terminal device, so that the second terminal device plays the video stream to realize the screen projection function.

[0089] Continuously send global screen recording (video, audio stream), that is, the system of the first terminal device continuously sends screen recording data including video stream and audio stream to the second application, and the first terminal device realizes screen projection by continuously sending screen recording data including video stream and audio stream to the second terminal device.

[0090] Based on Figure 1 The same principle as shown in the method, Figure 4 A schematic diagram of the structure of a screen projection device provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the screen projection device 40 may include:

[0091] a screen-to-be-cast page generation module 410 for, in response to a screen-casting instruction for a first application, adding a first page when the first application is running to a second page when a second application is running to obtain a screen-to-be-cast page, wherein the second application has a screen-casting function;

[0092] The screen projection module 420 is used to project the page to be projected to the second terminal device.

[0093] The device provided by the embodiment of the present disclosure, in response to a screen projection instruction for the first application, projects the first page when the first application is running to the second page when the second application is running to obtain a page to be projected, thereby projecting the page to be projected to the second terminal device based on the screen projection function of the second application. Based on this solution, by fusing the page when the first application is running with the page when the second application is running, and projecting the display based on the screen projection function of the second application, screen projection in the vehicle-machine interconnection scenario can be effectively realized, meeting the user's usage needs.

[0094] Optionally, the above device further includes:

[0095] The first page switching module is used to remove the first page in the page to be projected in response to the user's first page switching instruction to the first page in the page to be projected.

[0096] Optionally, the above device further includes:

[0097] The second page switching module is used to move the first page into the page to be projected in response to a second page switching instruction of the user to the second page in the page to be projected after the first page in the page to be projected is removed.

[0098] Optionally, the size of the first page is smaller than the page to be projected.

[0099] Optionally, the above device further includes:

[0100] The page adjustment module is used to adjust the first page in the screen to be projected in response to the user's page adjustment instruction for the first page in the screen to be projected.

[0101] Optionally, the page adjustment module is specifically used for any of the following:

[0102] In response to a user's instruction to adjust the size of a first page in the pages to be projected, adjusting the size of the first page in the pages to be projected;

[0103] In response to a user's instruction to adjust the position of a first page in the pages to be projected, the position of the first page in the pages to be projected is adjusted.

[0104] Optionally, when generating a screen projection instruction for the first application, the screen projection page generation module is specifically configured to:

[0105] presenting the candidate application list to the user via the second application;

[0106] Based on the user's selection operation on the candidate application list, a first application is determined from the candidate application list, and a screen projection instruction for the first application is generated.

[0107] Optionally, the above device further includes:

[0108] The candidate application list updating module is used to update the candidate application list.

[0109] Optionally, when updating the candidate application list, the candidate application list updating module is specifically used for any of the following:

[0110] In response to a user's editing operation on the candidate application list, updating the candidate application list;

[0111] Receive update information of the candidate application list sent by the server, and update the candidate application list based on the update information.

[0112] Optionally, when the screen page to be projected is obtained by adding the first page when the first application is running to the second page when the second application is running, the screen page generation module is specifically used to:

[0113] Display a prompt interface to the user to ask whether to enable screen recording permission;

[0114] In response to the user's operation of enabling screen recording permission in the prompt interface, the first page when the first application is running is added to the second page when the second application is running to obtain the page to be projected.

[0115] Optionally, the screen projection module is specifically used to:

[0116] Generate the video stream of the page to be projected;

[0117] The video stream is sent to the second terminal device so that the second terminal device plays the video stream.

[0118] It is understandable that the above modules of the screen projection device in the embodiment of the present disclosure have the function of realizing Figure 1 The functions of the corresponding steps of the screen projection method in the embodiment shown in . This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of each module of the above screen projection device, please refer to Figure 1 The corresponding description of the screen projection method in the embodiment shown in will not be repeated here.

[0119] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0121] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the screen projection method provided in the embodiment of the present disclosure.

[0122] Compared with the existing technology, this electronic device responds to the screen projection instruction of the first application and projects the first page of the first application when it is running to the second page of the second application when it is running to obtain the page to be projected, thereby projecting the page to be projected to the second terminal device based on the screen projection function of the second application. Based on this solution, by fusing the page of the first application when it is running with the page of the second application when it is running, and projecting the screen based on the screen projection function of the second application, it can effectively realize screen projection in the vehicle-machine interconnection scenario, meeting the user's usage needs.

[0123] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the screen projection method provided in the embodiment of the present disclosure.

[0124] Compared with the existing technology, this readable storage medium responds to the screen projection instruction of the first application and projects the first page of the first application when it is running to the second page of the second application when it is running to obtain the page to be projected, thereby projecting the page to be projected to the second terminal device based on the screen projection function of the second application. Based on this solution, by fusing the page of the first application when it is running with the page of the second application when it is running, and projecting the screen based on the screen projection function of the second application, it can effectively realize screen projection in the vehicle-machine interconnection scenario, meeting the user's usage needs.

[0125] The computer program product includes a computer program, which implements the screen projection method provided in the embodiment of the present disclosure when executed by a processor.

[0126] Compared to existing technologies, this computer program product responds to a screen projection instruction for a first application by projecting the first page of the first application running onto the second page of the second application running, thereby obtaining a page to be projected. This solution effectively implements screen projection in vehicle-to-machine interconnection scenarios by fusing the pages of the first application running with the pages of the second application running, and then projecting them based on the projection function of the second application, meeting user needs.

[0127] Figure 5 A schematic block diagram of an example electronic device 2000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0128] like Figure 5 As shown, the device 2000 includes a computing unit 2010, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 2020 or a computer program loaded from a storage unit 2080 into a random access memory (RAM) 2030. Various programs and data required for the operation of the device 2000 can also be stored in the RAM 2030. The computing unit 2010, the ROM 2020, and the RAM 2030 are connected to each other via a bus 2040. An input / output (I / O) interface 2050 is also connected to the bus 2040.

[0129] Various components in device 2000 are connected to I / O interface 2050, including an input unit 2060, such as a keyboard and mouse; an output unit 2070, such as various types of displays and speakers; a storage unit 2080, such as a magnetic disk and optical disk; and a communication unit 2090, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 2090 allows device 2000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 2010 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 2010 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 2010 executes the screen projection method provided in the embodiments of the present disclosure. For example, in some embodiments, the execution of the screen projection method provided in the embodiments of the present disclosure can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 2080. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 2000 via the ROM 2020 and / or the communication unit 2090. When the computer program is loaded into the RAM 2030 and executed by the computing unit 2010, one or more steps of the screen projection method provided in the embodiments of the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 2010 may be configured in any other appropriate manner (for example, by means of firmware) to execute the screen projection method provided in the embodiments of the present disclosure.

[0131] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0137] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0138] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A screen projection method, comprising: presenting the list of candidate applications to the user via a second application located on the same terminal as the first application; Based on the user's selection operation on the candidate application list, determining the first application from the candidate application list and generating a screen projection instruction for the first application; the candidate application list includes controls corresponding to the candidate application, and the candidate application is an application that is allowed to project the screen; In response to a screen projection instruction for the first application, a first page of the first application when it is running is added to a second page of the second application when it is running to obtain a page to be projected, wherein the first application does not have a screen projection function, the second application has a screen projection function, the page to be projected is obtained by merging the first page and the second page, and the page to be projected includes the content of the first page; the step of adding the first page of the first application when it is running to the second page of the second application when it is running to obtain the page to be projected includes: loading a display component for displaying the second page of the second application when it is running, adding a surface view component to the display component, obtaining a surface object from the surface view component after obtaining screen recording permission, assigning the surface object to a media projection object, grabbing the first page of the first application when it is running through the media projection object, and projecting the first page to the surface view component; The video of the media projection object is encoded by a media encoder object, and the page to be projected is projected onto the second terminal device.

2. The method according to claim 1, further comprising: In response to a user's first page switching instruction for the first page in the page to be projected, the first page in the page to be projected is removed.

3. The method according to claim 2, after removing the first page from the pages to be projected, further comprising: In response to the user's second page switching instruction for the second page in the page to be projected, the first page is moved into the page to be projected.

4. The method according to any one of claims 1 to 3, wherein The size of the first page is smaller than the page to be projected.

5. The method according to any one of claims 1 to 3, further comprising: In response to a user's page adjustment instruction for the first page in the pages to be projected, page adjustment is performed on the first page in the pages to be projected.

6. The method according to claim 5, wherein: The step of adjusting the first page of the to-be-projected pages in response to a user's page adjustment instruction for the first page of the to-be-projected pages comprises any one of the following: In response to a user's instruction to adjust the page size of a first page in the pages to be projected, adjusting the size of the first page in the pages to be projected; In response to a user's instruction to adjust the page position of the first page in the pages to be projected, the position of the first page in the pages to be projected is adjusted.

7. The method according to claim 1, further comprising: The candidate application list is updated.

8. The method according to claim 7, wherein: The updating of the candidate application list includes any one of the following: updating the candidate application list in response to a user's editing operation on the candidate application list; Receive update information of the candidate application list sent by a server, and update the candidate application list based on the update information.

9. The method according to any one of claims 1 to 3, wherein The step of adding the first page when the first application is running to the second page when the second application is running to obtain the page to be projected includes: Display a prompt interface to the user to ask whether to enable screen recording permission; In response to the user's operation of enabling screen recording permission in the prompt interface, the first page when the first application is running is added to the second page when the second application is running to obtain the page to be projected.

10. The method according to any one of claims 1 to 3, wherein: Projecting the page to be projected onto the second terminal device includes: Generate a video stream of the page to be projected; The video stream is sent to a second terminal device, so that the second terminal device plays the video stream.

11. A screen projection device, comprising: A screen projection page generation module is used to display a list of candidate applications to the user through a second application located on the same terminal as the first application; Based on the user's selection operation on the candidate application list, determining the first application from the candidate application list and generating a screen projection instruction for the first application; the candidate application list includes controls corresponding to the candidate application, and the candidate application is an application that is allowed to project the screen; In response to a screen projection instruction for the first application, a first page of the first application when it is running is added to a second page of the second application when it is running to obtain a page to be projected, wherein the first application does not have a screen projection function, the second application has a screen projection function, the page to be projected is obtained by merging the first page and the second page, and the page to be projected includes the content of the first page; the page to be projected generation module is further used to load a display component for displaying the second page when the second application is running, add a surface view component to the display component, obtain a surface object from the surface view component after obtaining screen recording permission, assign the surface object to a media projection object, capture the first page of the first application when it is running through the media projection object, and project the first page to the surface view component; The screen projection module is used to encode the video of the media projection object through the media encoder object and project the page to be projected to the second terminal device.

12. The apparatus according to claim 11, further comprising: The first page switching module is used to remove the first page in the page to be projected in response to the user's first page switching instruction for the first page in the page to be projected.

13. The apparatus according to claim 12, further comprising: The second page switching module is used to move the first page into the page to be projected in response to the user's second page switching instruction for the second page in the page to be projected after the first page in the page to be projected is removed.

14. The device according to any one of claims 11 to 13, wherein: The size of the first page is smaller than the page to be projected.

15. The apparatus according to any one of claims 11 to 13, further comprising: The page adjustment module is used to adjust the first page in the page to be projected in response to the user's page adjustment instruction for the first page in the page to be projected.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images