Screen projection method and device, electronic equipment and storage medium

Abstract

This application discloses a screen projection method, apparatus, electronic device, and storage medium, belonging to the field of screen projection technology. The method includes: upon receiving a request from an in-vehicle terminal to project N windows of an electronic device onto the in-vehicle terminal, creating N first virtual screens and one second virtual screen; each first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; for each first virtual screen, laying out and drawing the screen data of the corresponding window through the first virtual screen to obtain a root layer tree for each first virtual screen; rendering and compositing the root layer tree of each first virtual screen through the second virtual screen to obtain a projection frame, the projection frame containing screen data of the N windows; and projecting the projection frame onto the in-vehicle terminal.

Description

Technical Field

[0001] This application belongs to the field of screen projection technology, specifically relating to a screen projection method, device, electronic device, and storage medium. Background Technology

[0002] With the rapid development of automobiles, the number and size of in-vehicle screens are constantly increasing, and users' expectations for in-vehicle screen projection functions are no longer satisfied with the display of a single application window. Currently, in-vehicle screen projection solutions that support multi-application split-screen display have emerged in the industry.

[0003] In existing multi-window projection technology solutions, when the vehicle-mounted infotainment system initiates a projection request, the mobile terminal creates a binding combination of a "first virtual screen, second virtual screen, and encoder" for each application to be projected. That is, one application corresponds to one first virtual screen, one first virtual screen is bound to an independent second virtual screen, and one second virtual screen is bound to an encoder. For each application, the first virtual screen corresponding to that application is used for layout and rendering of the application's screen. The corresponding second virtual screen is responsible for rendering and compositing the application's screen layers. The composited application frame is then sent to the encoder corresponding to that application, where it is independently encoded and transmitted to the vehicle-mounted display terminal via an independent communication link.

[0004] To achieve screen mirroring for multiple applications, the above solution requires the creation of multiple second virtual screens. This consumes a large amount of system resources such as video memory and graphics cache on the mobile terminal, which can easily lead to excessive hardware load on the mobile terminal. In addition, the multiple "second virtual screens and encoders" work independently, generating encoded video streams, which are then transmitted to the vehicle terminal through multiple independent communication links. This results in complex screen mirroring links and low resource utilization. Summary of the Invention

[0005] The purpose of this application is to provide a screen projection method, device, electronic device, and storage medium to reduce resource consumption and improve resource utilization.

[0006] In a first aspect, embodiments of this application provide a screen mirroring method, the method comprising: Upon receiving a request from the vehicle terminal to project N windows of an electronic device onto the vehicle terminal, N first virtual screens and one second virtual screen are created; one first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; For each first virtual screen, the screen data of the window corresponding to the first virtual screen is laid out and drawn through the first virtual screen to obtain the root layer tree corresponding to each window; The root layer tree of each window is rendered and composited through the second virtual screen to obtain a projection frame, which contains the screen data of N windows. The screen projection frame is projected onto the vehicle terminal.

[0007] Secondly, embodiments of this application provide a screen projection device, which includes: A creation module is used to, upon receiving a request from an in-vehicle terminal to project N windows of an electronic device onto the in-vehicle terminal, have the electronic device create N first virtual screens and one second virtual screen; one first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; The determination module is used to lay out and draw the screen data of the window corresponding to the first virtual screen for each first virtual screen, so as to obtain the root layer tree corresponding to each window. The compositing module is used to render and composite the root layer tree corresponding to each window through the second virtual screen to obtain a projection frame, wherein the projection frame contains the screen data of N windows; The screen projection module is used to project the screen projection frame to the vehicle terminal.

[0008] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and the programs or instructions, when executed by the processor, implement the method as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, upon receiving a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal, the electronic device can create N first virtual screens and one second virtual screen. This eliminates the need for a single second virtual screen, transforming the traditional approach of creating a combined "first virtual screen and second virtual screen" for each window into a single second virtual screen. This fundamentally reduces the creation of the second virtual screen, thus reducing the number of second virtual screens that the system needs to manage and maintain to just one. This significantly reduces the amount of critical graphics system resources such as video memory and graphics cache consumed by the electronic device during projection, thereby significantly reducing the risk of excessive hardware load caused by excessive resource contention and scheduling overhead. Furthermore, by generating a unified projection frame from the images of N windows within the same second virtual screen using the same compositing process, only one projection frame is needed to project it to the vehicle terminal. This simplifies the projection transmission link, eliminates the additional computational and bandwidth overhead caused by encoding and transmitting multiple independent video streams, saves computational resources, and improves resource utilization. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the traditional multi-application window screen mirroring process; Figure 2 This is a flowchart illustrating the screen mirroring method provided in some embodiments of this application; Figure 3 This is a schematic diagram of the screen projection process for N windows provided in some embodiments of this application; Figure 4 This is a schematic diagram illustrating the mapping relationship between the display areas of N first virtual screens on a second virtual screen, provided in some embodiments of this application; Figure 5 These are schematic diagrams illustrating the structure of a projection device according to some embodiments of this application; Figure 6 These are schematic diagrams illustrating the structure of an electronic device according to some embodiments of this application; Figure 7 These are schematic diagrams illustrating the hardware structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0015] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0016] The terminology used in the embodiments of this invention will be explained below.

[0017] The identifiers in this application are text, symbols, images, etc. used to indicate information, and can be displayed in the form of controls or other containers, including but not limited to text identifiers, symbol identifiers, and image identifiers.

[0018] The first virtual screen is an abstract display unit that provides an independent display area for a window. It is used to control the window's size, position, and layout. In other words, the first virtual screen is for the window's use, and the window's size can vary. It is the abstract screen responsible for the window's layout. In this embodiment, one window corresponds to one first virtual screen, which provides an independent drawing environment for each window. Each window that needs to be projected performs its interface drawing in a dedicated first virtual screen, ensuring that the display logic between windows is isolated and does not interfere with each other. The size and resolution parameters of the first virtual screen are the same as the parameters of the native display screen of its corresponding window. In this embodiment, the first virtual screen can generate complete interface graphic data for its corresponding window, and this data exists in the form of a "root layer tree" data structure.

[0019] The second virtual screen: A display canvas created by the mobile terminal for image compositing and output, corresponding to a fixed resolution, receiving image mappings from multiple first virtual screens and rendering them uniformly. In this embodiment, the second virtual screen acts as a "compositing canvas." Its main responsibility is to receive and render the graphic data from one or more "first virtual screens" to generate a complete final image. In this embodiment, the size of the second virtual screen is typically consistent with the size of the vehicle terminal's display screen to ensure point-to-point display. The second virtual screen is bound to the input surface of an encoder; the composited final image is drawn on this surface for the encoder to read and compress into a video stream.

[0020] The encoder's input surface is a shared graphics data buffer located in the graphics processing pipeline of the mobile terminal. It is a memory area of ​​a specific size that carries pixel data, serving as a data exchange interface between the graphics compositing module and the video encoding module. In this embodiment, the encoder's input surface is the final output target of the second virtual screen compositing operation; that is, after the layers of multiple first virtual screens are unified and composited, the generated "complete multi-window projection frame" is directly drawn onto this input surface. It is also the data input source for the video encoder. The encoder reads the already drawn complete image from this input surface and then compresses and encodes it, such as converting it to an H.264 / H.265 bitstream for transmission via the vehicle-to-everything (V2X) communication link.

[0021] The compositing module is a core system service in the Android operating system responsible for managing and compositing all graphics layers and ultimately outputting them to the physical display. Specifically, it can be the SurfaceFlinger within the operating system. SurfaceFlinger receives graphics data from graphics buffers of multiple windows, renders and composites them into a final, unified image frame based on layer attributes, and then submits it to the display hardware for refresh and display. In other words, SurfaceFlinger is the physical canvas that actually composites the image and outputs it to the encoder. In this embodiment, SurfaceFlinger is used to uniformly render and composite the layers of multiple first virtual screens to generate a "complete multi-window projection frame," and then draws this "complete multi-window projection frame" onto the input surface of the encoder.

[0022] The technical solution of this application embodiment can be applied to scenarios where multiple windows of a mobile terminal need to be projected onto the display screen of an in-vehicle terminal. For example, a user is resting at service area A and wants to project the music playback window B of their mobile phone's music player, the chat interface window C of their instant messaging application, and the navigation window D of their navigation application onto the display screen of the in-vehicle terminal for easy viewing.

[0023] As described in the background section, existing multi-window projection technology solutions, if a user wants to project the music playback window B of their mobile phone's music player, the chat interface window C of an instant messaging application, and the navigation window D of a navigation application onto the display screen of the vehicle terminal, then... Figure 1 As shown, the electronic device needs to create 3 first virtual screens, 3 second virtual screens, and 3 encoders, as follows: Figure 1In this configuration, the first virtual screen 11 corresponds to the music playback window B, the second virtual screen 21 is bound to the first virtual screen 11, the input surface 311 of the encoder 31 is bound to the second virtual screen 21, the first virtual screen 12 corresponds to the chat interface window C, the second virtual screen 22 is bound to the first virtual screen 12, the input surface 321 of the encoder 32 is bound to the second virtual screen 22, the first virtual screen 13 corresponds to the navigation window D, the second virtual screen 23 is bound to the first virtual screen 13, and the input surface 331 of the encoder 33 is bound to the second virtual screen 23. The first virtual screen 11 extracts complete screen data from the music playback window B, the second virtual screen 21 renders and synthesizes the extracted screen data from the music playback window B to obtain a composite frame corresponding to the music playback window B, and then inputs this composite frame to the input surface 311 of the encoder 31 for display. The encoder 31 then extracts the composite frame from the input surface 311 of the encoder 31, compresses and encodes it to obtain a video stream, and sends this video stream to the vehicle's infotainment system via the corresponding communication link.

[0024] The input surfaces 321 of the corresponding first virtual screen 12, second virtual screen 22 and encoder 32 use the same operation as the input surfaces 311 of the first virtual screen 11, second virtual screen 21 and encoder 31 to project the chat interface window C to the vehicle terminal. The input surfaces 331 of the first virtual screen 13, second virtual screen 23 and encoder 33 use the same operation as the input surfaces 311 of the first virtual screen 11, second virtual screen 21 and encoder 31 to project the navigation window D to the vehicle terminal.

[0025] Thus, each window needs to create an independent second virtual screen, first virtual screen, and encoder. This consumes a large amount of system resources such as video memory and graphics cache of the mobile terminal, which can easily lead to excessive hardware load on the mobile terminal, resulting in screen projection stuttering and latency. In addition, the above-mentioned multiple "second virtual screens and encoders" work independently, generating encoded video streams, which are then transmitted to the vehicle terminal through multiple independent communication links. This results in complex screen projection links and low resource utilization.

[0026] To address the aforementioned issues, this application embodiment, upon receiving a request from an in-vehicle terminal to project N windows of an electronic device onto the terminal, allows the electronic device to create N first virtual screens and one second virtual screen. This eliminates the need for a single second virtual screen, transforming the traditional approach of creating a binding combination of "first virtual screen and second virtual screen" for each window into a single second virtual screen. This fundamentally reduces the creation of the second virtual screen, thus reducing the number of second virtual screens that the system needs to manage and maintain. This significantly lowers the mobile terminal's consumption of critical graphics system resources such as video memory and graphics cache during projection, thereby significantly reducing the risk of excessive hardware load due to resource contention and scheduling overhead. Furthermore, by generating a unified projection frame from multiple window images within the same second virtual screen using the same compositing process, only one projection frame is needed to project it to the in-vehicle terminal. This simplifies the projection transmission link, eliminates the additional computational and bandwidth overhead associated with encoding and transmitting multiple independent video streams, saves computational resources, and improves resource utilization.

[0027] The screen projection method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Figure 2 This is a flowchart illustrating a screen projection method provided in an embodiment of this application. The subject executing the screen projection method can be an electronic device, which can be, but is not limited to, a personal computer (PC), a smartphone, a tablet computer, or a personal digital assistant (PDA). In this embodiment, the electronic device is used as a mobile terminal, such as a smartphone, for example.

[0029] like Figure 2 As shown, the screen projection method provided in this application embodiment may include steps 210-240.

[0030] Step 210: Upon receiving a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal, the electronic device creates N first virtual screens and one second virtual screen.

[0031] Among the aforementioned N first virtual screens, each first virtual screen corresponds to one window in the N windows, that is, there is a one-to-one correspondence between the N first virtual screens and the N windows, where N is a positive integer and N≥2. All of the aforementioned N first virtual screens are bound to the second virtual screens.

[0032] As in the example above, refer to Figure 3The vehicle terminal requests that the three windows of the electronic device be projected onto the vehicle terminal. Therefore, the electronic device will create three first virtual screens, namely first virtual screen 1, first virtual screen 2 and first virtual screen 3. Among them, first virtual screen 1 corresponds to music playback window B, first virtual screen 2 corresponds to chat interface window C, and first virtual screen 3 corresponds to navigation window D. First virtual screen 1, first virtual screen 2 and first virtual screen 3 are all bound to the created second virtual screen 4.

[0033] In some embodiments of this application, the electronic device creates N first virtual screens and one second virtual screen only upon receiving a screen mirroring request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal. This screen mirroring request may be generated by the vehicle terminal in response to a user's screen mirroring operation. For example, in the above example, the user wants to project the music playback window B of their mobile phone's music player, the chat interface window C of their instant messaging application, and the navigation window D of their navigation application onto the display screen of the vehicle terminal. Therefore, the user initiates a screen mirroring request from the vehicle terminal to their mobile phone. For instance, the user may click the "Initiate Screen Mirroring" control on the display screen of the vehicle terminal, or the user may input a screen mirroring command based on a voice command on the vehicle terminal, and then the vehicle terminal initiates a screen mirroring request to the electronic device based on this command.

[0034] It should be noted that before the electronic device receives a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal, it must ensure that the electronic device and the vehicle terminal are in the same network environment, that is, that the electronic device and the vehicle terminal have established a network connection.

[0035] In some embodiments of this application, after creating N first virtual screens and one second virtual screen, it is necessary to bind the N first virtual screens to the second virtual screen. Specifically, this can be done as follows: Call the mounting interface corresponding to the second virtual screen; Based on the mounting interface, each first virtual screen is registered to the first virtual screen mounting container corresponding to the second virtual screen, so as to bind each first virtual screen to the second virtual screen.

[0036] The mounting interface can be a new interface added to the second virtual screen for mounting the first virtual screen. This mounting interface can be provided by a component service of the operating system to support the mounting of the first virtual screen in the second virtual screen managed by the operating system. Specifically, the operating system can be a Linux system. The component service can be, but is not limited to, built using the Spring framework, Spring Boot framework, or ToySpring framework.

[0037] In some embodiments of this application, a new mounting interface added for the second virtual screen can be called. Based on this mounting interface, each first virtual screen can be registered to the first virtual screen mounting container corresponding to the second virtual screen. Specifically, the identifier corresponding to each first virtual screen can be registered to the first virtual screen mounting container corresponding to the second virtual screen, thereby mounting each first virtual screen to the second virtual screen. In this way, each first virtual screen can be bound to the second virtual screen. The identifier corresponding to the first virtual screen can be the unique ID of the first virtual screen. The first virtual screen mounting container can be a management list in the second virtual screen used to manage the mounted first virtual screens. Specifically, the core code on the surface of the system framework layer composite module of the electronic device can be modified to extend the data structure and application programming interface (API) of the second virtual screen, so that the second virtual screen can be transformed from an entity that can only manage a single first virtual screen into a composite entity that can manage multiple lists of first virtual screens.

[0038] It's important to note that within the second virtual screen instance, a list or collection of first virtual screens (i.e., a first virtual screen mounting container) is maintained. When a first virtual screen is bound to a second virtual screen, its instance or key information such as its root layer tree node is stored in this container. Subsequently, during rendering and compositing, the second virtual screen iterates through all items in this container, processing each bound first virtual screen.

[0039] It should be noted that binding the first virtual screen to the second virtual screen is not just a simple addition to the list. It can also include establishing a synchronization of the association between the second virtual screen and the first virtual screen. For example, the display configuration information such as the resolution and refresh rate of the second virtual screen can also be synchronized to the first virtual screen to ensure that the layout drawing output of the first virtual screen can be correctly guided to the rendering and compositing process of the second virtual screen.

[0040] In the embodiments of this application, by calling the mounting interface corresponding to the second virtual screen, each first virtual screen is registered to the first virtual screen mounting container corresponding to the second virtual screen based on the mounting interface, so that each first virtual screen is bound to the second virtual screen. This removes the strong dependency relationship of one-to-one binding between the first virtual screen and the second virtual screen in the traditional solution, supports dynamic binding of multiple first virtual screens to a single second virtual screen, reduces the number of second virtual screens created, reduces the system resource consumption of electronic devices, and improves the smoothness of screen projection.

[0041] Step 220: For each first virtual screen, lay out and draw the screen data of the window corresponding to the first virtual screen to obtain the root layer tree of each first virtual screen.

[0042] In this context, for each first virtual screen, the root layer tree of the first virtual screen can be the screen data used to represent the window corresponding to the first virtual screen.

[0043] It's important to note that the "root layer tree" is a core data structure in a graphics display system. A "layer" is the basic unit of processing in a graphics system, representing a drawable area on the screen. An application interface is typically composed of multiple layers stacked on top of each other. For example, an interface might include a background layer, a text layer, a button layer, an animation layer, etc. These layers are organized together according to parent-child, sibling, and other relationships, forming a tree structure. The top-level node of this tree is the "root layer," and the entire hierarchical structure starting from it is the "root layer tree."

[0044] The root layer tree of a first virtual screen contains all the graphic information required to constitute the current screen display content of the window corresponding to that first virtual screen. Extracting the root layer tree of a first virtual screen is equivalent to obtaining all the visual content that the window corresponding to that first virtual screen will display on the screen. In the embodiments of this application, each window to be projected corresponds to an independent first virtual screen. The first virtual screen is an abstract display device that manages the window layout and drawing, while the root layer tree is the set of graphic data to be composited that is finally drawn and output by this window on the first virtual screen. It can be understood that the first virtual screen is a "canvas", and the root layer tree is the "painting" that has been drawn on the "canvas" and is ready to be rendered and composited.

[0045] In some embodiments of this application, each first virtual screen can lay out and draw the screen data of its corresponding window, thereby obtaining the root layer tree of each first virtual screen.

[0046] Step 230: Render and composite the root layer tree of each first virtual screen using the second virtual screen to obtain the projection frame.

[0047] The projection frame can be a single frame obtained by rendering and compositing the root layer tree of each first virtual screen. This projection frame contains the screen data of each of the N windows to be projected. In the example above, the projection frame includes the screen data of music playback window B, chat interface window C, and navigation window D. In other words, the projection frame renders and composites the screen data of music playback window B, chat interface window C, and navigation window D and displays them in a single image frame.

[0048] In some embodiments of this application, a single projection frame containing all window images is ultimately generated. Thus, when transmitting to the vehicle terminal, only one data transmission link needs to be maintained and managed, avoiding potential competition, congestion, and asynchrony issues between multiple streams.

[0049] In some embodiments of this application, step 230 may specifically include: Extract the root layer tree for each first virtual screen; Based on the mapping relationship between the display areas of each first virtual screen and the second virtual screen, each first virtual screen is preprocessed so that each preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the mapping relationship; The root layer tree of each preprocessed first virtual screen is rendered and composited to obtain the projection frame.

[0050] The mapping relationship between the display areas of each first virtual screen and the second virtual screen is a layout blueprint for each first virtual screen on the second virtual screen. This blueprint precisely defines the position, size, and display area of ​​each window corresponding to each first virtual screen on the final composite image. Without this blueprint, the images of N windows will be arranged irregularly on the second virtual screen, making it impossible to form an effective split-screen display.

[0051] The preprocessing of the first virtual screen includes at least one of the following: The first virtual screen is scaled proportionally. The position of the first virtual screen is adjusted.

[0052] In some embodiments of this application, the root layer tree of each first virtual screen can be extracted first, and then each first virtual screen can be preprocessed according to the pre-set mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, so that each preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the above mapping relationship.

[0053] Continue referring to the above example, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the second virtual screen. If the display area of ​​the first virtual screen 1 on the second virtual screen is set to area 41, the display area of ​​the first virtual screen 2 on the second virtual screen is set to area 42, and the display area of ​​the first virtual screen 3 on the second virtual screen is set to area 43, then the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 can be preprocessed according to the above correspondence, so that the image data corresponding to the root layer tree of the first virtual screen 1 is placed in area 41, the image data corresponding to the root layer tree of the first virtual screen 2 is placed in area 42, and the image data corresponding to the root layer tree of the first virtual screen 3 is placed in area 43.

[0054] Then, the root layer tree of each preprocessed first virtual screen is rendered and composited to obtain a projection frame. The image data in region 41, region 42 and region 43 of the second virtual screen can be rendered and composited to obtain an image frame. That is, the image data in region 41, region 42 and region 43 are rendered and composited into one image to obtain a projection frame.

[0055] It should be noted that in the above example, in the second virtual screen 4, the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 are arranged in a left-to-right order. In this embodiment, the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 can also be arranged in a top-to-bottom order. The specific arrangement of the display positions of each first virtual screen in the second virtual screen can be set by the user according to their needs, and is not limited in this embodiment.

[0056] In the embodiments of this application, by extracting the root layer tree of each first virtual screen and preprocessing each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, the root layer trees of all the preprocessed first virtual screens are then synthesized to obtain the projection frame. In this way, the root layer trees of all the first virtual screens are collected into the context of the same second virtual screen and a one-time global rendering synthesis is performed on the same "canvas". This fundamentally merges the compositing task of N windows into one unified compositing task and merges the N video encoding and transmission tasks into one task, greatly reducing the number of second virtual screens and saving a lot of graphics system resources required to create and maintain these instances. Moreover, the computational overhead of a single synthesis is far lower than the sum of the overhead of multiple independent compositings, thus reducing the load on the central processing unit or graphics processing unit of the electronic device and also reducing the power consumption of the electronic device.

[0057] In some embodiments of this application, after preprocessing each first virtual screen, the method described above may further include: Based on the sorting of the display positions of the root layer tree of each preprocessed first virtual screen on the second virtual screen, the root layer tree of each preprocessed first virtual screen is placed in the composition queue of the second virtual screen. The process of rendering and compositing the root layer tree of each preprocessed first virtual screen to obtain the projection frame may specifically include: The root layer tree of each first virtual screen in the compositing queue is rendered and composited to obtain the projection frame.

[0058] The synthesis queue of the second virtual screen can be a queue used to cache the data information of the second virtual screen.

[0059] In some embodiments of this application, the root layer tree of each preprocessed first virtual screen is placed in the composition queue of the second virtual screen according to the sorting of the display positions of the root layer tree of each first virtual screen on the second virtual screen, as in the example above. (Continuing to refer to...) Figure 4 ,exist Figure 4 In the diagram, the first virtual screen 1 is located in region 41, the first virtual screen 2 is located in region 42, and the first virtual screen 3 is located in region 43. That is, the first virtual screen 1 is located in the left region of the second virtual screen, the first virtual screen 2 is located in the middle region of the second virtual screen, and the first virtual screen 3 is located in the right region of the second virtual screen. Based on the display positions of the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 on the second virtual screen, the corresponding window of the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 can be placed into the composition queue of the second virtual screen in order from left to right. In the composition queue of the second virtual screen, the first virtual screen 1, the first virtual screen 2, and the first virtual screen 3 can also be arranged in the order of left, middle, and right.

[0060] Then, based on a compositing module, such as SurfaceFlinger, the root layer tree of each preprocessed first virtual screen in the compositing queue can be composited according to its position in the compositing queue, thereby obtaining the projection frame.

[0061] In the embodiments of this application, the root layer tree of each preprocessed first virtual screen is placed in the compositing queue of the second virtual screen according to the sorting of the display positions of the root layer tree of each first virtual screen on the second virtual screen. Then, the root layer trees of each preprocessed first virtual screen in the compositing queue are composited according to their position in the compositing queue to obtain the projection frame. In this way, it can be ensured that in the final composite projection frame, each window can be accurately located in the preset partition according to the pre-set mapping relationship of the display area of ​​each first virtual screen on the second virtual screen. The screens of each window will not be disordered, overlapped or misaligned, realizing the WYSIWYG multi-window layout. In this way, it can be ensured that the projection windows corresponding to each first virtual screen run independently and the display logic does not interfere with each other. The upper layer application does not need to adapt the split screen rules, which greatly reduces the application adaptation cost.

[0062] In some embodiments of this application, before preprocessing each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, the method described above may further include: Obtain the first size information of each first virtual screen and the second size information of each second virtual screen; Based on the first size information of each first virtual screen and the second size information of the second virtual screen, determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen.

[0063] Specifically, for each first virtual screen, the first size information of the first virtual screen is the size information of the first virtual screen that was created.

[0064] The second size information refers to the size information of the created second virtual screen. This second size information can be determined based on the first size information of the vehicle terminal's display screen. For example, the second size information can be proportional to the third size information, such as a 1:1 ratio or a 1:2 ratio. The specific relationship between the first and third size information can be set according to user needs and is not limited in this embodiment. However, to facilitate lossless and proportional projection display of subsequent projection frames and simplify subsequent cropping and layout calculations, the first size information can be consistent with the third size information.

[0065] In some embodiments of this application, first size information of each first virtual screen and second size information of the second virtual screen can be obtained. Then, based on the first size information of each first virtual screen and the second size information of the second virtual screen, the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen can be determined. Specifically, the system can automatically determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen based on the first size information of each first virtual screen and the second size information of the second virtual screen. Alternatively, the system can determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen according to the user's needs, i.e., in response to the user's arrangement instructions, based on the first size information of the first virtual screen and the second size information of the second virtual screen.

[0066] In the embodiments of this application, by determining the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, it is ensured that subsequent compositing steps are based on a solid foundation, thereby guaranteeing that the final multi-window layout projected onto the vehicle terminal screen is completely consistent with the design expectations, and the image will not be disordered, overlapped, or misaligned. Furthermore, before compositing begins, the layout parameters of all windows are calculated in batches at once, forming a global parameter set of the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen. This avoids repeating complex real-time calculations when scaling and offsetting each layer later, improving the overall efficiency of the compositing process.

[0067] Step 240: Project the screen frame to the vehicle terminal.

[0068] In some embodiments of this application, after obtaining the projection frame, the projection frame can be projected to the vehicle terminal.

[0069] In some embodiments of this application, when a request is received from an in-vehicle terminal to project N windows of an electronic device onto the in-vehicle terminal, the method described above may further include: Create an encoder corresponding to the second virtual screen; After obtaining the screen-casting frame, the methods mentioned above may also include: The projection frame is drawn onto the input surface of the encoder; Correspondingly, step 240 may specifically include: Obtain the coordinate information of each window drawn on the input surface of the encoder. The coordinate information corresponding to each window is synchronized to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window on the corresponding area of ​​the vehicle terminal's display screen.

[0070] The encoder can be an entity created by the electronic device that corresponds to the second virtual screen and is used to encode the projection frames, such as... Figure 3 Encoder 6 in the middle.

[0071] In some embodiments of this application, when a vehicle terminal requests that N windows of an electronic device be projected onto the vehicle terminal, in addition to creating N first virtual screens and one second virtual screen, an encoder corresponding to the second virtual screen is also created. Then, after obtaining the projection frame, the projection frame is drawn on the input surface of the encoder.

[0072] like Figure 3 As shown, after obtaining the projection frame, the projection frame can be drawn on the input surface 5 of the encoder 6, and then the projection frame drawn on the input surface of the encoder can be projected to the vehicle terminal. Specifically, the projection frame on the input surface 5 can be obtained by the encoder 6, the projection frame can be encoded to obtain a video stream, and then the video stream can be sent to the vehicle terminal.

[0073] It should be noted that the size of the aforementioned projection frame is consistent with the size of the encoder's input surface, so that the projection frame can be drawn proportionally on the encoder's input surface.

[0074] In some embodiments of this application, since each window is a complete large image on the input surface of the encoder, it is necessary to obtain the coordinate information of each window drawn on the input surface of the encoder. Then, the coordinate information corresponding to each window can be synchronized to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window in the corresponding area of ​​the display screen of the vehicle terminal.

[0075] In the embodiments of this application, compared to the traditional approach of creating an encoder for each window, this application embodiment only creates one encoder, thus fundamentally reducing encoder creation. This reduces the number of encoder instances that the system needs to manage and maintain to just one, significantly lowering the resource consumption of critical graphics systems such as video memory and graphics cache during screen projection. Therefore, the risk of excessive hardware load due to resource contention and scheduling overhead is significantly reduced. Furthermore, the coordinate information of each window drawn on the encoder's input surface is synchronized to the vehicle terminal. The vehicle terminal can then crop the projection frame based on the coordinate information of each window to obtain the application screen corresponding to each window, and display the application screen of each window in the corresponding area of ​​the vehicle terminal's display screen. This eliminates the need for complex screen analysis or layout calculations by the vehicle terminal; a simple cropping operation is sufficient to divide the projection frame and project it to different screen areas of the vehicle terminal, greatly simplifying the processing logic of the vehicle terminal and reducing its computational burden.

[0076] The screen projection method provided in this application can be executed by a screen projection device. This application uses an information processing device to execute the screen projection method as an example to illustrate the screen projection device provided in this application.

[0077] Figure 5 This is a schematic diagram illustrating the structure of a screen projection device according to an exemplary embodiment. For example... Figure 5 As shown, the projection device 500 may include: The creation module 510 is used to create N first virtual screens and one second virtual screen when the electronic device receives a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal; one first virtual screen corresponds to one window in the N windows, N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; The determination module 520 is used to lay out and draw the screen data of the window corresponding to the first virtual screen for each first virtual screen, so as to obtain the root layer tree of each first virtual screen. The compositing module 530 is used to render and composite the root layer tree of each first virtual screen through the second virtual screen to obtain a projection frame, wherein the projection frame contains the screen data of N windows; The screen projection module 540 is used to project the screen projection frame to the vehicle terminal.

[0078] In this embodiment, upon receiving a request from the vehicle terminal to project N windows of an electronic device onto the vehicle terminal, the electronic device can create N first virtual screens and one second virtual screen. This eliminates the need for a single second virtual screen, transforming the traditional approach of creating a combined "first virtual screen and second virtual screen" for each window into a single second virtual screen. This fundamentally reduces the creation of the second virtual screen, thus reducing the number of second virtual screens that the system needs to manage and maintain to just one. This significantly reduces the amount of critical graphics system resources such as video memory and graphics cache consumed by the electronic device during projection, thereby significantly reducing the risk of excessive hardware load caused by excessive resource contention and scheduling overhead. Furthermore, by generating a unified projection frame from the images of multiple windows within the same second virtual screen using the same compositing process, only one projection frame is needed to project it to the vehicle terminal. This simplifies the projection transmission link, eliminates the additional computational and bandwidth overhead caused by encoding and transmitting multiple independent video streams, saves computational resources, and improves resource utilization.

[0079] In some embodiments of this application, the synthesis module may specifically include: Extraction unit, used to extract the root layer tree of each first virtual screen; The preprocessing unit is used to preprocess each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, so that the preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the mapping relationship; The compositing unit is used to render and composite the root layer tree of each preprocessed first virtual screen to obtain the projection frame; The preprocessing of the first virtual screen includes at least one of the following: The first virtual screen is scaled proportionally. The position of the first virtual screen is adjusted.

[0080] In some embodiments of this application, the apparatus further includes: The first acquisition module is used to acquire first size information of each first virtual screen and second size information of the second virtual screen before preprocessing each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen. The second size information is determined based on the third size information of the display screen of the vehicle terminal. The determining module is used to determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen based on the first size information of each first virtual screen and the second size information of the second virtual screen.

[0081] In some embodiments of this application, the apparatus further includes: The calling module is used to call the mounting interface corresponding to the second virtual screen; The mounting module is used to register each first virtual screen to the first virtual screen mounting container corresponding to the second virtual screen based on the mounting interface, so as to bind each first virtual screen to the second virtual screen.

[0082] In some embodiments of this application, when a request is received from an in-vehicle terminal to project N windows of an electronic device onto the in-vehicle terminal, the creation module is further configured to create an encoder corresponding to the second virtual screen; The device further includes: A drawing module is used to draw the projection frame on the input surface of the encoder after the projection frame is obtained; The second acquisition module is used to acquire the coordinate information of each window drawn on the input surface of the encoder after the screen projection frame is projected to the vehicle terminal. The synchronization module is used to synchronize the coordinate information corresponding to each window to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window in the corresponding area of ​​the display screen of the vehicle terminal.

[0083] The screen projection device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0084] The screen projection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0085] The screen projection device provided in this application embodiment can achieve... Figure 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0086] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described screen projection method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0087] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0088] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0089] The electronic device 700 includes, but is not limited to, components such as: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0090] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0091] The processor 710 is configured to, upon receiving a request from a vehicle terminal to project N windows of the electronic device onto the vehicle terminal, create N first virtual screens and one second virtual screen; each first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; for each first virtual screen, the screen data of the window corresponding to the first virtual screen is laid out and drawn through the first virtual screen to obtain a root layer tree corresponding to each first virtual screen; the root layer tree of each first virtual screen is rendered and synthesized through the second virtual screen to obtain a projection frame, the projection frame containing screen data of N windows; and the projection frame is projected onto the vehicle terminal.

[0092] Thus, when receiving a request from the vehicle terminal to project N windows of an electronic device onto the terminal, the electronic device can create N first virtual screens and one second virtual screen. This eliminates the need for a single second virtual screen, transforming the traditional approach of creating a combined "first and second virtual screens" for each application window into a single second virtual screen. This fundamentally reduces the creation of second virtual screens, thus reducing the number of second virtual screens that the system needs to manage and maintain to just one. This significantly reduces the amount of critical graphics system resources such as video memory and graphics cache consumed by the electronic device during projection, thereby significantly reducing the risk of excessive hardware load caused by excessive resource contention and scheduling overhead. Furthermore, by generating a unified projection frame from the images of multiple windows within the same second virtual screen using the same compositing process, only one projection frame is needed to project it to the vehicle terminal. This simplifies the projection transmission link, eliminates the additional computational and bandwidth overhead caused by encoding and transmitting multiple independent video streams, saves computing resources, and improves resource utilization.

[0093] Optionally, the processor 710 is further configured to extract the root layer tree of each first virtual screen; preprocess each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, so that the preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the mapping relationship; render and synthesize the root layer tree of each preprocessed first virtual screen to obtain a projection frame; wherein, the preprocessing of the first virtual screen includes at least one of the following: scaling the first virtual screen proportionally; adjusting the position of the first virtual screen.

[0094] Optionally, the processor 710 is further configured to acquire first size information of each first virtual screen and second size information of the second virtual screen, wherein the second size information is determined based on third size information of the display screen of the vehicle terminal; and determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen according to the first size information of each first virtual screen and the second size information of the second virtual screen.

[0095] Optionally, the processor 710 is further configured to call the mounting interface corresponding to the second virtual screen; based on the mounting interface, register each first virtual screen to the first virtual screen mounting container corresponding to the second virtual screen, so as to bind each first virtual screen to the second virtual screen.

[0096] Optionally, the processor 710 is further configured to, upon receiving a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal, create an encoder corresponding to the second virtual screen; and, after obtaining the projection frame, draw the projection frame on the input surface of the encoder; and obtain the coordinate information of each window drawn on the input surface of the encoder on the input surface of the encoder; synchronize the coordinate information corresponding to each window to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window in the corresponding area of ​​the display screen of the vehicle terminal.

[0097] It should be understood that, in this embodiment, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0098] The memory 709 can be used to store software programs and various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0099] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0100] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described screen projection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0101] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0102] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described screen projection method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0103] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0104] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the screen projection method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0105] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0107] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A screen projection method, characterized in that, The method includes: Upon receiving a request from the vehicle terminal to project N windows of the electronic device onto the vehicle terminal, the electronic device creates N first virtual screens and one second virtual screen; one first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; For each first virtual screen, the screen data of the window corresponding to the first virtual screen is laid out and drawn through the first virtual screen to obtain the root layer tree of each first virtual screen; The root layer tree of each first virtual screen is rendered and synthesized through the second virtual screen to obtain a projection frame, which contains the screen data of N windows. The screen projection frame is projected onto the vehicle terminal.

2. The method according to claim 1, characterized in that, The process of rendering and compositing the root layer tree of each first virtual screen to obtain the projection frame includes: Extract the root layer tree for each first virtual screen; Based on the mapping relationship between the display areas of each first virtual screen and the second virtual screen, each first virtual screen is preprocessed so that each preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the mapping relationship; The root layer tree of each preprocessed first virtual screen is rendered and composited to obtain the projection frame; The preprocessing of the first virtual screen includes at least one of the following: The first virtual screen is scaled proportionally. The position of the first virtual screen is adjusted.

3. The method according to claim 2, characterized in that, Before preprocessing each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, the method further includes: Obtain the first size information of each first virtual screen and the second size information of the second virtual screen, wherein the second size information is determined based on the third size information of the display screen of the vehicle terminal; Based on the first size information of each first virtual screen and the second size information of the second virtual screen, the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen is determined.

4. The method according to claim 1, characterized in that, The method further includes: Call the mounting interface corresponding to the second virtual screen; Based on the mounting interface, each first virtual screen is registered to the first virtual screen mounting container corresponding to the second virtual screen, so as to bind each first virtual screen to the second virtual screen.

5. The method according to claim 1, characterized in that, When receiving a request from the vehicle terminal to project N windows of an electronic device onto the vehicle terminal, the method further includes: Create an encoder corresponding to the second virtual screen; After obtaining the projection frame, the method further includes: The projection frame is drawn on the input surface of the encoder; The step of projecting the screen frame to the vehicle terminal includes: Obtain the coordinate information of each window drawn on the input surface of the encoder on the input surface of the encoder; The coordinate information corresponding to each window is synchronized to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window in the corresponding area of ​​the display screen of the vehicle terminal.

6. A screen projection device, characterized in that, The device includes: A creation module is used to, upon receiving a request from an in-vehicle terminal to project N windows of an electronic device onto the in-vehicle terminal, have the electronic device create N first virtual screens and one second virtual screen; one first virtual screen corresponds to one window among the N windows, where N is a positive integer and N≥2, and all N first virtual screens are bound to the second virtual screen; The determination module is used to lay out and draw the screen data of the window corresponding to the first virtual screen for each first virtual screen, so as to obtain the root layer tree of each first virtual screen. The compositing module is used to render and composite the root layer tree of each first virtual screen through the second virtual screen to obtain a projection frame, wherein the projection frame contains the screen data of N windows; The screen projection module is used to project the screen projection frame to the vehicle terminal.

7. The apparatus according to claim 6, characterized in that, The synthesis module includes: Extraction unit, used to extract the root layer tree of each first virtual screen; The preprocessing unit is used to preprocess each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen, so that the preprocessed first virtual screen is placed on the corresponding area of ​​the second virtual screen based on the mapping relationship; The compositing unit is used to render and composite the root layer tree of each preprocessed first virtual screen to obtain the projection frame; The preprocessing of the first virtual screen includes at least one of the following: The first virtual screen is scaled proportionally. The position of the first virtual screen is adjusted.

8. The apparatus according to claim 7, characterized in that, The device further includes: The first acquisition module is used to acquire first size information of each first virtual screen and second size information of the second virtual screen before preprocessing each first virtual screen according to the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen. The second size information is determined based on the third size information of the display screen of the vehicle terminal. The determining module is used to determine the mapping relationship of the display area of ​​each first virtual screen on the second virtual screen based on the first size information of each first virtual screen and the second size information of the second virtual screen.

9. The apparatus according to claim 6, characterized in that, The device further includes: The calling module is used to call the mounting interface corresponding to the second virtual screen; The mounting module is used to register each first virtual screen to the first virtual screen mounting container corresponding to the second virtual screen based on the mounting interface, so as to bind each first virtual screen to the second virtual screen.

10. The apparatus according to claim 6, characterized in that, When a request is received from the vehicle terminal to project N windows of an electronic device onto the vehicle terminal, the creation module is further configured to create an encoder corresponding to the second virtual screen. The device further includes: A drawing module is used to draw the projection frame on the input surface of the encoder after the projection frame is obtained; The second acquisition module is used to acquire the coordinate information of each window drawn on the input surface of the encoder after the screen projection frame is projected to the vehicle terminal. The synchronization module is used to synchronize the coordinate information corresponding to each window to the vehicle terminal, so that the vehicle terminal can crop the projection frame based on the coordinate information corresponding to each window to obtain the application screen corresponding to each window, and display the application screen corresponding to each window in the corresponding area of ​​the display screen of the vehicle terminal.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the screen projection method as described in any one of claims 1-5.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of the screen projection method as described in any one of claims 1-5.

Images