A multi-window display projection method based on container execution mechanism

Abstract

The present invention provides a multi-window different-display screen projection method based on a container execution mechanism, which relates to the field of smart terminal display technology. The method comprises constructing multiple target containers based on active components and arranging them according to a preset layout template to generate a virtual main window, obtaining a user touch screen event through the virtual main window and converting its original coordinates into target coordinates, and then sending the event to a target application window in the target container, driving the target application window to execute logical operations and generate real-time picture frames to store in a frame buffer of the target container, reading all real-time picture frames through the virtual main window, splicing them according to a preset layout template to obtain a synthetic projection screen, encoding the synthetic projection screen and transmitting it to an external screen projection end through a projection protocol, thereby improving development efficiency, realizing multi-window different-display screen projection, enriching projection diversity, and flexibly arranging a variety of containers in the main virtual window to adapt to different large-screen scenarios and meet diverse projection needs.

Description

Technical Field

[0001] The present invention relates to the field of smart terminal display technology, and in particular to a multi-window different-display projection method based on a container execution mechanism. Background Art

[0002] With the development of screen projection technology, asynchronous screen projection has been widely used. It can use the computing power of the host smart device to run programs and project the running results to another terminal in the form of a virtual screen.

[0003] However, the current asynchronous screen projection technology has obvious defects. On the one hand, various application execution windows can only use the virtual screen exclusively, which means that only the execution screen or results of a single application can be presented during asynchronous screen projection, which seriously hinders the multi-tasking screen projection process and reduces the screen projection effect. On the other hand, asynchronous screen projection based on computing power adopts a single virtual window mode, which only supports the projection of the execution results of a single process window and cannot handle cross-window execution elements. This not only limits the diversity and richness of large-screen projection content, but also makes it impossible for functional modules based on asynchronous computing power to be directly executed in the virtual window. At this time, if the virtual window is forced to be projected with a multi-window mechanism, a lot of customized development is required, and the functional modules are synthesized through sub-windows in a coded manner to simulate virtual windows. This method requires a large amount of development and has a high implementation cost.

[0004] Therefore, it is necessary to provide a multi-window display projection method based on a container execution mechanism to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a multi-window different-display screen projection method based on the container execution mechanism, which is used to solve the problems that various application execution windows monopolize the virtual screen, multi-task screen projection is limited, and the effect is poor; a single virtual window can only project the results of a single process and cannot process cross-window elements, which limits the richness of large-screen projection; and the development of multi-window projection is large and the cost is high.

[0006] The present invention provides a multi-window different-display screen projection method based on a container execution mechanism, comprising:

[0007] Constructing multiple target containers based on the active component, and arranging all the target containers according to a preset layout template to generate a virtual main window;

[0008] Acquiring a user touch screen event through the virtual main window, converting original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container;

[0009] Driving the target application window to perform logical operations and generate real-time picture frames to store in the frame buffer of the target container;

[0010] Reading the real-time picture frames in the frame buffers of all the target containers through the virtual main window, and splicing all the real-time picture frames according to the preset layout template to obtain a synthetic projection picture;

[0011] The synthesized projection screen is encoded and transmitted to the external screen projection end through the projection protocol.

[0012] Preferably, the step of constructing multiple target containers based on the active component and arranging all the target containers according to a preset layout template to generate a virtual main window specifically includes:

[0013] constructing a plurality of target containers based on the active component, setting structural parameters and container functions of the target containers, and loading the target application window in the target containers;

[0014] Obtaining the preset layout template, and arranging all the target containers in sequence according to the preset layout template to generate the virtual main window;

[0015] The structural parameters include length, width and coordinate system; and the container function is a coordinate mapping function.

[0016] Preferably, acquiring a user touch screen event through the virtual main window, converting original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container specifically includes:

[0017] Monitoring the touch screen operation of the user terminal through the virtual main window and obtaining the corresponding user touch screen event;

[0018] Acquire the original coordinates of the user touch screen event in the virtual main window, and map the original coordinates to the coordinate system corresponding to the target container to obtain the target coordinates;

[0019] The user touch screen event is sent to the target application window in the target container.

[0020] Preferably, in the virtual main window, a display mapping relationship between the target containers is established based on an affine transformation matrix, and the expression of the affine transformation matrix is ​​as follows:

[0021] Where M represents the affine transformation matrix; Indicates the ratio of the width of the target container to the width of the virtual main window in the x-axis direction; Indicates the ratio of the height of the target container to the height of the virtual main window in the y-axis direction; Indicates the width of the target container; Indicates the width of the virtual main window; Indicates the height of the target container; Indicates the height of the virtual main window; Indicates the coordinate offset of the target container in the x-axis direction of the virtual main window; Indicates the coordinate offset of the target container in the y-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the x-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the y-axis direction of the virtual main window.

[0022] Preferably, based on the inverse matrix algorithm, the target coordinates of the user touch screen event are determined as follows:

[0023] Where, Indicates the target coordinates of the user touch screen event; M represents the affine transformation matrix; Represents the inverse transformation matrix of the affine transformation matrix; Indicates the original coordinates of the user touch screen event; Indicates the touch event correction factor.

[0024] Preferably, an event distribution mechanism of the virtual main window is obtained, and all the real-time picture frames are synchronized based on the event distribution mechanism, and the degree of synchronization of the real-time picture frames is as follows:

[0025] Where, Indicates the degree of synchronization of the real-time picture frame corresponding to the i-th target container; represents the attenuation coefficient; Indicates the weight coefficient corresponding to the priority of the i-th target container; Indicates the distance between the target coordinates of the user touch event and the center coordinates of the i-th target container; Indicates the weight coefficient corresponding to the priority of the j-th target container; Represents the distance between the target coordinates of the user touch screen event and the center coordinates of the j-th target container; N represents the number of target containers; e represents a natural constant.

[0026] Preferably, the composite projection screen is at coordinates The pixel values ​​at are as follows:

[0027] Where, Indicates that the composite projection screen is at coordinates The pixel value at ; Q represents the number of real-time picture frames; Indicates the transparency of the kth real-time picture frame; Indicates that the kth real-time picture frame is at coordinate The pixel value at ; Indicates the dynamic offset coordinates of the kth real-time picture frame; Indicates that the kth real-time picture frame is at coordinate The mask matrix of the overlapping area of ​​the picture.

[0028] Preferably, the transmission rate of the encoded composite projection image is as follows:

[0029] Where R represents the transmission rate of the encoded composite projection image; represents the transmission adjustment coefficient; Q represents the number of real-time picture frames; represents the picture complexity factor of the kth real-time picture frame; B represents the compression ratio of the encoded synthetic projection picture; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

[0030] Preferably, after the external screen projection terminal receives the encoded composite projection screen image, it decodes the composite projection screen image and optimizes the image quality.

[0031] Compared with related technologies, the multi-window display projection method based on the container execution mechanism provided by the present invention has the following beneficial effects:

[0032] The present invention can construct multiple target containers based on active components, and arrange all target containers according to a preset layout template to generate a virtual main window; obtain user touch screen events through the virtual main window, convert the original coordinates of the user touch screen events into target coordinates, and send the user touch screen events to the target application window in the target container; drive the target application window to perform logical operations, and generate real-time picture frames to store in the frame buffer of the target container; read the real-time picture frames in the frame buffer of all target containers through the virtual main window, and splice all real-time picture frames according to the preset layout template to obtain a synthetic projection picture; encode the synthetic projection picture, and transmit it to the external screen projection end through the projection protocol, so that the container development technology can be used to improve development efficiency, realize multi-window different display projection, and improve the shortcomings of single application projection, enrich the diversity of projection, and at the same time, multiple containers can be flexibly arranged in the main virtual window to adapt to different large-screen scenes and meet diverse projection needs.

[0033] The present invention can utilize container development technology to easily implement a multi-window projection mechanism, significantly reduce the amount of code porting, and improve development efficiency to 75%; the method of the present invention can improve the limitations of single application projection and enrich the diversity of large-screen scene projection; the present invention can realize the coexistence of multiple containers in the main virtual window, and by adjusting the container layout, adapt to the projection scenes of different large screens, thereby improving the flexibility of projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a multi-window different display projection method based on a container execution mechanism of the present invention;

[0035] Figure 2 Schematic diagram of the structural parameters of the target container of the present invention;

[0036] Figure 3 Schematic diagram of the positional relationship between the virtual main window and the target container of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, a multi-window different display projection method based on a container execution mechanism includes:

[0039] Constructing multiple target containers based on the active component, and arranging all the target containers according to a preset layout template to generate a virtual main window;

[0040] Among them, active components are the basic units that constitute the user interface of an application, which can respond to user interactive operations such as clicks and slides. Preset layout templates refer to templates with pre-set window arrangement rules and styles, which determine the position, size and display order of each target container in the virtual main window. Preset layout templates can be left-right split-screen layouts, top-bottom split-screen layouts, etc. The virtual main window refers to a logical window used to integrate and manage multiple target containers. It can coordinate the display and interaction of each target container and has the function of obtaining user touch screen events. That is, when the user performs a touch operation on the device screen, the virtual main window can accurately capture this event.

[0041] Acquiring a user touch screen event through the virtual main window, converting original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container;

[0042] It should be noted that touch screen events refer to user touch screen actions, such as clicks, swipes, and long presses. Original coordinates refer to the coordinates of the touch point at the time of the touch. Target coordinates refer to the corresponding converted coordinates. A target container refers to the container used to hold different application windows. The target application window refers to the application interface within the target container that receives and processes user actions.

[0043] It is understood that after receiving a touch event, the virtual main window can perform a coordinate conversion operation. Specifically, it converts the original coordinates of the user touch event to target coordinates based on the location and layout information of the target container. After completing the coordinate conversion, the virtual main window can send the user touch event, including the target coordinate information, to the target application window in the target container.

[0044] Driving the target application window to perform logical operations and generate real-time picture frames to store in the frame buffer of the target container;

[0045] In practical applications, the real-time image frame refers to the real-time image of the application interface. The frame buffer refers to the area in the target container used to store the image frame.

[0046] Specifically, the target application window can be driven to perform logical operations according to a pre-set program. These operations may include data processing, interface updates, and more. After the logical operations are completed, the target application window can generate real-time image frames corresponding to the current state. These frames can then be stored in the target container's frame buffer.

[0047] Reading the real-time picture frames in the frame buffers of all the target containers through the virtual main window, and splicing all the real-time picture frames according to the preset layout template to obtain a synthetic projection picture;

[0048] A preset layout template is a pre-set layout rule that specifies the position, size, and other parameters of each real-time image frame in the final projection. A composite projection image is a composite image created by stitching together real-time image frames from different target containers.

[0049] In practice, the virtual main window can sequentially read the real-time frames stored in the frame buffers of all target containers. These frames can then be spliced ​​together according to a pre-defined layout template, integrating the real-time frames originally scattered across different target containers into a complete, composite projection screen that meets specific layout requirements for subsequent encoding, transmission, and external display.

[0050] The synthesized projection screen is encoded and transmitted to the external screen projection end through the projection protocol.

[0051] It is understood that the external screen projection terminal refers to the external device used to display the projected content. The screen projection protocol refers to the protocol used to specify the format, sequence, and rules of data transmission during the screen projection process.

[0052] First, the spliced ​​composite projection screen can be encoded, converting the original data of the composite projection screen into a format more suitable for transmission, thereby reducing the data volume and improving data transmission efficiency. After the encoding is completed, the encoded image data can be transmitted to the external screen projection end based on the projection transmission technology.

[0053] In a specific implementation process, the process of constructing multiple target containers based on the active component and arranging all the target containers according to a preset layout template to generate a virtual main window specifically includes:

[0054] constructing a plurality of target containers based on the active component, setting structural parameters and container functions of the target containers, and loading the target application window in the target containers;

[0055] Obtaining the preset layout template, and arranging all the target containers in sequence according to the preset layout template to generate the virtual main window;

[0056] The structural parameters include length, width and coordinate system; and the container function is a coordinate mapping function.

[0057] In actual applications, you can use active components to build multiple target containers and set the structural parameters and container functions of the target containers. Among them, the structural parameters include length, width and coordinate system. The length and width define the spatial size of the target container, and the coordinate system provides a precise reference for the positioning of elements in the target container. Figure 2 As shown, the structural parameters of container 1 include length, width, and a rectangular coordinate system. The width corresponds to the x-axis of the rectangular coordinate system, and the length corresponds to the y-axis of the rectangular coordinate system. The container function is a coordinate mapping function that accurately converts externally input coordinate information into coordinates applicable within the target container. Furthermore, the target application window can be loaded into the target container.

[0058] It is understandable that all target containers can be arranged in sequence according to the preset layout template to eventually generate a virtual main window. The virtual main window integrates each target container and the application window inside it, which helps to realize the subsequent multi-window display projection. Figure 3 As shown, the virtual main window may include three target containers, namely container 1, container 2 and container 3, and container 1, container 2 and container 3 are arranged in sequence from left to right.

[0059] The acquiring of a user touch screen event through the virtual main window, converting original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container specifically includes:

[0060] Monitoring the touch screen operation of the user terminal through the virtual main window and obtaining the corresponding user touch screen event;

[0061] Acquire the original coordinates of the user touch screen event in the virtual main window, and map the original coordinates to the coordinate system corresponding to the target container to obtain the target coordinates;

[0062] The user touch screen event is sent to the target application window in the target container.

[0063] The user terminal refers to the user's corresponding terminal. First, the virtual main window can monitor the user's touch screen operations in real time. Once the user performs a touch action on the terminal device screen, such as clicking or sliding, the virtual main window can quickly respond and accurately obtain the corresponding user touch screen event, thereby capturing the user's action in a timely manner.

[0064] Furthermore, the original coordinates of the user touch event within the virtual main window can be obtained. To ensure that the target container and its internal application window can correctly handle user operations, the original coordinates must be mapped to the target container's coordinate system. A coordinate conversion algorithm, combined with information such as the target container's position and size within the virtual main window, calculates the target coordinates. This conversion process ensures that user operations are accurately mapped to specific locations within the target container.

[0065] Finally, the user touch screen event containing the target coordinate information can be sent to the target application window in the target container. The target application window can then perform corresponding processing according to the preset program logic, such as interface updates and data interaction, thereby achieving smooth interaction between the user and the application.

[0066] In the virtual main window, a display mapping relationship between the target containers is established based on an affine transformation matrix. The expression of the affine transformation matrix is ​​as follows:

[0067] Where M represents the affine transformation matrix; Indicates the ratio of the width of the target container to the width of the virtual main window in the x-axis direction; Indicates the ratio of the height of the target container to the height of the virtual main window in the y-axis direction; Indicates the width of the target container; Indicates the width of the virtual main window; Indicates the height of the target container; Indicates the height of the virtual main window; Indicates the coordinate offset of the target container in the x-axis direction of the virtual main window; Indicates the coordinate offset of the target container in the y-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the x-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the y-axis direction of the virtual main window.

[0068] In order to achieve orderly coordination and accurate presentation of each target container at the display level, the display mapping relationship between the target containers can be accurately established in the virtual main window based on the affine transformation matrix to ensure that the display effect of each target container in the virtual main window meets expectations.

[0069] Based on the inverse matrix algorithm, the target coordinates of the user touch screen event are determined as follows:

[0070] Where, Indicates the target coordinates of the user touch screen event; M represents the affine transformation matrix; Represents the inverse transformation matrix of the affine transformation matrix; Indicates the original coordinates of the user touch screen event; Indicates the touch event correction factor.

[0071] Among them, the inverse matrix algorithm refers to an algorithm that reversely derives coordinate information based on matrix relationships.

[0072] In practical applications, the affine transformation matrix can reflect the transformation rule from the virtual main window to the target container. By performing an inverse operation on the affine transformation matrix, its inverse transformation matrix can be obtained.

[0073] Since the original coordinates of the user touch screen event are based on the virtual main window, in order to convert them to the coordinate system of the target container, it is necessary to consider the touch event correction factor. This factor is used to correct coordinate deviations that may be caused by factors such as the device and environment.

[0074] Finally, the target coordinates of the user touch screen event in the target container can be accurately obtained by combining the inverse transformation matrix, the original coordinates, and the touch event correction factor.

[0075] The event distribution mechanism of the virtual main window is obtained, and all the real-time picture frames are synchronized based on the event distribution mechanism. The degree of synchronization of the real-time picture frames is as follows:

[0076] Where, Indicates the degree of synchronization of the real-time picture frame corresponding to the i-th target container; represents the attenuation coefficient; Indicates the weight coefficient corresponding to the priority of the i-th target container; Indicates the distance between the target coordinates of the user touch event and the center coordinates of the i-th target container; Indicates the weight coefficient corresponding to the priority of the j-th target container; Represents the distance between the target coordinates of the user touch screen event and the center coordinates of the j-th target container; N represents the number of target containers; e represents a natural constant.

[0077] All real-time image frames can be synchronized based on the event distribution mechanism. The degree of synchronization of real-time image frames may be affected by various factors, including the attenuation coefficient, the target container priority weight coefficient, and the distance between the target coordinates of the user touch event and the center coordinates of the target container.

[0078] It should be noted that the attenuation coefficient refers to the degree of attenuation caused by external changes. A corresponding weight coefficient is set for each target container's priority, and the higher the priority of the target container, the larger the corresponding weight coefficient. In addition, the distance between the target coordinates of the user's touch event and the center coordinates of each target container also affects the degree of synchronization. Combining these factors, the degree of synchronization of the real-time image frame corresponding to each target container can be calculated, thereby enabling precise control of the real-time image frame synchronization process.

[0079] The composite projection screen is at coordinates The pixel values ​​at are as follows:

[0080] Where, Indicates that the composite projection screen is at coordinates The pixel value at ; Q represents the number of real-time picture frames; Indicates the transparency of the kth real-time picture frame; Indicates that the kth real-time picture frame is at coordinate The pixel value at ; Indicates the dynamic offset coordinates of the kth real-time picture frame; Indicates that the kth real-time picture frame is at coordinate The mask matrix of the overlapping area of ​​the picture.

[0081] The pixel value of the composite projection at a specific coordinate depends on several factors, including the number of live frames, the degree of transparency, the dynamic offset coordinate, and the mask matrix of the overlapped area at that coordinate. By combining these factors, we can accurately determine the pixel value of the composite projection at that coordinate.

[0082] The transmission rate of the encoded composite projection image is as follows:

[0083] Where R represents the transmission rate of the encoded composite projection image; represents the transmission adjustment coefficient; Q represents the number of real-time picture frames; represents the picture complexity factor of the kth real-time picture frame; B represents the compression ratio of the encoded synthetic projection picture; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

[0084] In practical applications, the transmission rate of the encoded composite projection image is determined by the transmission adjustment coefficient, the number of real-time image frames, the complexity factor of each frame, the image compression ratio, and the signal-to-noise ratio of the transmission channel of the projection protocol.

[0085] After the external screen projection end receives the encoded composite projection screen image, it decodes the composite projection screen image and optimizes the image quality.

[0086] In addition, when the external screen projection end successfully receives the encoded composite projection image, it can decode it and restore the encoded image data to the original displayable image information.

[0087] Furthermore, in order to improve the visual effect of the composite projection screen, the image quality can be optimized, including enhancing colors, sharpening images, and other operations to present a better quality picture.

[0088] Through the introduction of the above embodiments, the present invention can construct multiple target containers based on active components through a multi-window different-display screen projection method based on a container execution mechanism, and arrange all target containers according to a preset layout template to generate a virtual main window; obtain user touch screen events through the virtual main window, convert the original coordinates of the user touch screen event into target coordinates, and send the user touch screen event to the target application window in the target container; drive the target application window to perform logical operations, and generate real-time picture frames to store in the frame buffer of the target container; read the real-time picture frames in the frame buffer of all target containers through the virtual main window, and splice all real-time picture frames according to the preset layout template to obtain a synthetic projection picture; encode the synthetic projection picture and transmit it to the external screen projection end through the projection protocol, so that the container development technology can be used to improve development efficiency, realize multi-window different-display screen projection, and improve the shortcomings of single application projection, enrich the diversity of projection, and at the same time, multiple containers can be flexibly arranged in the main virtual window to adapt to different large-screen scenes and meet diverse projection needs.

[0089] The present invention can utilize container development technology to easily implement a multi-window projection mechanism, significantly reduce the amount of code porting, and improve development efficiency to 75%; the method of the present invention can improve the limitations of single application projection and enrich the diversity of large-screen scene projection; the present invention can realize the coexistence of multiple containers in the main virtual window, and by adjusting the container layout, adapt to the projection scenes of different large screens, thereby improving the flexibility of projection.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0092] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. A multi-window display projection method based on a container execution mechanism, characterized in that: include: Multiple target containers are constructed based on the active component, and all the target containers are arranged according to a preset layout template to generate a virtual main window. The active component is the basic unit of the user interface of the application and is used to respond to user interactive operations, including clicks and slides. The preset layout template includes window arrangement rules and styles, which are used to determine the position, size, and display order of each target container in the virtual main window. The preset layout template is a left-right split-screen layout or a top-bottom split-screen layout. The virtual main window is used to integrate and manage the logical windows of multiple target containers, coordinate the display and interaction of each target container, and has the function of obtaining user touch screen events. That is, when the user performs a touch operation on the device screen, the virtual main window is used to capture the user touch event. Acquire a user touch screen event through the virtual main window, perform a coordinate conversion operation, convert the original coordinates of the user touch screen event into target coordinates based on the position and layout information of the target container, and send the user touch screen event containing the target coordinates to the target application window in the target container, wherein the user touch screen event includes click, slide and long press; the original coordinates are the position coordinates of the touch point when the user touches the screen; the target coordinates are the position coordinates after the original coordinates are converted; the target container is used to carry different target application windows; the target application window is the application interface in the target container, which is used to receive and process the user touch screen event; Driving the target application window to perform logical operations according to a preset program, including data processing and interface update operations, and generating a real-time image frame corresponding to the current state, and storing the real-time image frame in a frame buffer of the target container, wherein the real-time image frame is a real-time image of the application interface; the frame buffer is an area in the target container for storing real-time image frames; The real-time picture frames in the frame buffers of all the target containers are sequentially read through the virtual main window, and the real-time picture frames scattered in different target containers are spliced ​​according to the preset layout template to obtain a complete composite projection screen that meets the layout requirements, wherein the preset layout template is a screen layout rule, which is used to specify the position and size parameters of each real-time picture frame in the composite projection screen; the composite projection screen is a comprehensive screen obtained by splicing the real-time picture frames of different target containers; The composite projection screen is encoded, that is, the original data of the composite projection screen is converted into a format suitable for transmission, and based on the projection transmission technology, the encoded composite projection screen is transmitted to the external screen projection end through the projection protocol.

2. A multi-window different display projection method based on container execution mechanism according to claim 1, characterized in that: The step of constructing multiple target containers based on the active component and arranging all the target containers according to a preset layout template to generate a virtual main window specifically includes: constructing a plurality of target containers based on the active component, setting structural parameters and container functions of the target containers, and loading the target application window in the target containers; Obtaining the preset layout template, and arranging all the target containers in sequence according to the preset layout template to generate the virtual main window, where the virtual main window is used to integrate the target containers and the target application windows therein; Among them, the structural parameters include length, width and coordinate system, among which the length and width are used to clarify the spatial dimensions of the target container, and the coordinate system is used to provide a reference for the positioning of elements in the target container. The length direction corresponds to the y-axis direction of the coordinate system, and the width direction corresponds to the x-axis direction of the coordinate system; the container function is a coordinate mapping function, which is used to convert the coordinate information input externally into coordinates applicable inside the target container.

3. The multi-window different display projection method based on the container execution mechanism according to claim 1 is characterized in that: The acquiring of a user touch screen event through the virtual main window, converting original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container specifically includes: Monitoring the touch screen operation of the user terminal through the virtual main window and obtaining the corresponding user touch screen event; Obtaining the original coordinates of the user touch event in the virtual main window, and mapping the original coordinates to the coordinate system corresponding to the target container, and calculating the target coordinates using a coordinate conversion algorithm combined with the position and size of the target container in the virtual main window. This conversion process is used to ensure that the user touch event corresponds to a certain position within the target container; The user touch screen event is sent to the target application window in the target container, and the target application window processes the user touch screen event according to preset program logic, including interface update and data interaction.

4. The multi-window different display projection method based on the container execution mechanism according to claim 1 is characterized in that: In the virtual main window, a display mapping relationship between the target containers is established based on an affine transformation matrix. The expression of the affine transformation matrix is ​​as follows: T x =W h *δ x T y =H h *d y Where M represents the affine transformation matrix; s x Indicates the ratio of the width of the target container to the width of the virtual main window in the x-axis direction; s y Indicates the ratio of the height of the target container to the virtual main window in the y-axis direction; W c Indicates the width of the target container; W h Indicates the width of the virtual main window; H c Indicates the height of the target container; H h Indicates the height of the virtual main window; T x Indicates the coordinate offset of the target container in the x-axis direction of the virtual main window; T y Indicates the coordinate offset of the target container in the y-axis direction of the virtual main window; δ x Indicates the normalized offset ratio of the target container in the x-axis direction of the virtual main window; δ y Indicates the normalized offset ratio of the target container in the y-axis direction of the virtual main window.

5. The multi-window different display projection method based on the container execution mechanism according to claim 4 is characterized in that: Based on an inverse matrix algorithm, that is, an algorithm for reversely deducing coordinate information based on a matrix relationship, the target coordinates of the user touch screen event are determined as follows: (u ' ,v ' )=M -1 *(u,v)+∈ Where, (u ' ,v ' ) represents the target coordinates of the user touch screen event; M represents the affine transformation matrix, which is used to reflect the transformation rule from the virtual main window to the target container; M -1 Represents the inverse transformation matrix of the affine transformation matrix; (u, v) represents the original coordinates of the user touch screen event; ∈ represents the touch event correction factor, which is used to correct the coordinate deviation caused by device and environmental factors.

6. The multi-window different display projection method based on the container execution mechanism according to claim 1 is characterized in that: The event distribution mechanism of the virtual main window is obtained, and all the real-time picture frames are synchronized based on the event distribution mechanism. The degree of synchronization of the real-time picture frames is as follows: Where, P i Indicates the degree of synchronization of the real-time picture frame corresponding to the i-th target container; λ represents the attenuation coefficient; w i Indicates the weight coefficient corresponding to the priority of the i-th target container; d i represents the distance between the target coordinates of the user touch screen event and the center coordinates of the i-th target container; w j Indicates the weight coefficient corresponding to the priority of the j-th target container; d j represents the distance between the target coordinates of the user touch screen event and the center coordinates of the jth target container; N represents the number of target containers; e represents a natural constant; The degree of synchronization of real-time image frames is affected by the attenuation coefficient, the weight coefficient corresponding to the priority of the target container, and the distance between the target coordinates of the user touch screen event and the center coordinates of the target container. The attenuation coefficient is the degree of attenuation caused by external changes. A corresponding weight coefficient is set for the priority of each target container, and the higher the priority of the target container, the larger the corresponding weight coefficient.

7. The multi-window different display projection method based on the container execution mechanism according to claim 1 is characterized in that: The pixel values ​​of the composite projection image at coordinates (m, n) are as follows: Where I(m,n) represents the pixel value of the composite projection image at coordinate (m,n); Q represents the number of real-time image frames; TMCD k Indicates the transparency of the kth real-time picture frame; G k Indicates the kth real-time picture frame at coordinate (mo k ,nt k ) at the pixel value; (o k ,t k ) represents the dynamic offset coordinates of the kth real-time picture frame; MBJZ k (m,n) represents the mask matrix of the overlapping area of ​​the k-th real-time picture frame at the coordinate (m,n); The pixel value of the composite projection screen at a certain coordinate is related to the number, transparency and dynamic offset coordinates of the real-time picture frames, as well as the pixel value of the real-time picture frame at the coordinate and the mask matrix of the picture overlap area.

8. The multi-window different display projection method based on the container execution mechanism according to claim 7 is characterized in that: The transmission rate of the encoded composite projection image is as follows: Where R represents the transmission rate of the encoded composite projection image; β represents the transmission adjustment coefficient; Q represents the number of real-time image frames; A k represents the picture complexity factor of the kth real-time picture frame; B represents the compression ratio of the encoded synthetic projection picture; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

9. The multi-window different display projection method based on the container execution mechanism according to claim 1 is characterized in that: After the external screen projection end receives the encoded composite projection screen image, it decodes and optimizes the image quality of the composite projection screen image. Decoding is to restore the encoded composite projection screen image to the original displayable image. The image quality optimization includes enhancing colors and sharpening images.

Images