Multi-window different-display screen projection method based on container execution mechanism

Abstract

The invention provides a multi-window different display screen projection method based on a container execution mechanism, and relates to the technical field of intelligent terminal display, and the method comprises the steps: constructing a plurality of target containers based on an active component, and arranging the target containers according to a preset layout template to generate a virtual main window, acquiring a user touch screen event through the virtual main window, converting an original coordinate of the user touch screen event into a target coordinate, sending the target coordinate to a target application window in a target container, driving the target application window to execute logic operation, generating a real-time picture frame, storing the real-time picture frame in a frame buffer area of the target container, reading all the real-time picture frames through the virtual main window, and displaying the real-time picture frame. According to the method, the virtual windows are selected, the virtual windows are spliced according to a preset layout template to obtain a synthesized projection screen picture, and the synthesized projection screen picture is coded and transmitted to an external screen projection end through a projection screen protocol, so that the development efficiency can be improved, multi-window different-display projection screen is realized, the projection screen diversity is enriched, various containers are flexibly arranged in the main virtual window, and the development efficiency is improved. And the method is suitable for different large-screen scenes and meets diversified screen projection requirements.

Description

Technical Field

[0001] The present invention relates to the field of intelligent terminal display technology, and in particular to a multi-window different-display screen projection method based on a container execution mechanism. Background Art

[0002] With the development of screen projection technology, asynchronous screen projection has been widely used. It can use the computing power of the host smart device to run programs and project the running results to another terminal in the form of a virtual screen.

[0003] However, the current asynchronous screen projection technology has obvious defects. On the one hand, various application execution windows can only enjoy the virtual screen exclusively, which means that only the execution screen or results of a single application can be presented during asynchronous screen projection, which seriously hinders the multi-task screen projection process and reduces the screen projection effect. On the other hand, asynchronous screen projection based on computing power adopts a single virtual window mode, which only supports the projection of the execution results of a single process window and cannot handle cross-window execution elements. This not only limits the diversity and richness of large-screen projection content, but also makes it impossible for functional modules based on asynchronous computing power to be directly executed in virtual windows. At this time, if the virtual window is forced to be projected with a multi-window mechanism, a lot of customized development is required, and the functional modules are synthesized through sub-windows in a coded manner to simulate virtual windows. This method requires a large amount of development and has a high implementation cost.

[0004] Therefore, it is necessary to provide a multi-window different-display screen projection method based on a container execution mechanism to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a multi-window different-display screen projection method based on a container execution mechanism, which is used to solve the problems that various application execution windows monopolize the virtual screen, multi-task screen projection is limited, and the effect is poor; a single virtual window can only project a single process result and cannot process cross-window elements, which limits the richness of large-screen projection; and multi-window projection has large development volume and high cost.

[0006] The present invention provides a multi-window different-display screen projection method based on a container execution mechanism, comprising: Constructing multiple target containers based on the active component, and arranging all the target containers according to a preset layout template to generate a virtual main window; Acquire a user touch screen event through the virtual main window, convert the original coordinates of the user touch screen event into target coordinates, and send the user touch screen event to the target application window in the target container; Driving the target application window to perform logic operations and generate real-time picture frames to store in the frame buffer of the target container; Read the real - time picture frames in the frame buffers of all the target containers through the virtual main window, and splice all the real - time picture frames according to the preset layout template to obtain a composite projection screen picture; Encode the composite projection screen picture and transmit it to an external screen projection terminal through a projection protocol.

[0007] Preferably, constructing a plurality of target containers based on active components and arranging all the target containers according to a preset layout template to generate a virtual main window specifically includes: Construct a plurality of the target containers based on the active components, set the structural parameters and container functions of the target containers, and load the target application windows in the target containers; Obtain the preset layout template, and arrange all the target containers in sequence according to the preset layout template to generate the virtual main window; Among them, the structural parameters include length, width, and coordinate system; the container function is a coordinate mapping function.

[0008] Preferably, obtaining a user touch - screen event through the virtual main window, converting the original coordinates of the user touch - screen event into target coordinates, and sending the user touch - screen event to the target application window in the target container specifically includes: Monitor the touch - screen operation of the user end through the virtual main window and obtain the corresponding user touch - screen event; Obtain the original coordinates of the user touch - screen event in the virtual main window, and map the original coordinates to the coordinate system corresponding to the target container to obtain the target coordinates; Send the user touch - screen event to the target application window in the target container.

[0009] Preferably, in the virtual main window, establish a display mapping relationship between the target containers based on an affine transformation matrix, and the expression of the affine transformation matrix is as follows: In the formula, M represents the affine transformation matrix; represents the ratio of the width of the target container to the width of the virtual main window in the x - axis direction; represents the ratio of the height of the target container to the height of the virtual main window in the y - axis direction; represents the width of the target container; represents the width of the virtual main window; represents the height of the target container; represents the height of the virtual main window; represents the coordinate offset of the target container in the x - axis direction of the virtual main window; Indicates the coordinate offset of the target container in the y-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the x-axis direction of the virtual main window; Indicates the normalized offset ratio of the target container in the y-axis direction of the virtual main window.

[0010] Preferably, based on the inverse matrix algorithm, the target coordinates of the user touch event are determined as follows: In the formula, Indicates the target coordinates of the user touch event; M represents the affine transformation matrix; Indicates the inverse transformation matrix of the affine transformation matrix; Indicates the original coordinates of the user touch event; Indicates the touch event correction factor.

[0011] Preferably, obtain the event distribution mechanism of the virtual main window, and synchronize all the real-time picture frames based on the event distribution mechanism. The synchronization degree of the real-time picture frames is as follows: In the formula, Indicates the synchronization degree of the real-time picture frame corresponding to the i-th target container; Indicates the attenuation coefficient; Indicates the weight coefficient corresponding to the priority of the i-th target container; Indicates the distance between the target coordinates of the user touch event and the center coordinates of the i-th target container; Indicates the weight coefficient corresponding to the priority of the j-th target container; Indicates the distance between the target coordinates of the user touch event and the center coordinates of the j-th target container; N represents the number of target containers; e represents the natural constant.

[0012] Preferably, the pixel value of the composite projection screen at the coordinate is as follows: In the formula, Indicates the pixel value of the composite projection screen at the coordinate ; Q represents the number of real-time picture frames; Indicates the transparency of the k-th real-time picture frame; Indicates the k-th real-time picture frame at the coordinate The pixel value at; Indicates the dynamic offset coordinates of the k-th real-time picture frame; Indicates the k-th real-time picture frame at the coordinate The mask matrix of the picture overlapping area at;

[0013] Preferably, the transmission rate of the encoded synthetic projection screen image is as follows: In the formula, R represents the transmission rate of the encoded synthetic projection screen image; represents the transmission adjustment coefficient; Q represents the number of real-time video frames; represents the video complexity factor of the k-th real-time video frame; B represents the compression ratio of the encoded synthetic projection screen image; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

[0014] Preferably, after receiving the encoded synthetic projection screen image, the external screen projection terminal decodes and optimizes the image quality of the synthetic projection screen image.

[0015] Compared with the related art, a multi-window different display projection method based on a container execution mechanism provided by the present invention has the following beneficial effects: The present invention can construct multiple target containers based on active components, and arrange all target containers according to a preset layout template to generate a virtual main window; obtain a user touch event through the virtual main window, convert the original coordinates of the user touch event into target coordinates, and send the user touch event to the target application window in the target container; drive the target application window to execute logical operations, and generate real-time video frames and store them in the frame buffer of the target container; read the real-time video frames in the frame buffers of all target containers through the virtual main window, and splice all the real-time video frames according to a preset layout template to obtain a synthetic projection screen image; encode the synthetic projection screen image, and transmit it to the external screen projection terminal through a projection protocol, so as to improve the development efficiency by using container development technology, realize multi-window different display projection, and improve the deficiencies of single-application single projection, enrich the diversity of projection, and at the same time, various containers can be flexibly arranged in the main virtual window to adapt to different large-screen scenarios and meet diverse projection requirements.

[0016] The present invention can utilize container development technology to easily implement a multi-window projection mechanism, greatly reduce the code transplantation volume, and improve the development efficiency to 75%; the method of the present invention can improve the limitations of single-application single projection and enrich the diversity of large-screen scenario projection; the present invention can realize the coexistence of multiple containers in the main virtual window, and adapt to different large-screen projection scenarios by adjusting the container layout, improving the flexibility of projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a multi-window different display projection method based on a container execution mechanism of the present invention; Figure 2 is a schematic diagram of the structural parameters of the target container of the present invention; Figure 3 is a schematic diagram of the positional relationship between the virtual main window and the target container of the present invention. Specific implementation mode

[0018] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0019] As Figure 1 shown, a multi-window heterogeneous display screen mirroring method based on a container execution mechanism includes: Constructing a plurality of target containers based on active components, and arranging all the target containers according to a preset layout template to generate a virtual main window; Among them, an active component is a basic unit that constitutes the user interface of an application program, and it can respond to user interaction operations such as clicks and swipes. The preset layout template refers to a template that pre-sets window arrangement rules and styles, which determines the positions, sizes, and display orders of each target container in the virtual main window. The preset layout template can be a left-right split screen layout, an up-down split screen layout, etc. The virtual main window refers to a logical window used to integrate and manage multiple target containers, which can coordinate the display and interaction of each target container and has the function of obtaining user touch screen events, that is, when the user performs a touch operation on the device screen, the virtual main window can accurately capture this event.

[0020] Obtaining a user touch screen event through the virtual main window, converting the original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to the target application window in the target container; It should be noted that the touch screen event refers to an operation event when the user touches the screen, such as clicks, swipes, long presses, etc. The original coordinates refer to the position coordinates of the touch point when the user touches the screen. The target coordinates are the corresponding converted position coordinates. The target container refers to a carrier for carrying different application windows. The target application window refers to the application interface in the target container that receives and processes user operations.

[0021] It can be understood that after the virtual main window obtains the touch screen event, it can perform a coordinate conversion operation, that is, according to the position and layout information of the target container, convert the original coordinates when the user touch screen event occurs into target coordinates. After completing the coordinate conversion, the virtual main window can send the user touch screen event containing the target coordinate information to the target application window in the target container.

[0022] Driving the target application window to execute logical operations, and generating a real-time picture frame and storing it in the frame buffer of the target container; In practical applications, the real-time picture frame refers to the real-time picture of the application interface. The frame buffer refers to the area in the target container used to store the picture frame.

[0023] Specifically, the target application window can be driven to perform logical operations according to a preset program. These operations may cover data processing, interface update, etc. After the logical operations are completed, the target application window can generate real-time picture frames corresponding to the current state. Then, these picture frames can be stored in the frame buffer of the target container.

[0024] Read all the real-time picture frames in the frame buffers of all the target containers through the virtual main window, and splice all the real-time picture frames according to the preset layout template to obtain a composite projection screen picture; Among them, the preset layout template refers to the picture layout rules set in advance, which stipulate parameters such as the position and size of each real-time picture frame in the final projection screen picture. The composite projection screen picture refers to the comprehensive picture obtained by splicing the real-time picture frames of different target containers.

[0025] In practical applications, through the virtual main window, the real-time picture frames stored in the frame buffers of all target containers can be read in sequence. Then, according to the preset layout template, these picture frames can be spliced and processed to integrate the real-time picture frames originally scattered in different target containers into a complete composite projection screen picture that meets specific layout requirements for subsequent encoding transmission and external display.

[0026] Encode the composite projection screen picture and transmit it to the external screen projection end through the projection protocol.

[0027] It can be understood that the external screen projection end refers to an external device used to display the projection content. The projection protocol refers to the protocol used to stipulate the data transmission format, sequence, and rules during the projection process.

[0028] First, the spliced composite projection screen picture can be encoded, and the original data of the composite projection screen picture can be converted into a format more suitable for transmission to reduce the data volume and improve the data transmission efficiency. After encoding, based on the projection transmission technology, the encoded picture data can be transmitted to the external screen projection end.

[0029] In the specific implementation process, constructing multiple target containers based on the active components and arranging all the target containers according to the preset layout template to generate a virtual main window specifically includes: Construct multiple target containers based on the active components, set the structural parameters and container functions of the target containers, and load the target application window in the target containers; Obtain the preset layout template, and arrange all the target containers in sequence according to the preset layout template to generate the virtual main window; Among them, the structural parameters include length, width, and coordinate system; the container function is the coordinate mapping function.

[0030] In practical applications, multiple target containers can be constructed using active components, and the structural parameters and container functions of the target containers can be set. Among them, the structural parameters cover length, width, and coordinate system. The length and width define the spatial dimensions of the target container, and the coordinate system provides an accurate reference for the positioning of elements within the target container. As Figure 2 shown, the structural parameters of Container 1 include length, width, and a rectangular coordinate system in the plane, and the width direction corresponds to the x-axis direction of the rectangular coordinate system in the plane, and the length direction corresponds to the y-axis direction of the rectangular coordinate system in the plane. The container function is the coordinate mapping function, which can accurately convert the externally input coordinate information into coordinates applicable inside the target container. In addition, a target application window can be loaded into the target container.

[0031] It can be understood that all target containers can be arranged in sequence according to a preset layout template to finally generate a virtual main window. The virtual main window integrates each target container and the application windows inside it, which helps to realize multi-window different display screen projection subsequently. As Figure 3 shown, the virtual main window can include three target containers, namely Container 1, Container 2, and Container 3, and Container 1, Container 2, and Container 3 are arranged in sequence from left to right.

[0032] Obtaining a user touch event through the virtual main window, converting the original coordinates of the user touch event into target coordinates, and sending the user touch event to the target application window in the target container specifically includes: Monitoring the touch operation of the user end through the virtual main window and obtaining the corresponding user touch event; Obtaining the original coordinates of the user touch event in the virtual main window and mapping the original coordinates to the coordinate system corresponding to the target container to obtain the target coordinates; Sending the user touch event to the target application window in the target container.

[0033] Among them, the user end refers to the terminal corresponding to the user. First, the touch operation of the user can be monitored in real time through the virtual main window. Once the user performs a touch action on the screen of the terminal device, such as clicking, swiping, etc., the virtual main window can quickly respond and accurately obtain the corresponding user touch event, and then the user operation can be captured in time.

[0034] Furthermore, the original coordinates of the user's touch screen event in the virtual main window can be obtained. To enable the target container and the application windows within it to correctly handle user operations, the original coordinates need to be mapped to the coordinate system corresponding to the target container. Through a coordinate conversion algorithm, combined with information such as the position and size of the target container in the virtual main window, the target coordinates are calculated. This conversion process ensures that user operations can accurately correspond to specific positions within the target container.

[0035] Finally, the user touch screen event containing the target coordinate information can be sent to the target application window in the target container. The target application window can perform corresponding processing according to the preset program logic, such as interface update, data interaction, etc., so as to achieve smooth interaction between the user and the application.

[0036] In the virtual main window, a display mapping relationship between the target containers is established based on an affine transformation matrix, and the expression of the affine transformation matrix is as follows: In the formula, M represents the affine transformation matrix; represents the ratio of the width of the target container to the width of the virtual main window in the x-axis direction; represents the ratio of the height of the target container to the height of the virtual main window in the y-axis direction; represents the width of the target container; represents the width of the virtual main window; represents the height of the target container; represents the height of the virtual main window; represents the coordinate offset of the target container in the x-axis direction in the virtual main window; represents the coordinate offset of the target container in the y-axis direction in the virtual main window; represents the normalized offset ratio of the target container in the x-axis direction in the virtual main window; represents the normalized offset ratio of the target container in the y-axis direction in the virtual main window.

[0037] To achieve the orderly coordination and precise presentation of each target container at the display level, a display mapping relationship between the target containers can be accurately established in the virtual main window based on the affine transformation matrix to ensure that the display effects of each target container in the virtual main window meet expectations.

[0038] Based on the inverse matrix algorithm, the target coordinates of the user touch screen event are determined as follows: In the formula, represents the target coordinates of the user touch screen event; M represents the affine transformation matrix; represents the inverse transformation matrix of the affine transformation matrix; Represents the original coordinates of the user's touch screen event; Represents the touch event correction factor.

[0039] Among them, the inverse matrix algorithm refers to an algorithm that reversely derives coordinate information based on matrix relationships.

[0040] In practical applications, the affine transformation matrix can reflect the transformation rules from the virtual main window to the target container. By performing an inverse operation on the affine transformation matrix, its inverse transformation matrix can be obtained.

[0041] Since the original coordinates initially generated by the user's touch screen event are based on the virtual main window, therefore, in order to convert them to the coordinate system corresponding to the target container, the touch event correction factor needs to be considered, and this factor is used to correct the coordinate deviation that may be caused by factors such as devices and environments.

[0042] Finally, the target coordinates of the user's touch screen event in the target container can be accurately obtained by combining the inverse transformation matrix, the original coordinates, and the touch event correction factor.

[0043] Obtain the event distribution mechanism of the virtual main window, and synchronize all the real-time picture frames based on the event distribution mechanism. The synchronization degree of the real-time picture frames is as follows: In the formula, Represents the synchronization degree of the real-time picture frame corresponding to the i-th target container; Represents the attenuation coefficient; Represents the weight coefficient corresponding to the priority of the i-th target container; Represents the distance between the target coordinates of the user's touch screen event and the center coordinates of the i-th target container; Represents the weight coefficient corresponding to the priority of the j-th target container; Represents the distance between the target coordinates of the user's touch screen event and the center coordinates of the j-th target container; N represents the number of target containers; e represents the natural constant.

[0044] Among them, all real-time picture frames can be synchronized based on the event distribution mechanism. The synchronization degree of the real-time picture frames may be affected by various factors. These factors include the attenuation coefficient, the target container priority weight coefficient, and the distance between the target coordinates of the user's touch screen event and the center coordinates of the target container.

[0045] It should be noted that the attenuation coefficient refers to the degree of attenuation caused by external changes. For the priority of each target container, a corresponding weight coefficient is set, and the higher the priority of the target container, the greater the corresponding weight coefficient. In addition, the distance between the target coordinates of the user's touch screen event and the center coordinates of each target container also affects the synchronization degree. Considering these factors, the synchronization degree of the real-time video frame corresponding to each target container can be calculated, and then the synchronization process of the real-time video frame can be accurately controlled.

[0046] The pixel value of the synthesized projection screen at the coordinate is as follows: In the formula, represents the pixel value of the synthesized projection screen at the coordinate ; Q represents the number of real-time video frames; represents the transparency of the k-th real-time video frame; represents the pixel value of the k-th real-time video frame at the coordinate ; represents the dynamic offset coordinate of the k-th real-time video frame; represents the mask matrix of the overlapping area of the k-th real-time video frame at the coordinate ;

[0047] The pixel value of the synthesized projection screen at a specific coordinate is related to multiple factors, specifically including the number of real-time video frames, transparency, dynamic offset coordinates, and the mask matrix of the overlapping area of the video frame at this coordinate. Considering these factors, the pixel value of the synthesized projection screen at this coordinate can be accurately obtained.

[0048] The transmission rate of the encoded synthesized projection screen is as follows: In the formula, R represents the transmission rate of the encoded synthesized projection screen; represents the transmission adjustment coefficient; Q represents the number of real-time video frames; represents the video complexity factor of the k-th real-time video frame; B represents the compression ratio of the encoded synthesized projection screen; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

[0049] In practical applications, the transmission adjustment coefficient, the number of real-time video frames, the video complexity factor of each frame, the video compression ratio, and the signal-to-noise ratio of the transmission channel of the projection protocol jointly determine the transmission rate of the encoded synthesized projection screen.

[0050] After receiving the encoded synthesized projection screen, the external screen projection terminal decodes and optimizes the image quality of the synthesized projection screen.

[0051] In addition, when the external screen projection terminal successfully receives the synthesized screen projection image after encoding and processing, it can perform a decoding operation on it and restore the encoded image data to the original displayable image information.

[0052] Furthermore, in order to enhance the visual effect of the synthesized screen projection image, image quality optimization can be performed on it, including operations such as enhancing colors and sharpening images, so as to present a better-quality image.

[0053] Through the introduction of the above embodiments, the multi-window heterogeneous display screen projection method based on the container execution mechanism of the present invention can construct multiple target containers based on active components, and arrange all target containers according to a preset layout template to generate a virtual main window; obtain user touch screen events through the virtual main window, convert the original coordinates of the user touch screen events into target coordinates, and send the user touch screen events to the target application window in the target container; drive the target application window to execute logical operations, and generate real-time frame images and store them in the frame buffer of the target container; read the real-time frame images in the frame buffers of all target containers through the virtual main window, and splice all the real-time frame images according to the preset layout template to obtain a synthesized screen projection image; encode the synthesized screen projection image and transmit it to the external screen projection terminal through the screen projection protocol, so that the development efficiency can be improved by using container development technology, multi-window heterogeneous display screen projection can be realized, the deficiencies of single-application separate projection can be improved, the diversity of screen projection can be enriched, and at the same time, multiple containers can be flexibly arranged in the main virtual window to adapt to different large-screen scenarios and meet diverse screen projection requirements.

[0054] The present invention can easily implement a multi-window screen projection mechanism by using container development technology, greatly reducing the code transplantation volume and improving the development efficiency to 75%; the method of the present invention can improve the limitations of single-application separate projection and enrich the diversity of large-screen scenario projection; the present invention can achieve the coexistence of multiple containers in the main virtual window and adapt to different large-screen projection scenarios by adjusting the container layout, improving the flexibility of projection.

[0055] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0056] Those of ordinary skill in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other medium that can be used to carry or store data and is readable by a computer.

[0057] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent in such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of another identical element in the process, method, commodity or device comprising the element.

Claims

1. A multi-window different display screen mirroring method based on a container execution mechanism, characterized in that, Including: Constructing a plurality of target containers based on active components, and arranging all the target containers according to a preset layout template to generate a virtual main window; Obtaining a user touch screen event through the virtual main window, converting the original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container; Driving the target application window to execute a logical operation, and generating a real-time picture frame to be stored in the frame buffer of the target container; Reading the real-time picture frames in the frame buffers of all the target containers through the virtual main window, and splicing all the real-time picture frames according to the preset layout template to obtain a synthesized projection screen picture; Encoding the synthesized projection screen picture, and transmitting it to an external screen projection end through a projection protocol.

2. The multi-window different display screen mirroring method based on a container execution mechanism according to claim 1, wherein The constructing a plurality of target containers based on active components, and arranging all the target containers according to a preset layout template to generate a virtual main window specifically includes: Constructing a plurality of the target containers based on the active components, setting the structural parameters and container functions of the target containers, and loading the target application window in the target containers; Obtaining the preset layout template, and arranging all the target containers in sequence according to the preset layout template to generate the virtual main window; Wherein, the structural parameters include length, width and coordinate system; the container function is a coordinate mapping function.

3. A multi-window different display screen projection method based on a container execution mechanism according to claim 1, characterized in that, The obtaining a user touch screen event through the virtual main window, converting the original coordinates of the user touch screen event into target coordinates, and sending the user touch screen event to a target application window in the target container specifically includes: Monitoring a touch screen operation of a user end through the virtual main window, and obtaining the corresponding user touch screen event; Obtaining the original coordinates of the user touch screen event in the virtual main window, and mapping the original coordinates to the coordinate system corresponding to the target container to obtain the target coordinates; Sending the user touch screen event to the target application window in the target container.

4. A multi-window different display screen projection method based on a container execution mechanism according to claim 1, characterized in that In the virtual main window, establishing a display mapping relationship between the target containers based on an affine transformation matrix, and the expression of the affine transformation matrix is as follows: Where M represents the affine transformation matrix; represents the ratio of the width of the target container to that of the virtual main window in the x-axis direction; represents the ratio of the height of the target container to that of the virtual main window in the y-axis direction; represents the width of the target container; represents the width of the virtual main window; represents the height of the target container; represents the height of the virtual main window; represents the coordinate offset of the target container in the x-axis direction in the virtual main window; represents the coordinate offset of the target container in the y-axis direction in the virtual main window; represents the normalized offset ratio of the target container in the x-axis direction in the virtual main window; represents the normalized offset ratio of the target container in the y-axis direction in the virtual main window.

5. A multi-window different display screen projection method based on a container execution mechanism according to claim 4, characterized in that, Based on an inverse matrix algorithm, determining the target coordinates of the user touch screen event as follows: In the formula, represents the target coordinates of the user's touch screen event; M represents the affine transformation matrix; represents the inverse transformation matrix of the affine transformation matrix; represents the original coordinates of the user's touch screen event; represents the touch event correction factor.

6. A multi-window different display screen projection method based on a container execution mechanism according to claim 1, characterized in that Obtaining the event distribution mechanism of the virtual main window, and synchronizing all the real-time picture frames based on the event distribution mechanism, and the synchronization degree of the real-time picture frames is as follows: In the formula, represents the synchronization degree of the real-time video frame corresponding to the i-th target container; represents the attenuation coefficient; represents the weight coefficient corresponding to the priority of the i-th target container; represents the distance between the target coordinate of the user touch event and the center coordinate of the i-th target container; represents the weight coefficient corresponding to the priority of the j-th target container; represents the distance between the target coordinate of the user touch event and the center coordinate of the j-th target container; N represents the number of target containers; e represents the natural constant.

7. A multi-window different display screen mirroring method based on a container execution mechanism according to claim 1, characterized in that The pixel value of the synthesized projection screen at the coordinate is as follows: In the formula, represents the pixel value of the composite projection screen at the coordinate ; Q represents the number of real-time video frames; represents the transparency of the k-th real-time video frame; represents the pixel value of the k-th real-time video frame at the coordinate ; represents the dynamic offset coordinate of the k-th real-time video frame; represents the mask matrix of the video overlapping area of the k-th real-time video frame at the coordinate .

8. A multi-window different display screen mirroring method based on a container execution mechanism according to claim 7, characterized in that, The transmission rate of the encoded synthesized projection screen picture is as follows: In the formula, R represents the transmission rate of the encoded composite projection screen image; represents the transmission adjustment coefficient; Q represents the number of real-time image frames; represents the image complexity factor of the k-th real-time image frame; B represents the compression ratio of the encoded composite projection screen image; L represents the signal-to-noise ratio of the transmission channel corresponding to the projection protocol.

9. A multi-window different display screen projection method based on a container execution mechanism according to claim 1, characterized in that After receiving the encoded synthesized projection screen picture, the external screen projection end decodes and optimizes the picture quality of the synthesized projection screen picture.

Images