A virtual screen projection control method, device and computer readable storage medium

Abstract

This invention discloses a virtual screen projection control method, device, and computer-readable storage medium. The method includes: creating an outer screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the outer screen; launching the application to be projected in the proxy screen, and using the display area of ​​the proxy screen as the layout border of the application; rendering the application in full screen within the layout border; scaling the proxy screen and mapping it to the outer screen according to the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of projection, and greatly enhancing the user experience of the projection function.

Description

Technical Field

[0001] This invention relates to the field of mobile communications, and more particularly to a virtual screen projection control method, device, and computer-readable storage medium. Background Technology

[0002] In the current technology, with the continuous development of smart terminal devices, screen projection has become an important function of mobile phones and other devices. This function can project the rich resources on the mobile phone onto a large screen and share them with others. It can also connect the mobile phone with external display devices (TV, monitor, computer, etc.) to collaboratively solve problems, combining the portability of the mobile phone with the ease of operation and comfort of other display devices.

[0003] The existing screen mirroring technologies mainly include two types: screen mirroring and extended screen mirroring.

[0004] Screen mirroring is the process of scaling up the content displayed on your phone and projecting it onto an external display in a mirrored manner.

[0005] Extended screen mirroring creates an "external screen" on your phone for an external display, allowing you to directly store and display the content you want to mirror from your phone on this "external screen".

[0006] All of the above methods have certain drawbacks.

[0007] When mirroring the screen, if the aspect ratio of the phone is different from that of the external display, black borders will remain on the external display, and the screen will not be able to fill the entire screen.

[0008] Extended screen mirroring has a narrower scope of application. It requires the external display to have fixed parameters such as resolution. If the parameters change, the screen associated with the external display will also change accordingly. However, mobile applications do not support this dynamic change, resulting in display abnormalities. Therefore, it cannot be used with third-party screen mirroring software.

[0009] Therefore, improving the completeness of the screen projection display and the stability of the screen projection function based on the two existing screen projection solutions has become an urgent technical problem to be solved. Summary of the Invention

[0010] To address the aforementioned technical deficiencies in the prior art, this invention proposes a virtual screen projection control method, which includes:

[0011] During the initialization phase of screen casting, an external screen corresponding to the screen casting terminal is created, as well as a proxy screen with the same aspect ratio as the external screen.

[0012] Launch the application to be projected in the proxy screen, and use the display area of ​​the proxy screen as the layout border of the application.

[0013] The application is rendered in full screen within the area of ​​the layout border.

[0014] Based on the size relationship between the proxy screen and the outer screen, the proxy screen is scaled and mapped to the outer screen so that the projection terminal can obtain the mirror projection data of the outer screen.

[0015] Optionally, the step of creating an external screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the external screen, includes:

[0016] Create a screen mirroring link with the screen mirroring client.

[0017] The display parameters of the screen mirroring terminal are obtained through the screen mirroring link.

[0018] Optionally, the step of creating an external screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the external screen, further includes:

[0019] The outer screen is created based on the display parameters.

[0020] Obtain the first size and aspect ratio of the outer screen.

[0021] Optionally, the step of creating an external screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the external screen, further includes:

[0022] Detect whether the first size exceeds a preset size threshold.

[0023] If the first size exceeds the preset size threshold, a proxy screen with a second size smaller than the first size is created with the aspect ratio.

[0024] Optionally, launching the application to be projected in the proxy screen and using the display area of ​​the proxy screen as the layout border of the application includes:

[0025] When selecting the application to be projected, the proxy screen is used as the virtual running screen of the application.

[0026] The virtual reality range of the virtual running screen is used as the layout border.

[0027] Optionally, scaling the proxy screen and mapping it to the outer screen based on the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, includes:

[0028] Identify whether the landscape and portrait orientations of the proxy screen and the outer screen are consistent.

[0029] If the horizontal and vertical orientations of the proxy screen and the outer screen are the same, the proxy screen is scaled and mapped to the outer screen. If the horizontal and vertical orientations of the proxy screen and the outer screen are different, the proxy screen is scaled and mapped to the outer screen according to the horizontal and vertical orientations of the proxy screen.

[0030] Optionally, scaling the proxy screen and mapping it to the outer screen based on the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, further includes:

[0031] Monitor the update parameters of the first size and the aspect ratio of the outer screen.

[0032] The proxy screen is scaled and mapped to the external screen according to the updated parameters.

[0033] Optionally, scaling the proxy screen and mapping it to the outer screen based on the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, further includes:

[0034] When the projection link is disconnected, the drawing state of the proxy screen is maintained.

[0035] When the screen mirroring connection is restored, the mirrored screen mirroring data of the external screen continues to be transmitted to the screen mirroring terminal.

[0036] The present invention also proposes a virtual screen projection control device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the virtual screen projection control method as described in any of the preceding claims.

[0037] The present invention also proposes a computer-readable storage medium storing a virtual screen projection control program, which, when executed by a processor, implements the steps of the virtual screen projection control method as described in any of the preceding claims.

[0038] The present invention provides a virtual screen projection control method, device, and computer-readable storage medium. During the projection initialization phase, an outer screen corresponding to the projection terminal is created, along with a proxy screen with the same aspect ratio as the outer screen. The application to be projected is launched in the proxy screen, and the display area of ​​the proxy screen is used as the layout border of the application. The application is rendered in full screen within the layout border. Based on the size relationship between the proxy screen and the outer screen, the proxy screen is scaled and mapped to the outer screen, enabling the projection terminal to obtain mirrored projection data from the outer screen. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of projection, and greatly enhancing the user experience of the projection function. Attached Figure Description

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0040] Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal according to the present invention;

[0041] Figure 2 This is a communication network system architecture diagram provided in an embodiment of the present invention;

[0042] Figure 3 This is a flowchart of the first embodiment of the virtual screen projection control method of the present invention;

[0043] Figure 4 This is a flowchart of the second embodiment of the virtual screen projection control method of the present invention;

[0044] Figure 5 This is a flowchart of the third embodiment of the virtual screen projection control method of the present invention;

[0045] Figure 6 This is a flowchart of the fourth embodiment of the virtual screen projection control method of the present invention;

[0046] Figure 7 This is a flowchart of the fifth embodiment of the virtual screen projection control method of the present invention;

[0047] Figure 8 This is a flowchart of the sixth embodiment of the virtual screen projection control method of the present invention;

[0048] Figure 9 This is a flowchart of the seventh embodiment of the virtual screen projection control method of the present invention;

[0049] Figure 10 This is a flowchart of the eighth embodiment of the virtual screen projection control method of the present invention;

[0050] Figure 11This is a schematic diagram of mirrored screen projection, representing the first embodiment of the virtual screen projection control method of the present invention.

[0051] Figure 12 This is an extended projection schematic diagram of the first embodiment of the virtual screen projection control method of the present invention;

[0052] Figure 13 This is a schematic diagram of proxy screen projection in the first embodiment of the virtual screen projection control method of the present invention;

[0053] Figure 14 This is another schematic diagram of proxy screen projection in the first embodiment of the virtual screen projection control method of the present invention. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0055] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0056] Terminals can be implemented in various forms. For example, the terminals described in this invention may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0057] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to embodiments of the present invention can also be applied to fixed-type terminals.

[0058] Please see Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present invention. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0059] The following is combined Figure 1 A detailed introduction to each component of the mobile terminal:

[0060] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution).

[0061] WiFi is a short-range wireless transmission technology. Mobile terminals using the WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0062] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0063] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage media) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0064] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0065] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0066] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands from processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being limited here.

[0067] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0068] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0069] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0070] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0071] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0072] although Figure 1 As not shown, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0073] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.

[0074] Please see Figure 2 , Figure 2 This invention provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and an operator's IP service 204, which are connected in sequence.

[0075] Specifically, UE201 can be the aforementioned terminal 100, which will not be elaborated here.

[0076] E-UTRAN202 includes eNodeB2021 and other eNodeB2022s. Among them, eNodeB2021 can connect to other eNodeB2022s through backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203. eNodeB2021 can provide UE201 with access to EPC203.

[0077] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 provides registers to manage functions such as the Home Location Register (not shown in the diagram) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0078] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0079] Although the above description uses the LTE system as an example, those skilled in the art should understand that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems, etc., which are not limited here.

[0080] Based on the aforementioned mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.

[0081] Example 1

[0082] Figure 3 This is a flowchart of the first embodiment of the virtual screen projection control method of the present invention. A virtual screen projection control method, the method comprising:

[0083] S1. During the initialization phase of screen casting, create an external screen corresponding to the screen casting terminal, and create a proxy screen with the same aspect ratio as the external screen.

[0084] S2. Launch the application to be projected in the proxy screen, and use the display range of the proxy screen as the layout border of the application.

[0085] S3. Draw the application in full screen within the area of ​​the layout border.

[0086] S4. Based on the size relationship between the proxy screen and the outer screen, the proxy screen is scaled and mapped to the outer screen so that the projection terminal can obtain the mirror projection data of the outer screen.

[0087] In this embodiment, please refer to Figure 11 The diagram illustrating screen mirroring shows that screen mirroring projects all content from the phone screen onto an external display. During the mirroring process, it tries to fill the display space as much as possible. However, if the aspect ratio of the external display differs from the phone's, filling the entire screen can cause stretching and distortion in one direction. To avoid distortion, the mirroring process needs to scale the display while keeping the aspect ratio locked. This results in black borders in two opposite directions because there is no content displayed. Figure 11 As shown, the phone's aspect ratio is 20:9, and the external display's aspect ratio is 16:9. Therefore, under screen mirroring, there will be black borders in both the top and bottom directions.

[0088] In this embodiment, please refer to Figure 12The diagram illustrating extended screen mirroring shows that when extended screen mirroring is used, the phone has two screens, each displaying different application content. The phone's own screen has global windows such as the status bar; the outer screen can have customized windows such as the status bar, or it can only display application content. For example... Figure 12 As shown, application A is displayed on the phone screen, and application B is displayed on an external screen. The external screen's size and other parameters are the same as the external display. Application B is displayed directly according to the size and layout of the external screen, so the screen mirroring effect is to fill the entire screen. At this time, if the external display parameters change, application B must support corresponding changes; otherwise, an anomaly will occur. However, in the current technological environment, third-party applications other than system applications cannot make the corresponding changes.

[0089] In this embodiment, please refer to Figure 13 The diagram illustrates the proxy screen projection. As can be seen, this embodiment adds a proxy screen compared to extended projection, and the proxy screen has the same aspect ratio as the outer screen. Application B, which needs to be projected, is drawn in the proxy screen according to its size. The content in the proxy screen is then scaled and mapped onto the outer screen. Simultaneously, another application A is displayed on the phone screen, displaying different content than the external display; both can be used simultaneously.

[0090] Specifically, in this embodiment, changes to the phone's screen are monitored. When a screen mirroring action is detected, the system creates an outer screen. At this time, a corresponding virtual screen is created as the proxy screen for this embodiment. The aspect ratio of the virtual screen is set to be the same as the outer screen. Optionally, if the outer screen is too large, the virtual screen can be smaller than the outer screen; that is, as long as the aspect ratio of the proxy screen and the outer screen is consistent, it is acceptable. Optionally, a smaller virtual screen reduces memory usage and improves performance to some extent. In this embodiment, if an application needs screen mirroring, the application is launched onto the proxy screen, using the size of the proxy screen as the outer border of the application layout, allowing the application to fill the entire screen and be drawn within the proxy screen. In this embodiment, the proxy screen is scaled and mapped onto the outer screen. Specifically, the external display is generally rectangular; when its width is greater than its height, it is called a landscape screen; otherwise, it is a portrait screen. There are two scenarios: When the application's orientation matches the external screen, the proxy screen and the external screen are similar rectangles in the same direction, and are directly scaled and mapped proportionally. When the application's orientation differs from the external screen, the proxy screen, while still a similar rectangle, is oriented in different directions. For details, please refer to [link / reference]. Figure 14Another schematic diagram of proxy screen projection is shown. In this diagram, the proxy screen has a width of w and a height of h; the outer screen has a width of W and a height of H. After mapping, the content area rectangle displayed by the proxy screen on the outer screen has a width of wH / h and a height of H. The horizontal distance between the upper left corner of the content area and the upper left corner of the outer screen is (W-wH / h) / 2. Optionally, when the length and width of the outer screen change dynamically, the above scaling steps are repeated to adjust the mapping relationship.

[0091] Optionally, in this embodiment, during wireless screen mirroring, the wireless data transmission rate between the mobile phone and the external display may change due to environmental interference or changes in distance. To ensure smooth playback, third-party screen mirroring software typically needs to dynamically adjust the size of the external screen. The external screen may become larger or smaller while maintaining the same aspect ratio. In this case, the system only needs to recalculate the mapping ratio according to the scaling steps described above, without notifying the application layout to make corresponding size changes. Since most applications do not support dynamic size changes, this improves the application compatibility of this embodiment.

[0092] Optionally, this embodiment addresses scenarios where the physical connection for screen mirroring is unexpectedly lost, causing extended screen mirroring to be interrupted. In this embodiment, when the physical connection for screen mirroring is unexpectedly lost for a short period (e.g., due to a poor connection in the HDMI screen mirroring cable), the external screen will be destroyed by the system. In a typical extended screen mirroring solution, the application will stop when the external screen is destroyed. However, in this embodiment, because of the existence of a proxy screen, the application runs on the proxy screen and is unaffected. After the user reconnects and a new external screen is created, they can remap the proxy screen to the new external screen to continue using the application.

[0093] In this embodiment, extended screen casting and mirroring technologies are combined. A virtual screen is created on the mobile phone as a proxy screen between the application content to be cast and the external screen. The application content to be cast is displayed on this virtual screen, and then the virtual screen is mirrored onto the external screen as a whole. As can be seen, in this embodiment, because the virtual screen and the external screen are geometrically similar, the application fills the entire screen in the virtual screen, and when it is finally presented on the external screen, it also fills the entire screen, resulting in a good user visual experience. The virtual screen and the external screen have a mirror mapping relationship. When the external screen changes dynamically, the virtual screen only needs to be remapped according to the new parameters. The application in the virtual screen does not need to perceive the change, so the range of applicable applications is wide. Only the virtual screen, which acts as a proxy, is operated, and the external screen is not operated. The implementation method of the external screen does not require special requirements, so the range of applicable screen casting is wide. It can be cast to monitors, computers, and TVs; it can be used for both wired and wireless screen casting; it can be used for screen casting with general standard protocols, as well as third-party screen casting software.

[0094] The beneficial effects of this embodiment are as follows: During the initialization phase of screen casting, an outer screen corresponding to the casting terminal is created, along with a proxy screen with the same aspect ratio as the outer screen. The application to be cast is launched in the proxy screen, and the display area of ​​the proxy screen is used as the layout border of the application. The application is rendered in full screen within the layout border. Based on the size relationship between the proxy screen and the outer screen, the proxy screen is scaled and mapped to the outer screen, enabling the casting terminal to obtain mirrored casting data from the outer screen. This achieves a user-friendly virtual screen casting control scheme, improving the display effect, stability, and applicability of screen casting, and greatly enhancing the user experience of the screen casting function.

[0095] Example 2

[0096] Figure 4 This is a flowchart of the second embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of creating an outer screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the outer screen, includes:

[0097] S11. Create a screen mirroring link with the screen mirroring terminal.

[0098] S12. Obtain the display parameters of the screen projection terminal through the screen projection link.

[0099] The beneficial effect of this embodiment is that by creating a projection link with the projection terminal and obtaining the display parameters of the projection terminal through the projection link, a user-friendly virtual screen projection control scheme is achieved, improving the display effect, stability, and applicability of projection, and greatly enhancing the user experience of the projection function.

[0100] Example 3

[0101] Figure 5 This is a flowchart of the third embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of creating an outer screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the outer screen, further includes:

[0102] S13. Create the outer screen according to the display parameters.

[0103] S14. Obtain the first size and aspect ratio of the outer screen.

[0104] Optionally, in this embodiment,

[0105] Optionally, in this embodiment,

[0106] The beneficial effect of this embodiment is that by creating the outer screen according to the display parameters and obtaining the first size and aspect ratio of the outer screen, a user-friendly virtual screen projection control scheme is achieved, improving the display effect, stability, and applicability of the projection, and greatly enhancing the user experience of the projection function.

[0107] Example 4

[0108] Figure 6 This is a flowchart of the fourth embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of creating an outer screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the outer screen, further includes:

[0109] S15. Detect whether the first size exceeds a preset size threshold.

[0110] S16. If the first size exceeds the preset size threshold, a proxy screen with a second size smaller than the first size is created with the aspect ratio.

[0111] The beneficial effect of this embodiment is that by detecting whether the first size exceeds a preset size threshold, and if the first size exceeds the preset size threshold, a proxy screen of a second size smaller than the first size is created with the specified aspect ratio. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of the projection, and greatly enhancing the user experience of the projection function.

[0112] Example 5

[0113] Figure 7 This is a flowchart of the fifth embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of launching the application to be projected in the proxy screen and using the display area of ​​the proxy screen as the layout border of the application includes:

[0114] S21. When selecting the application to be projected, the proxy screen is used as the virtual running screen of the application.

[0115] S22. Use the virtual reality range of the virtual running screen as the layout border.

[0116] The beneficial effect of this embodiment is that, when the application to be projected is selected, the proxy screen is used as the virtual running screen of the application; and the virtual reality range of the virtual running screen is used as the layout border. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of the projection, and greatly enhancing the user experience of the projection function.

[0117] Example 6

[0118] Figure 8 This is a flowchart of the sixth embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of scaling the proxy screen and mapping it to the outer screen according to the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, includes:

[0119] S41. Identify whether the horizontal and vertical orientations of the proxy screen and the outer screen are consistent.

[0120] S42. If the horizontal and vertical orientations of the proxy screen and the outer screen are the same, then the proxy screen is scaled and mapped to the outer screen. If the horizontal and vertical orientations of the proxy screen and the outer screen are not the same, then the proxy screen is scaled and mapped to the outer screen according to the horizontal and vertical orientations of the proxy screen.

[0121] The beneficial effect of this embodiment lies in identifying whether the landscape and portrait orientations of the proxy screen and the outer screen are consistent. If the landscape and portrait orientations of the proxy screen and the outer screen are consistent, the proxy screen is scaled and mapped to the outer screen. If the landscape and portrait orientations of the proxy screen and the outer screen are inconsistent, the proxy screen is scaled and mapped to the outer screen according to the landscape and portrait orientation of the proxy screen. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of projection, and greatly enhancing the user experience of the projection function.

[0122] Example 7

[0123] Figure 9 This is a flowchart of the seventh embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of scaling the proxy screen and mapping it to the outer screen according to the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, further includes:

[0124] S43. Monitor the update parameters of the first size and the aspect ratio of the outer screen.

[0125] S44. Adjust the scaling ratio of the proxy screen according to the updated parameters and map it to the external screen.

[0126] The beneficial effect of this embodiment is that by monitoring the update parameters of the first size and the aspect ratio of the outer screen, and adjusting the scaling of the proxy screen according to the update parameters and mapping it to the scaling ratio of the outer screen, a user-friendly virtual screen projection control scheme is achieved, improving the display effect, stability, and applicability of the projection, and greatly enhancing the user experience of the projection function.

[0127] Example 8

[0128] Figure 10 This is a flowchart of the eighth embodiment of the virtual screen projection control method of the present invention. Based on the above embodiment, the step of scaling the proxy screen and mapping it to the outer screen according to the size relationship between the proxy screen and the outer screen, so that the projection terminal obtains the mirrored projection data of the outer screen, further includes:

[0129] S45. When the projection link is disconnected, maintain the drawing state of the proxy screen.

[0130] S46. When the screen mirroring link is restored, continue to transmit the mirrored screen mirroring data of the external screen to the screen mirroring terminal.

[0131] The beneficial effect of this embodiment is that it maintains the drawing state of the proxy screen when the projection link is disconnected; and continues to transmit the mirror projection data of the outer screen to the projection terminal when the projection link is restored. This achieves a user-friendly virtual screen projection control scheme, improving the display effect, stability, and applicability of projection, and greatly enhancing the user experience of the projection function.

[0132] Example 9

[0133] Based on the above embodiments, the present invention also proposes a virtual screen projection control device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the virtual screen projection control method as described in any of the above embodiments.

[0134] It should be noted that the above-described device embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the device embodiments, which will not be repeated here.

[0135] Example 10

[0136] Based on the above embodiments, the present invention also proposes a computer-readable storage medium storing a virtual screen projection control program, which, when executed by a processor, implements the steps of the virtual screen projection control method as described in any of the above embodiments.

[0137] It should be noted that the above-described medium embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the medium embodiments, which will not be repeated here.

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0141] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A virtual screen projection control method, characterized in that, The method includes: During the initialization phase of screen casting, an external screen corresponding to the screen casting terminal is created, as well as a proxy screen with the same aspect ratio as the external screen; Launch the application to be projected in the proxy screen, and use the display area of ​​the proxy screen as the layout border of the application; The application is rendered in full screen within the area of ​​the layout border. Based on the size relationship between the proxy screen and the outer screen, the proxy screen is scaled and mapped to the outer screen so that the projection terminal can obtain the mirror projection data of the outer screen; The step of creating an external screen corresponding to the projection terminal during the projection initialization phase, and creating a proxy screen with the same aspect ratio as the external screen, includes: Create a screen mirroring link with the screen mirroring client; The display parameters of the screen mirroring terminal are obtained through the screen mirroring link; The outer screen is created according to the display parameters; Obtain the first size and aspect ratio of the outer screen; Detect whether the first size exceeds a preset size threshold; If the first size exceeds the preset size threshold, a proxy screen with a second size smaller than the first size is created with the aspect ratio. The step of launching the application to be projected in the proxy screen and using the display area of ​​the proxy screen as the layout border of the application includes: When selecting the application to be projected, the proxy screen is used as the virtual running screen of the application; The virtual reality range of the virtual running screen is used as the layout border; The step of scaling and mapping the proxy screen to the external screen based on the size relationship between the proxy screen and the external screen, so that the projection terminal obtains the mirrored projection data of the external screen, includes: Identify whether the landscape and portrait orientations of the proxy screen and the outer screen are consistent; If the horizontal and vertical orientations of the proxy screen and the outer screen are the same, the proxy screen is scaled and mapped to the outer screen; if the horizontal and vertical orientations of the proxy screen and the outer screen are different, the proxy screen is scaled and mapped to the outer screen according to the horizontal and vertical orientations of the proxy screen. Monitor the update parameters of the first size and the aspect ratio of the outer screen; The scaling ratio of the proxy screen is adjusted according to the updated parameters and mapped to the scaling ratio of the external screen; When the projection link is disconnected, the drawing state of the proxy screen is maintained; When the screen mirroring connection is restored, the mirrored screen mirroring data of the external screen continues to be transmitted to the screen mirroring terminal.

2. A virtual screen projection control device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the virtual screen projection control method as described in claim 1.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a virtual screen projection control program, which, when executed by a processor, implements the steps of the virtual screen projection control method as described in claim 1.

Images