Screen projection method and device, equipment and storage medium

Abstract

The invention discloses a screen projection method and device, equipment and a storage medium, and is applied to the technical field of terminal products and the like. The method is used for realizing partitioned screen projection of the folding screen. In the embodiment of the invention, for terminal equipment with a first display screen and a second display screen, in response to a screen projection instruction, a first display function display corresponding to the first display screen is created, and a second display function display corresponding to the second display screen is created; displaying a screen projection prompt in the terminal equipment; determining target projection equipment and a target display screen based on the screen projection prompt; and projecting the interface of the target display screen to the target projection equipment based on the display corresponding to the target display screen. In the embodiment of the invention, the corresponding display is set for each display screen, so that projection can be only performed on the first display screen or the second display screen according to the screen projection demand of the user, and the personalized screen projection demand of the user is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of terminals, and in particular, to a screen mirroring method, device, equipment and storage medium. Background Art

[0002] With the development of application display technology, the functions of terminal devices such as laptop computers are becoming more and more abundant, and more and more users use mobile terminals to watch videos, browse files, etc. However, sometimes due to actual needs, it is necessary to mirror the display content on the terminal device to be displayed on a large screen / smart screen, monitor, etc., for convenient viewing.

[0003] For conventional terminal devices such as laptop computers, the upper half is a screen, the lower half is a keyboard, and the operating system is Windows or MacOS. Users can wirelessly mirror the physical internal screen to devices such as a large screen / smart screen, monitor, etc. through the screen mirroring entry provided by the system, and the devices such as a large screen / smart screen, monitor, etc. display the same content as the laptop computer.

[0004] However, for terminal devices with foldable full screens, if the screen mirroring method of the above laptop computer is followed, devices such as a large screen / smart screen, monitor, etc. can only project based on the entire screen, resulting in a poor display effect of screen mirroring. Summary of the Invention

[0005] The purpose of the present invention is to provide a screen mirroring method, device, equipment and storage medium to solve the technical problems in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a screen mirroring method, which is applied to a terminal device. The terminal device includes: a first display screen and a second display screen. The method includes: in response to a screen mirroring instruction, creating a first display corresponding to the first display screen and creating a second display corresponding to the second display screen; displaying a screen mirroring prompt in the terminal device; determining a target projection device and a target display screen based on the screen mirroring prompt; and mirroring the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

[0007] In the embodiment of the present application, a corresponding display is set for each display screen. Therefore, according to the user's screen mirroring requirements, only the first display screen or the second display screen can be projected, realizing the user's personalized screen mirroring requirements.

[0008] In some possible embodiments, creating the first display corresponding to the first display screen includes: determining the first screen width of the first display screen; and adjusting the height coordinate parameter in the initial display of the terminal device according to the first screen height to obtain the first display.

[0009] In the embodiments of the present application, by adjusting the parameters of the initial display, it is ensured that the obtained first display can accurately represent the size of the first display screen, thereby ensuring the accuracy of subsequent screen mirroring operations.

[0010] In some possible embodiments, creating a second display corresponding to the second display screen includes: determining the second screen height and the second screen width of the second display screen; generating a first coordinate parameter according to the second screen height, and generating a second coordinate parameter according to the second screen width; determining the second display according to the first coordinate parameter and the second coordinate parameter.

[0011] In the embodiments of the present application, by creating the second display, one display corresponds to each display screen, thereby realizing subsequent screen mirroring operations only on the first display screen or the second display screen according to the user's selection.

[0012] In some possible embodiments, the method further includes: obtaining the comprehensive load of the terminal device; determining that the frame rate of the non-target display screen needs to be adjusted according to the comprehensive load, and determining the frame rate percentage of the non-target display screen; wherein, if the target display screen is the first display screen, the non-target display screen is the second display screen, and if the target display screen is the second display screen, the non-target display screen is the first display screen; determining the target frame rate according to the frame rate percentage; adjusting the frame rate of the non-target display screen to the target frame rate.

[0013] In the embodiments of the present application, when the computing power of the comprehensive load of the terminal device is sufficient, there is no need to reduce the frame rate of the non-target display screen, and only when the computing power is insufficient, the frame rate of the non-target display screen is reduced, thereby reducing the load of the operating system of the terminal device.

[0014] In some possible embodiments, determining the frame rate percentage of the non-target display screen includes: determining the focal distance between the first focus of the target display screen and the second focus of the non-target display screen; obtaining the physical distance between the first focus and the second focus according to the focal distance and the pixel density; determining the frame rate calculation function according to the physical distance and the preset distance threshold; obtaining the frame rate percentage according to the frame rate calculation function and the physical distance.

[0015] In the embodiments of the present application, by setting the focal distance to determine the frame rate percentage, it is possible to reduce the load of the terminal device while ensuring the user experience.

[0016] In some possible embodiments, after performing a screen mirroring operation on the target projection device based on the display corresponding to the target display screen, the method further includes: determining that there is a refresh area in the target display screen according to the first frame data, and sending the display data corresponding to the refresh area to the target projection device; the first frame data is the next frame data of the current frame data being displayed in the target display screen.

[0017] In the embodiments of the present application, the load of the terminal device is reduced by means of local refreshing.

[0018] In some possible embodiments, before creating the first display function display corresponding to the first display screen and the second display corresponding to the second display screen, the method further includes: determining the folding angle between the first display screen and the second display screen; and determining that the terminal device is in a folded state according to the folding angle.

[0019] In the embodiments of the present application, the state of the terminal device is determined by calculating the folding angle between the first display screen and the second display screen, which ensures the accuracy of the determined state of the terminal device.

[0020] In a second aspect, the embodiments of the present application provide a screen mirroring device applied to a terminal device. The terminal device includes: a first display screen and a second display screen. The device includes: a response module configured to create a first display corresponding to the first display screen and a second display corresponding to the second display screen in response to a screen mirroring instruction; a prompt module configured to display a screen mirroring prompt in the terminal device; a determination module configured to determine a target projection device and a target display screen based on the screen mirroring prompt; and a screen mirroring module configured to mirror the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

[0021] In some possible embodiments, the response module is specifically configured to: determine the first screen width of the first display screen; and adjust the height coordinate parameter in the initial display of the terminal device according to the first screen height to obtain the first display.

[0022] In some possible embodiments, the response module is specifically configured to: determine the second screen height and the second screen width of the second display screen; generate a first coordinate parameter according to the second screen height, and generate a second coordinate parameter according to the second screen width; and determine the second display according to the first coordinate parameter and the second coordinate parameter.

[0023] In some possible embodiments, the screen mirroring module is further configured to: obtain the comprehensive load of the terminal device; determine that the frame rate of the non-target display screen needs to be adjusted according to the comprehensive load, and determine the frame rate percentage of the non-target display screen; wherein, if the target display screen is the first display screen, the non-target display screen is the second display screen, and if the target display screen is the second display screen, the non-target display screen is the first display screen; determine the target frame rate according to the frame rate percentage; and adjust the frame rate of the non-target display screen to the target frame rate.

[0024] In some possible embodiments, the screen mirroring module is specifically configured to: determine the focal distance between the first focus of the target display screen and the second focus of the non-target display screen; obtain the physical distance between the first focus and the second focus according to the focal distance and the pixel density; determine the frame rate calculation function according to the physical distance and the preset distance threshold; and obtain the frame rate percentage according to the frame rate calculation function and the physical distance.

[0025] In some possible embodiments, the screen mirroring module is further configured to: determine that there is a refresh area in the target display screen according to the first frame of data, and send the display data corresponding to the refresh area to the target projection device; the first frame of data is the next frame of data of the current frame of data being displayed on the target display screen.

[0026] In some possible embodiments, the response module is further configured to: determine the folding angle between the first display screen and the second display screen; and determine that the terminal device is in a folded state according to the folding angle.

[0027] In a third aspect, another embodiment of the present application further provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method provided in the first aspect or the second aspect embodiment of the present application.

[0028] In a fourth aspect, another embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to cause a computer to execute any method provided in the first aspect or the second aspect embodiment of the present application.

[0029] In a fifth aspect, another embodiment of the present application further provides a computer program product, characterized in that the computer program product includes: computer program code, when the computer program code runs on a computer, it causes the computer to execute any method provided in the first aspect or the second aspect embodiment above.

[0030] Other features and advantages of the present application will be described in the following specification, and, in part, will become apparent from the specification, or will be understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained through the structures specifically pointed out in the written specification, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of screen mirroring of a conventional laptop computer and other devices provided in an embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of a folding screen device provided in an embodiment of the present application;

[0033] Figure 3 Schematic diagram of screen mirroring for a foldable screen in the related art provided by an embodiment of the present application;

[0034] Figure 4 Schematic diagram of the screen mirroring process in the related art provided by an embodiment of the present application;

[0035] Figure 5 Schematic diagram of an application scenario of a screen mirroring method provided by an embodiment of the present application;

[0036] Figure 6 Schematic diagram of the system architecture of a screen mirroring method provided by an embodiment of the present application;

[0037] Figure 7 Schematic diagram of the process flow of a screen mirroring method provided by an embodiment of the present application;

[0038] Figure 8 Schematic diagram of the folding angle of a screen mirroring method provided by an embodiment of the present application;

[0039] Figure 9 Schematic diagram for determining whether it is in a folded state based on the folding angle of a screen mirroring method provided by an embodiment of the present application;

[0040] Figure 10 Schematic diagram of the target device list and the screen mirroring area list of a screen mirroring method provided by an embodiment of the present application;

[0041] Figure 11 Schematic diagram of the screen size of a terminal device of a screen mirroring method provided by an embodiment of the present application;

[0042] Figure 12A Schematic diagram of sending the refresh area of the second display screen to the target projection device of a screen mirroring method provided by an embodiment of the present application;

[0043] Figure 12B Schematic diagram of sending the refresh area of the first display screen to the target projection device of a screen mirroring method provided by an embodiment of the present application;

[0044] Figure 13 Schematic diagram of reducing the frame rate of a screen mirroring method provided by an embodiment of the present application;

[0045] Figure 14 Schematic diagram of the process flow of reducing the frame rate of a screen mirroring method provided by an embodiment of the present application;

[0046] Figure 15 Schematic diagram of the preset gear adjustment of a screen mirroring method provided by an embodiment of the present application;

[0047] Figure 16Schematic flowchart for determining the frame rate percentage of a non-target display screen in a screen mirroring method provided by an embodiment of the present application;

[0048] Figure 17 Schematic diagram of a frame rate calculation function in a screen mirroring method provided by an embodiment of the present application;

[0049] Figure 18 Schematic diagram showing the relationship between the frame rate percentage and the target frame rate in a screen mirroring method provided by an embodiment of the present application;

[0050] Figure 19 Overall flowchart of a screen mirroring method provided by an embodiment of the present application;

[0051] Figure 20 Schematic diagram of a device in a screen mirroring method provided by an embodiment of the present application;

[0052] Figure 21 Schematic diagram of an electronic device in a screen mirroring method provided by an embodiment of the present application. Detailed implementation manners

[0053] The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] In order to better understand the technical solution of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0055] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0056] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms of "a", "the" and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should be understood that the term " / and / " used herein is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, a and / or b can represent: a exists alone, a and b exist simultaneously, and b exists alone. In addition, the character " / " herein generally represents an "or" relationship between the preceding and following associated objects.

[0058] With the development of application display technologies, the functions of terminal devices such as laptops have become increasingly rich, and more and more users use mobile terminals to watch videos, browse files, etc. However, sometimes due to actual needs, it is necessary to project the display content on the terminal device onto a large screen / smart screen or monitor for convenient viewing.

[0059] As Figure 1 shown, the upper half of a conventional device such as a laptop is the screen, and the lower half is the keyboard. The operating system is Windows or MacOS. Through the screen mirroring entry provided by the system, users can wirelessly mirror the physical internal screen onto devices such as a large screen / smart screen or monitor, and the large screen / smart screen or monitor displays the same content as the laptop.

[0060] For a folding full-screen terminal device as Figure 2 shown, the entire screen is a complete physical flexible screen, but it can be folded inward to present two different screens, the upper and the lower. Users can operate their respective desktops and windows on different screens to achieve partition management.

[0061] If the screen mirroring method of a conventional laptop is adopted, there are significant differences in the screen size ratio, resolution, etc. between the terminal device and devices such as a large screen / smart screen or monitor, and the projection device does not support changes in the folding, unfolding, and rotation scenarios of the terminal device. As Figure 3 shown, therefore, the terminal device can only project the entire screen. It is impossible to project only a part of the screen of the terminal device, resulting in a poor display effect of the screen mirroring.

[0062] In related technologies, the PC operating system of the sending end (terminal device) provides a prefabricated screen mirroring entry and screen mirroring capability. For example, Windows projects the screen based on the Miracast protocol, and Mac projects the screen based on the AirPlay protocol. However, both of the above methods perform video encoding on the entire content of the internal screen of the sending end and then project it onto the receiving end for display. The specific screen mirroring process, as Figure 4 shown, mainly includes the following steps:

[0063] 1. Device discovery process: The operating system of the sending end provides a screen mirroring entry. Based on the screen mirroring operation triggered by the user, the operating system of the sending end sends a broadcast to surrounding devices based on the relevant screen mirroring protocol. Through the broadcast, it discovers devices that support being screen mirrored. After receiving the reply from the device that can be screen mirrored, the operating system of the sending end displays a list of devices that can be screen mirrored.

[0064] 2. Device connection authentication / screen projection process: The user selects a target device from the list of devices that can be screen mirrored, uses the selected target device as the receiving end, establishes a communication connection with the receiving end based on the wireless transmission protocol, and performs authentication and networking.

[0065] 3. Ability negotiation process: The sender and the receiver respectively transmit their own capabilities based on the secure transmission link, including but not limited to: screen resolution, dots per inch (DPI), and relevant software version capabilities, and negotiate the parameter information corresponding to the encoding and decoding at both ends.

[0066] 4. Receiver window registration process: The receiver registers a window Surface for receiving and displaying the video stream with the screen mirroring protocol based on the information such as the screen resolution and DPI negotiated at both ends.

[0067] 5. Audio and video stream transmission process: The sender creates a virtual display function (Display) based on the information such as the screen resolution and DPI negotiated at both ends, records the physical inner screen desktop onto the virtual Display, and the screen mirroring protocol performs video encoding and packaging. At the same time, the local audio stream data is packaged and transmitted to the screen mirroring protocol of the receiver based on the wireless transmission link. After decoding, it is sent to the corresponding window of the receiver for display.

[0068] Adopt Figure 4 When performing screen mirroring using the screen mirroring process shown, the user can only project the entire physical screen. If the user uses a foldable screen, it is impossible to adapt according to the desktop and window partition conditions after folding the physical screen of the foldable screen, and it is impossible to meet the user's personalized screen mirroring requirements.

[0069] To solve the above problems, the embodiments of the present application provide a screen mirroring method, device, device, and storage medium. The inventive concept of the present application can be summarized as: for a terminal device having a first display screen and a second display screen, in response to a screen mirroring instruction, creating a first display corresponding to the first display screen and creating a second display corresponding to the second display screen; displaying a screen mirroring prompt on the terminal device; determining a target projection device and a target display screen based on the screen mirroring prompt; and projecting the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

[0070] In the embodiments of the present application, a corresponding display is set for each display screen. Therefore, according to the user's screen mirroring requirements, only the first display screen or the second display screen can be projected, realizing the user's personalized screen mirroring needs.

[0071] For ease of understanding, the following describes in detail a screen mirroring method provided by the embodiments of the present application with reference to the accompanying drawings:

[0072] First, the execution subject of a screen mirroring method provided by the embodiments of the present application is a terminal device, which is a foldable flexible screen and includes: a first display screen, a second display screen; as Figure 5As shown in the figure, it is a schematic diagram of an application scenario of a screen mirroring method provided by an embodiment of the present application, where:

[0073] In a meeting presentation scenario, the user can move the presentation materials to be presented and discussed to the first display screen (upper half screen) of the terminal device, and place some prompt materials in the second display screen (lower half screen) of the terminal device. Select the projection device through the screen mirroring entry provided by the operating system of the terminal device, and then synchronously project the content displayed on the upper half screen to the projection device (monitor or large screen device). Through the projection device, multiple people can jointly view the meeting materials. At the same time, the lower half screen of the terminal device can still provide prompt materials for the user or the user can record the meeting content in the lower half screen.

[0074] In a home entertainment scenario, the user can place content such as videos, pictures, and games to be shared and played on the first display screen (upper half screen) of the terminal device, and place some instant messaging (IM) message software in the lower half screen. Select the projection device through the screen mirroring entry provided by the operating system of the terminal device, and synchronously project the upper half screen of the terminal device to the projection device (monitor or large screen device). Through the projection device, multiple people can jointly view audio and video information. At the same time, the lower half screen of the terminal device is still available for the user to view or process some IM messages.

[0075] It should be noted that a screen mirroring method proposed by an embodiment of the present application is not only applicable to Figure 5 the application scenarios shown in the figure, but also applicable to any scenario with a screen mirroring requirement.

[0076] After introducing the application scenarios of a screen mirroring method provided by an embodiment of the present application, the system architecture of a screen mirroring method provided by an embodiment of the present application will be introduced below. As Figure 6 shown in the figure, where:

[0077] The terminal device interacts with the user through the desktop. The PC operating system in the terminal device discovers the target device that supports screen mirroring in the near field through the service discovery protocol, and displays the target device list on the desktop. Determine the projection device based on the user's selection, and establish a two-way trusted channel through device authentication; the terminal device records and encodes the local screen, and transmits the encoded video stream through the wireless communication (WIFI) peer-to-peer (P2P) transmission channel to the projection device; the graphics subsystem in the terminal device provides virtual dual-Display capabilities. When the terminal device is in the folded state, the upper and lower half screens are virtualized into two logical screens. The graphics subsystem provides screen mirroring scenario area encoding capabilities and frame rate adaptation capabilities. The projection device decodes the received video stream and then renders it to the local window or screen to complete the screen mirroring operation.

[0078] For ease of understanding, the following takes the upper half screen of the terminal device as the first display screen and the lower half screen as the second display screen as an example for illustration. As Figure 7 shown, it is a schematic diagram of the overall process of a screen mirroring method provided by an embodiment of the present application, where:

[0079] In step 701: In response to a screen mirroring instruction, create a first display corresponding to the first display screen and create a second display corresponding to the second display screen.

[0080] In the embodiment of the present application, when the user has a screen mirroring requirement, a screen mirroring instruction can be triggered in the terminal device. Based on this screen mirroring instruction, the terminal device determines its current state. When it is determined that the terminal device is in a folded state, displays corresponding to the first display screen and the second display screen are respectively created.

[0081] In some possible embodiments, determining whether the terminal device is in a folded state can be specifically implemented as: determining the folding angle between the first display screen and the second display screen; determining that the terminal device is in a folded state according to the folding angle.

[0082] In the embodiment of the present application, as Figure 8 shown, the folding angle is the angle between the first display screen and the second display screen. When calculating the folding angle, it can be calculated through the sensor signals of the sensors in the detected terminal device. The specific calculation process is the same as that in the related art and will not be elaborated here.

[0083] After determining the folding angle, it can be determined whether the terminal device is in a folded state according to the relationship between the folding angle and a preset angle threshold; if the folding angle is greater than the first angle threshold and less than the second angle threshold, it is determined that the terminal device is in a folded state; if the folding angle is greater than or equal to the second angle threshold and less than or equal to the third angle threshold, it is determined that the terminal device is in an unfolded state; if the folding angle is less than or equal to the first angle threshold, it is determined that the terminal device is in a closed state.

[0084] Among them, the first angle threshold can be set to 30 degrees (°), the second angle threshold can be set to 150°, and the third angle threshold can be set to 180°.

[0085] It should be noted that the values of the first angle threshold, the second angle threshold, and the third angle threshold given above in the present application are only for ease of illustration and do not limit the specific values of the first angle threshold, the second angle threshold, and the third angle threshold. In specific implementation, the specific values of the first angle threshold, the second angle threshold, and the third angle threshold can be set according to the implementation situation.

[0086] In some possible embodiments, when determining whether to implement the above step 701, it can be specifically implemented asFigure 9 The process shown, where: First, a sensor signal is obtained through an acceleration sensor, and then the sensor signal is processed by the Basic Input Output System (BIOS) and the Embedded Controller (EC). The folding angle is determined based on the processed sensor signal. If it is determined that the folding angle is greater than the first angle threshold and less than the second angle threshold, it is determined that the terminal device is in the folded state. At this time, a first display function display corresponding to the first display screen needs to be created, and a second display corresponding to the second display screen needs to be created. If it is determined that the folding angle is greater than the second angle threshold and less than or equal to the third angle threshold, it is determined that the terminal is in the unfolded state. At this time, the first display screen and the second display screen correspond to one display.

[0087] In step 702: A screen mirroring prompt is displayed on the terminal device.

[0088] In an embodiment of the present application, after the user triggers a screen mirroring instruction, the PC operating system in the terminal device discovers a target device that supports screen mirroring in the near field through the service discovery protocol, and displays a list of target devices on the desktop. At the same time, a list of screen mirroring areas is displayed on the desktop. The user can select a target projection device and a target display screen based on the list of target devices and the list of screen mirroring areas.

[0089] For example: As Figure 10 shown, a list of target devices and a list of screen mirroring areas are displayed on the first display screen of the terminal device. The list of target devices includes: Device 1, Device 2, Device 3, Device 4, and Device 5. The list of screen mirroring areas includes: the upper half screen, the lower half screen, and the full screen. If the user selects Device 3 and the upper half screen, then Device 3 is used as the target projection device, and the upper half screen is used as the target display screen.

[0090] In step 703: Determine the target projection device and the target display screen based on the screen mirroring prompt.

[0091] In an embodiment of the present application, after presenting the Figure 10 shown list of target devices and list of screen mirroring areas to the user, the device and the screen mirroring area selected by the user in the list of target devices and the list of screen mirroring areas are used as the target projection device and the target display screen.

[0092] In step 704: Based on the display corresponding to the target display screen, mirror the interface of the target display screen to the target projection device.

[0093] In an embodiment of the present application, after determining the target display screen, the interface of the target display screen can be sent to the target projection device based on the display corresponding to the target display screen, so that the target projection device can display it.

[0094] For the sake of further understanding, the processes of creating the first display corresponding to the first display screen and the second display corresponding to the second display screen in step 701 above will be described in detail below:

[0095] In some possible embodiments, creating the first display function display corresponding to the first display screen can be specifically implemented as follows: determining the first screen width of the first display screen; adjusting the height coordinate parameter in the initial display of the terminal device according to the first screen height to obtain the first display.

[0096] That is, in the embodiments of the present application, the parameters of the initial display corresponding to the terminal device are adjusted, and the adjusted initial display is used as the first display corresponding to the first display screen. By adjusting the parameters of the initial display, it is ensured that the obtained first display can accurately represent the size of the first display screen, ensuring the accuracy of subsequent screen mirroring operations.

[0097] As Figure 11 shown, when the terminal device is in the unfolded state, the screen size parameters in the initial display represent the width (w) and height (h) of the entire screen of the terminal device; a two-dimensional coordinate system is established with the upper left vertex of the screen as the origin, then the coordinates of the upper left vertex can be marked as (0, 0), and the coordinates of the lower right vertex are (w, h). At this time, the screen size parameters in the initial display are (0, 0)(w, h); after the terminal device is adjusted to the folded state, the width of the first display screen is w at this time, and the height is half of the original height, that is, h / 2. At this time, the coordinates of the upper left vertex are (0, 0), and the coordinates of the lower right vertex are (w, h / 2). By adjusting the screen size parameters in the initial display, the screen size parameters in the obtained first display are (0, 0), (w, h / 2).

[0098] In some other possible embodiments, creating the second display corresponding to the second display screen can be specifically implemented as follows: step of determining the second screen height and the second screen width of the second display screen; generating a first coordinate parameter according to the second screen height, and generating a second coordinate parameter according to the second screen width; determining the second display according to the first coordinate parameter and the second coordinate parameter.

[0099] That is, in the embodiments of the present application, when it is determined that the screen is in the folded state, it is necessary to generate screen size parameters according to the screen size of the second display screen, and then obtain the second display. By creating the second display, one display corresponds to each display screen, and then it is realized that only the first display screen or the second display screen can be projected according to the user's selection in the subsequent process.

[0100] As Figure 11 shown, when the terminal device is in the unfolded state, the screen size parameters in the initial display identify the width (w) and height (h) of the entire screen of the terminal device; taking the upper left vertex of the screen as the origin to establish a two-dimensional coordinate system, the coordinates of the upper left vertex can be identified as (0, 0), and the coordinates of the lower right vertex are (w, h). At this time, the screen size parameters in the initial display are (0, 0)(w, h); after adjusting the terminal device to the folded state, the width of the second display screen is w, and the height is half of the original height, that is, h / 2. At this time, the coordinates of the upper left vertex, that is, the first coordinate parameter, are (0, h / 2), and the coordinates of the lower right vertex, that is, the second coordinate parameter, are (w, h). After obtaining the first coordinate parameter and the second coordinate parameter, the second display can be generated, and the screen size parameters of the generated second display are (0, h / 2), (w, h).

[0101] In some possible embodiments, in order to ensure the frame rate of the projection area, a local refresh method is set, that is, after implementing the Figure 7 steps shown, the following can be implemented: determining a refresh area in the target display screen according to the first frame of data, and sending the display data corresponding to the refresh area to the target projection device; the first frame of data is the next frame of data of the current frame of data being displayed in the target display screen.

[0102] That is, in the embodiments of the present application, when it is determined that there is a refresh area in the target display screen, only the refresh area is sent to the target projection device, and it is not necessary to send all the content of the target display screen to the target projection device. By means of local refresh, the load is reduced and the load of the projection area is ensured.

[0103] For example: as Figure 12A shown, the target display screen is the second display screen, and the content of the second display screen is displayed in the target projection device. If the refresh area is determined to be area A according to the current frame of data and the first frame of data, then area A is sent to the target projection device; or, as Figure 12B shown, the target display screen is the first display screen, and the content of the first display screen is displayed in the target projection device. If the refresh area is determined to be area B according to the current frame of data and the first frame of data, then area B is sent to the target projection device.

[0104] In some possible embodiments, when the graphic resources of the operating system of the terminal device are limited, it is necessary to prioritize ensuring the frame rate of the target display screen, so as to ensure clear image output. Therefore, the frame rate of the non-target display screen can be appropriately reduced, that is, as Figure 13 shown, first, the frame rate decision module is used to determine whether it is necessary to adjust the load of the non-target display screen. When adjustment is required, the frame rate adjustment module is used to reduce the frame rate of the non-target display screen to ensure the frame rate of the target display screen. Specifically, it can be implemented as Figure 14 shown in the process, where:

[0105] In step 1401: Obtain the comprehensive load of the terminal device.

[0106] In the embodiments of the present application, when the computing power of the comprehensive load of the terminal device is sufficient, there is no need to reduce the frame rate of the non-target display screen. Only when the computing power is insufficient, the frame rate of the non-target display screen is reduced. Therefore, it is first necessary to determine the current comprehensive load of the terminal device.

[0107] In some possible embodiments, after obtaining the comprehensive load, it is possible to determine whether it is necessary to adjust the frame rate of the non-target display screen and the adjustment level according to the comprehensive load and the preset gear adjustment table.

[0108] For example: The preset gear adjustment table is as Figure 15 shown. If the determined current comprehensive load is 80, it can be determined that the computing power is insufficient at this time and the load needs to be increased by one gear.

[0109] In step 1402: Determine that it is necessary to adjust the frame rate of the non-target display screen according to the comprehensive load, and determine the frame rate percentage of the non-target display screen.

[0110] Among them, if the target display screen is the first display screen, the non-target display screen is the second display screen; if the target display screen is the second display screen, the non-target display screen is the first display screen.

[0111] In the embodiments of the present application, considering that during the process of the user using the terminal device for screen mirroring operation, the distance between the human eye and the screen of the terminal device is relatively fixed (or fluctuates within a small range), the distance between the non-target display screen (focus window) and the target display screen (non-focus window) can be used to identify the viewing angle of the human eye. Therefore, in the present application, a frame rate percentage is set based on this, and the target frame rate of the non-target display screen is determined according to the calculated frame rate percentage.

[0112] In some possible embodiments, determining the frame rate percentage of the non-target display screen can be specifically implemented as the steps Figure 16 shown, where:

[0113] In step 1601: Determine the focal distance between the first focal point of the target display screen and the second focal point of the non-target display screen.

[0114] In an embodiment of the present application, the first focal point is the center point of the target display screen, and the second focal point is the center point of the non-target display screen. According to the coordinates of the first focal point and the coordinates of the second focal point, the focal distance between the first focal point and the second focal point can be obtained, and the focal distance is denoted as x.

[0115] In step 1602: Obtain the physical distance between the first focal point and the second focal point according to the focal distance and the pixel density.

[0116] In an embodiment of the present application, the pixel density is the pixel density of the terminal device, and the ratio of the focal distance to the pixel density is used as the physical distance between the first focal point and the second focal point.

[0117] For example: The focal distance is denoted as x, the pixel density is denoted as ppi, then the physical distance is x1 = x / ppi.

[0118] In step 1603: Determine the frame rate calculation function according to the physical distance and the preset distance threshold.

[0119] That is, in the present application, according to the relationship between the non-target display screen, the target display screen and the human eye attention, the frame rate calculation function as shown Figure 17 is set. 3.44 is used as the demarcation point of the two frame rate calculation functions. When x1 is less than 3.44, the frame rate calculation function is the first frame rate calculation function. When x1 is greater than or equal to 3.44, the frame rate calculation function is the second frame rate calculation function.

[0120] Exemplarily, the first frame rate calculation function can be y = -0.11 * x1 + 1.0038, and the second frame rate calculation function can be y = -0.0315 * x1 + 0.7429.

[0121] It can be understood that the specific data of the above-mentioned demarcation point and the frame rate function are only for illustrative purposes, and do not limit the demarcation point and the frame rate function.

[0122] In step 1604: Obtain the frame rate percentage according to the frame rate calculation function and the physical distance.

[0123] In an embodiment of the present application, after obtaining the physical distance and the frame rate calculation function, substitute the physical distance into the corresponding frame rate calculation function to obtain the frame rate percentage.

[0124] In step 1403: Determine the target frame rate according to the frame rate percentage.

[0125] In an embodiment of the present application, after obtaining the frame rate percentage, it can be according to Figure 18Determine the target frame rate according to the relationship table between the shown frame rate percentage and the target frame rate.

[0126] For example: First, according to Figure 16 the preset gear adjustment table shown, it is determined that the current comprehensive load is 80. It can be determined that the computing power is insufficient at this time, and the load needs to be increased by 1 gear. If the current gear is determined to be gear 1, the adjusted gear is gear 2. If the frame rate percentage is determined to be 82%, then the target frame rate can be determined to be 30 frames per second (FPS).

[0127] In step 1404: Adjust the frame rate of the non-target display screen to the target frame rate.

[0128] In the embodiment of the present application, after obtaining the target frame rate, the frame rate adjustment value of the non-target display screen is set to the target frame rate, thereby reducing the frame rate of the non-target display screen and reducing the load of the operating system of the terminal device.

[0129] It can be understood that the above description takes the terminal device including the first display screen and the second display screen as an example, but it does not limit the number of display screens of the terminal device. The screen mirroring method provided in the embodiment of the present application is also applicable to terminal devices with multiple display screens. For example: In response to a screen mirroring instruction, create a display corresponding to each display screen and display a screen mirroring prompt on the terminal device; determine the target projection device and the target display screen based on the screen mirroring prompt; project the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

[0130] For the convenience of further understanding, the overall process of a screen mirroring method provided in the embodiment of the present application will be described below, as Figure 19 shown, where:

[0131] In step 1901: In response to a screen mirroring instruction, display a screen mirroring prompt on the terminal device.

[0132] The specific implementation of this step is the same as that of step 702 and will not be elaborated here.

[0133] In step 1902: Determine the target projection device and the target display screen based on the screen mirroring prompt.

[0134] The specific implementation of this step is the same as that of step 703 and will not be elaborated here.

[0135] In step 1903: Determine whether the target display screen is the first display screen or the second display screen. If it is determined that the target display screen is the first display screen or the second display screen, then enter step 1904; otherwise, enter step 1905.

[0136] In step 1904: Create a first display corresponding to the first display screen, and create a second display corresponding to the second display screen.

[0137] The specific implementation of this step is the same as that of step 704, and will not be elaborated here.

[0138] In step 1905: Based on the initial display of the terminal device, project the display interface to the target projection device.

[0139] In step 1906: Determine whether there is a refresh area in the terminal device according to the first frame of data. If it is determined that there is, enter step 1907; otherwise, enter step 1905.

[0140] In step 1907: Send the display data corresponding to the refresh area to the target projection device.

[0141] In step 1908: Based on the display corresponding to the target display screen, project the interface of the target display screen to the target projection device.

[0142] The specific implementation of this step is the same as that of step 701, and will not be elaborated here.

[0143] In step 1909: Determine whether there is a refresh area in the target display screen according to the first frame of data. If there is, enter step 1910; otherwise, enter step 1908.

[0144] In step 1910: Send the display data corresponding to the refresh area to the target projection device.

[0145] In step 1911: Obtain the comprehensive load of the terminal device.

[0146] The specific implementation of this step is the same as that of step 1401, and will not be elaborated here.

[0147] In step 1912: Determine whether it is necessary to adjust the frame rate of the non-target display screen according to the comprehensive load. If it is necessary, enter step 1913; otherwise, enter step 1908.

[0148] The specific implementation of this step is the same as that of step 1401, and will not be elaborated here.

[0149] In step 1913: Determine the frame rate percentage of the non-target display screen.

[0150] The specific implementation of this step is the same as that of step 1402, and will not be elaborated here.

[0151] In step 1914: Determine the target frame rate according to the frame rate percentage.

[0152] The specific implementation of this step is the same as that of step 1403, and will not be elaborated here.

[0153] In step 1915: Adjust the frame rate of the non-target display screen to the target frame rate.

[0154] The specific implementation of this step is the same as that of step 1404, and will not be elaborated here.

[0155] Based on the same inventive concept, an embodiment of the present application further provides a screen mirroring device, as Figure 20 shown, applied to a terminal device, the terminal device includes: a first display screen, a second display screen, and the device includes:

[0156] A response module 20001, a prompt module 20002, a determination module 20003, and a screen mirroring module 20004. The response module 20001 is used to execute step 701 in the above Figure 7 , Figure 19 step 1901 and step 1904 in Figure 7 . The prompt module is used to execute step 702 in the above Figure 19 , Figure 9 step 1902 in Figure 7 , Figure 7 step 1903 in Figure 14 . The determination module is used to execute step 703 in the above Figure 16 . The screen mirroring module 20004 is used to execute step 704 in the above Figure 19 the steps shown in

[0157] and the steps shown in Figure 21 step 1905, step 1906, step 1907, step 1908, step 1909, step 1910, step 1911, step 1912, step 1913, step 1914, step 1915 in

[0158] Among them, the communication unit 2103 is used to establish a communication channel so that the electronic device can communicate with other devices. Receive user data sent by other devices or send user data to other devices.

[0159] The processor 2101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 2102, and invoking data stored in the memory, it executes various functions of the electronic device and / or processes data. The processor may be composed of an integrated circuit (IC). For example, it may be composed of a single packaged IC, or may be composed of multiple packaged ICs with the same or different functions connected together. For example, the processor 2101 may only include a central processing unit (CPU). In the embodiment of the present invention, the CPU may be a single-core processor or may include multiple cores.

[0160] The memory 2102 is used to store the execution instructions of the processor 2101. The memory 2102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0161] When the execution instructions in the memory 2102 are executed by the processor 2101, the electronic device 2100 is enabled to execute Figure 7 Some or all of the steps in the illustrated embodiments.

[0162] In a specific implementation, the present invention further provides a computer storage medium. The computer storage medium may store a program, and when the program is executed, it may include some or all of the steps in the embodiments of the screen mirroring method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0163] In some possible implementation manners, various aspects of a terminal device control method provided in this application may also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in a screen mirroring method according to various exemplary embodiments described above in this specification.

[0164] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0165] The program product for controlling a terminal device according to an embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may run on an electronic device. However, the program product of the present application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0166] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution in the embodiments of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0167] For the same or similar parts among the various embodiments in this specification, reference can be made to each other. In particular, for the device embodiments and the terminal embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A screen mirroring method, characterized in that, Applied to a terminal device, the terminal device includes: a first display screen and a second display screen, and the method includes: In response to a screen mirroring instruction, create a first display function display corresponding to the first display screen, and create a second display corresponding to the second display screen; Display a screen mirroring prompt on the terminal device; Determine a target projection device and a target display screen based on the screen mirroring prompt; Mirror the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

2. The method according to claim 1, wherein The creating the first display corresponding to the first display screen includes: Determine the first screen width of the first display screen; Adjust the height coordinate parameter in the initial display of the terminal device according to the first screen height to obtain the first display.

3. The method according to claim 1, wherein The creating the second display corresponding to the second display screen includes: Determine the second screen height and the second screen width of the second display screen; Generate a first coordinate parameter according to the second screen height, and generate a second coordinate parameter according to the second screen width; Determine the second display according to the first coordinate parameter and the second coordinate parameter.

4. The method according to claim 1, wherein The method further includes: Obtain the comprehensive load of the terminal device; Determine that the frame rate of the non-target display screen needs to be adjusted according to the comprehensive load, and determine the frame rate percentage of the non-target display screen; wherein, if the target display screen is the first display screen, the non-target display screen is the second display screen, and if the target display screen is the second display screen, the non-target display screen is the first display screen; Determine the target frame rate according to the frame rate percentage; Adjust the frame rate of the non-target display screen to the target frame rate.

5. The method according to claim 4, characterized in that, The determining the frame rate percentage of the non-target display screen includes: Determine the focal distance between the first focus of the target display screen and the second focus of the non-target display screen; Obtain the physical distance between the first focus and the second focus according to the focal distance and the pixel density; Determine a frame rate calculation function according to the physical distance and a preset distance threshold; Obtain the frame rate percentage according to the frame rate calculation function and the physical distance.

6. The method according to claim 1, wherein After the screen mirroring operation is performed on the target projection device based on the display corresponding to the target display screen, the method further includes: Determine that there is a refresh area in the target display screen according to the first frame data, and send the display data corresponding to the refresh area to the target projection device; the first frame data is the next frame data of the current frame data being displayed on the target display screen.

7. The method according to claim 1, characterized in that, Before the creating the first display corresponding to the first display screen and creating the second display corresponding to the second display screen, the method further includes: Determine the folding angle between the first display screen and the second display screen; Determine that the terminal device is in a folded state according to the folding angle.

8. A screen mirroring device, characterized in that, Applied to a terminal device, the terminal device includes: a first display screen and a second display screen, and the apparatus includes: A response module, configured to create a first display corresponding to the first display screen and create a second display corresponding to the second display screen in response to a screen mirroring instruction; A prompt module, configured to display a screen mirroring prompt on the terminal device; A determination module, configured to determine a target projection device and a target display screen based on the screen mirroring prompt; A screen mirroring module, configured to mirror the interface of the target display screen to the target projection device based on the display corresponding to the target display screen.

9. An electronic device, characterized in that, It includes a memory for storing computer program instructions and a processor for executing the program instructions. Wherein, when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein when the program runs, it controls the device where the computer-readable storage medium is located to execute the method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes: computer program code, which when running on a computer, causes the computer to execute the method according to any one of claims 1-7 above.

Images