A screen projection method, device, apparatus and storage medium

Abstract

A screen projection method, device, equipment and storage medium are disclosed, which are applied to the technical field of terminal products. The method is used for implementing partition projection of a folding screen. In the embodiment of the present application, for a terminal device with a first display screen and a second display screen, a first display function display corresponding to the first display screen is created, and a second display corresponding to the second display screen is created in response to a screen projection instruction; a screen projection prompt is displayed on the terminal device; a target projection device and a target display screen are determined based on the screen projection prompt; and an interface of the target display screen is projected to the target projection device based on the display corresponding to the target display screen. In the embodiment of the present application, a corresponding display is set for each display screen, so that the first display screen or the second display screen can be projected according to the screen projection demand of a user, and the individualized screen projection demand of the user is realized.

Description

Technical Field

[0001] This invention relates to the field of terminal technology, and in particular to a screen projection method, apparatus, device, and storage medium. Background Technology

[0002] With the development of application display technology, the functions of terminal devices such as laptops are becoming increasingly rich, and more and more users are using mobile devices to watch videos, browse files, and so on. However, sometimes due to practical needs, it is necessary to project the content displayed on the terminal device onto a large screen / smart screen or monitor for convenient viewing.

[0003] Conventional laptops and other terminal devices have a screen on the top half and a keyboard on the bottom half, with Windows or macOS as the operating system. Users can wirelessly mirror the physical internal screen to large screens / smart screens, monitors, and other devices through the screen mirroring interface provided by the system. The large screens / smart screens, monitors, and other devices display the same content as the laptop.

[0004] However, for foldable full-screen devices, if the screen mirroring method used for laptops is applied, large-screen / smart screens, monitors, and other devices can only project the entire screen, resulting in poor display quality. Summary of the Invention

[0005] The purpose of this invention is to provide a screen projection method, apparatus, device, and storage medium to solve the technical problems in the prior art.

[0006] In a first aspect, embodiments of this application provide a screen casting method applied to a terminal device, the terminal device including: a first display screen and a second display screen, the method including: in response to a screen casting command, creating a first display screen corresponding to the first display screen and creating a second display screen corresponding to the second display screen; displaying a screen casting prompt on the terminal device; determining a target projection device and a target display screen based on the screen casting prompt; and casting the interface of the target display screen to the target projection device based on the display screen corresponding to the target display screen.

[0007] In this embodiment, a corresponding display is set for each display screen, so that the projection can be made only to the first display screen or the second display screen according to the user's projection needs, thus realizing the user's personalized projection needs.

[0008] In some possible embodiments, creating a first display corresponding to the first display screen includes: determining the first screen width of the first display screen; and adjusting the height coordinate parameters in the initial display of the terminal device according to the first screen height to obtain the first display screen.

[0009] In this embodiment of the application, by adjusting the parameters of the initial display, it is ensured that the obtained first display can accurately represent the size of the first display screen, thus ensuring the accuracy of subsequent screen projection operations.

[0010] In some possible embodiments, creating a second display corresponding to the second display screen includes: determining the second screen height and the second screen width of the second display screen; generating a first coordinate parameter based on the second screen height and generating a second coordinate parameter based on the second screen width; and determining the second display screen based on the first coordinate parameter and the second coordinate parameter.

[0011] In this embodiment, by creating a second display, each display screen corresponds to one display screen, thereby enabling subsequent screen projection operations to be performed only on the first display screen or the second display screen according to the user's selection.

[0012] In some possible embodiments, the method further includes: obtaining the overall load of the terminal device; determining, based on the overall load, that the frame rate of the non-target display needs to be adjusted, and determining the frame rate percentage of the non-target display; wherein, if the target display is a first display, then the non-target display is a second display, and if the target display is a second display, then the non-target display is a first display; determining the target frame rate based on the frame rate percentage; and adjusting the frame rate of the non-target display to the target frame rate.

[0013] In this embodiment, when the overall computing power of the terminal device is sufficient, there is no need to reduce the frame rate of the non-target display screen. The frame rate of the non-target display screen is reduced only when the computing power is insufficient, thereby reducing the load on the terminal device's operating system.

[0014] In some possible embodiments, determining the frame rate percentage of a non-target display includes: determining the focal distance between a first focal point of the target display and a second focal point of the non-target display; obtaining the physical distance between the first and second focal points based on the focal distance and pixel density; determining a frame rate calculation function based on the physical distance and a preset distance threshold; and obtaining the frame rate percentage based on the frame rate calculation function and the physical distance.

[0015] In this embodiment of the application, the frame rate percentage is determined by setting the focus distance, thereby reducing the load on the terminal device while ensuring a good user experience.

[0016] In some possible embodiments, after the projection operation is performed on the target projection device based on the display corresponding to the target display screen, the method further includes: determining that there is a refresh area in the target display screen according to the first frame data, and sending the display data corresponding to the refresh area to the target projection device; the first frame data is the next frame data of the current frame data being displayed in the target display screen.

[0017] In this embodiment, the load on the terminal device is reduced by partial refresh.

[0018] In some possible embodiments, before creating the first display function corresponding to the first display screen and creating the second display function corresponding to the second display screen, the method further includes: determining the folding angle between the first display screen and the second display screen; and determining that the terminal device is in a folded state based on the folding angle.

[0019] In this embodiment, the state of the terminal device is determined by calculating the folding angle between the first display screen and the second display screen, thus ensuring the accuracy of the determined state of the terminal device.

[0020] Secondly, embodiments of this application provide a screen projection device applied to a terminal device. The terminal device includes a first display screen and a second display screen. The device includes: a response module, used to respond to a screen projection command, create a first display screen corresponding to the first display screen, and create a second display screen corresponding to the second display screen; a prompt module, used to display a screen projection prompt on the terminal device; a determination module, used to determine the target projection device and the target display screen based on the screen projection prompt; and a screen projection module, used to project the interface of the target display screen to the target projection device based on the display screen corresponding to the target display screen.

[0021] In some possible embodiments, the response module is specifically configured to: determine the first screen width of the first display screen; and adjust the height coordinate parameters in the initial display of the terminal device according to the first screen height to obtain the first display.

[0022] In some possible embodiments, the response module is specifically configured to: determine the second screen height and the second screen width of the second display; generate a first coordinate parameter based on the second screen height and a second coordinate parameter based on the second screen width; and determine the second display based on the first coordinate parameter and the second coordinate parameter.

[0023] In some possible embodiments, the screen projection module is further configured to: obtain the overall load of the terminal device; determine the frame rate of the non-target display screen that needs to be adjusted based on the overall load, and determine the frame rate percentage of the non-target display screen; wherein, if the target display screen is the first display screen, then the non-target display screen is the second display screen, and if the target display screen is the second display screen, then the non-target display screen is the first display screen; determine the target frame rate based on the frame rate percentage; and adjust the frame rate of the non-target display screen to the target frame rate.

[0024] In some possible embodiments, the projection module is specifically used to: determine the focal distance between the first focal point of the target display screen and the second focal point of the non-target display screen; obtain the physical distance between the first focal point and the second focal point based on the focal distance and pixel density; determine the frame rate calculation function based on the physical distance and a preset distance threshold; and obtain the frame rate percentage based on the frame rate calculation function and the physical distance.

[0025] In some possible embodiments, the projection module is further configured to: determine that there is a refresh area in the target display screen based on the first frame data, and send the display data corresponding to the refresh area to the target projection device; the first frame data is the next frame data of the current frame data being displayed in the target display screen.

[0026] In some possible embodiments, the response module is also configured to: determine the folding angle between the first display screen and the second display screen; and determine that the terminal device is in a folded state based on the folding angle.

[0027] Thirdly, another embodiment of this application also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the methods provided in the first or second aspect of this application.

[0028] Fourthly, another embodiment of this application provides a computer-readable storage medium storing a computer program for causing a computer to perform any of the methods provided in the first or second aspect of this application.

[0029] Fifthly, another embodiment of this application also provides a computer program product, characterized in that the computer program product includes: computer program code, which, when run on a computer, causes the computer to perform any of the methods provided in the first or second aspect of the embodiments described above.

[0030] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a conventional laptop computer screen mirroring device provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of a foldable screen device provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram of foldable screen projection in the related technology provided in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of the screen projection process in the related technologies provided in the embodiments of this application;

[0035] Figure 5 This is a schematic diagram illustrating an application scenario of a screen projection method provided in an embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the system architecture of a screen projection method provided in an embodiment of this application;

[0037] Figure 7 A flowchart illustrating a screen mirroring method provided in an embodiment of this application;

[0038] Figure 8 A schematic diagram of the folding angle of a screen projection method provided in an embodiment of this application;

[0039] Figure 9 This is a schematic diagram illustrating a screen projection method provided in this application, which determines whether the screen is in a folded state based on the folding angle.

[0040] Figure 10 A schematic diagram illustrating a target device list and a screen projection area list for a screen projection method provided in this application embodiment;

[0041] Figure 11 A schematic diagram of the screen size of a terminal device for a screen projection method provided in an embodiment of this application;

[0042] Figure 12A This is a schematic diagram illustrating a screen projection method provided in this application, in which the refresh area of ​​a second display screen is sent to a target projection device.

[0043] Figure 12B This is a schematic diagram illustrating a screen projection method provided in this application, in which the refresh area of ​​a first display screen is sent to a target projection device.

[0044] Figure 13 A schematic diagram illustrating a screen mirroring method for reducing frame rate, provided in an embodiment of this application;

[0045] Figure 14 A flowchart illustrating a screen mirroring method for reducing frame rate, provided in an embodiment of this application;

[0046] Figure 15 This application provides a schematic representation of the preset level adjustment of a screen projection method according to an embodiment of the present application.

[0047] Figure 16A flowchart illustrating the process of determining the frame rate percentage of a non-target display screen in a screen projection method provided in this application embodiment;

[0048] Figure 17 A schematic diagram of a frame rate calculation function for a screen projection method provided in this application embodiment;

[0049] Figure 18 This application provides an embodiment of a screen projection method that illustrates the relationship between the frame rate percentage and the target frame rate.

[0050] Figure 19 This is a schematic diagram of the overall process of a screen projection method provided in an embodiment of this application;

[0051] Figure 20 A schematic diagram of an apparatus for a screen projection method provided in an embodiment of this application;

[0052] Figure 21 This is a schematic diagram of an electronic device for a screen projection method provided in an embodiment of this application. Detailed Implementation

[0053] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0055] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0056] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0058] With the development of application display technology, the functions of terminal devices such as laptops are becoming increasingly rich, and more and more users are using mobile devices to watch videos, browse files, and so on. However, sometimes due to practical needs, it is necessary to project the content displayed on the terminal device onto a large screen / smart screen or monitor for convenient viewing.

[0059] like Figure 1 As shown, the upper half of a typical laptop or similar device is a screen, and the lower half is a keyboard. The operating system is Windows or macOS. Users can wirelessly mirror the physical internal screen to a large screen / smart screen, monitor, or other device through the screen mirroring interface provided by the system. The large screen / smart screen, monitor, or other device displays the same content as the laptop.

[0060] For example Figure 2 The foldable full-screen terminal device shown has a single, physical flexible screen that can be folded inwards to present two different screens, one above and one below. Users can operate their respective desktops and windows on different screens, enabling partitioned management.

[0061] If using conventional laptop screen mirroring methods, the screen size ratio and resolution of the terminal device differ significantly from those of large-screen / smart screens and monitors. Furthermore, the projection device does not support changes in the terminal device's orientation, such as folding, unfolding, or rotating. Figure 3 As shown, the terminal device can only project the entire screen. It cannot project only a portion of the terminal device's screen, resulting in poor display quality.

[0062] In related technologies, the PC operating system of the sending end (terminal device) provides a pre-built screen mirroring entry and casting capabilities, such as Windows based on the Miracast protocol and Mac based on the AirPlay protocol for screen casting. However, both of these methods encode the entire content of the sending end's internal screen and then project it onto the receiving end for display. The specific screen casting process, such as... Figure 4 As shown, it mainly includes the following steps:

[0063] 1. Device discovery process: The sending end's operating system provides a screen mirroring entry point. Based on the screen mirroring operation triggered by the user, the sending end's operating system sends a broadcast to surrounding devices based on the relevant screen mirroring protocol. Through the broadcast, it discovers devices that support screen mirroring. After receiving a reply from the devices that can be mirrored, the sending end's operating system displays a list of devices that can be mirrored.

[0064] 2. Device connection authentication / screen projection process: The user selects the target device from the list of devices that can be projected. The user-selected target device is used as the receiving end. A communication connection is established with the receiving end based on the wireless transmission protocol, and authentication and networking are performed.

[0065] 3. Capability Negotiation Process: The sending end and the receiving end transmit their respective capabilities based on a secure transmission link, including but not limited to: screen resolution, dots per inch (DPI), and related software version capabilities, and negotiate the corresponding parameter information for dual-end encoding and decoding.

[0066] 4. Receiver window registration process: Based on the screen resolution, DPI and other information negotiated between the two ends, the receiver registers a window Surface with the screen projection protocol to receive and display the video stream.

[0067] 5. Audio and video stream transmission process: The sending end creates a virtual display function (Display) based on the screen resolution, DPI and other information negotiated between the two ends, and records the physical internal screen desktop onto the virtual Display. The video is encoded and packaged by the screen projection protocol, and the local audio stream data is packaged at the same time. After being decoded by the screen projection protocol transmitted to the receiving end through the wireless transmission link, it is sent to the corresponding window of the receiving end for display.

[0068] use Figure 4 The screen mirroring process shown can only project the entire physical screen. If the user is using a foldable screen, it cannot adapt to the desktop and window partitioning after the physical screen is folded, thus failing to meet the user's personalized screen mirroring needs.

[0069] To address the aforementioned problems, embodiments of this application provide a screen casting method, apparatus, device, and storage medium to solve these problems. The inventive concept of this application can be summarized as follows: For a terminal device having a first display screen and a second display screen, in response to a screen casting command, a first display corresponding to the first display screen is created, and a second display corresponding to the second display screen is created; a screen casting prompt is displayed on the terminal device; a target projection device and a target display screen are determined based on the screen casting prompt; and the interface of the target display screen is cast to the target projection device based on the display corresponding to the target display screen.

[0070] In this embodiment, a corresponding display is set for each display screen, so that the projection can be made only to the first display screen or the second display screen according to the user's projection needs, thus realizing the user's personalized projection needs.

[0071] For ease of understanding, the screen projection method provided in this application embodiment will be described in detail below with reference to the accompanying drawings:

[0072] First, the screen projection method provided in this application is implemented by a terminal device, which is a foldable flexible screen, including: a first display screen and a second display screen; as shown... Figure 5The diagram shown illustrates an application scenario of a screen mirroring method provided in this application embodiment, wherein:

[0073] In a meeting presentation scenario, users can move the presentation materials to be presented and discussed to the first display screen (upper half screen) of the terminal device, and place some prompting materials on the second display screen (lower half screen) of the terminal device. They can then select the projection device through the screen projection entry provided by the terminal device's operating system, and then simultaneously project the content displayed on the upper half screen onto the projection device (monitor or large screen device). Multiple people can view the meeting materials together through the projection device. At the same time, the lower half screen of the terminal device can still provide prompting materials for users or allow users to record meeting content on the lower half screen.

[0074] In home entertainment scenarios, users can place videos, pictures, games, and other content to be shared on the first display screen (upper half screen) of the terminal device, and place instant messaging (IM) software on the lower half screen. By selecting the projection device through the screen mirroring entry provided by the terminal device's operating system, the upper half screen of the terminal device can be synchronously projected onto the projection device (monitor or large screen device). Through the projection device, multiple people can watch audio and video information together. At the same time, the lower half screen of the terminal device can still be used by users to view or process some IM messages.

[0075] It should be noted that the screen projection method proposed in this application is not only applicable to... Figure 5 The application scenarios shown are applicable to any scenario where screen mirroring is required.

[0076] After introducing the application scenarios of the screen projection method provided in the embodiments of this application, the system architecture of the screen projection method provided in the embodiments of this application will be described below, such as... Figure 6 As shown, where:

[0077] The terminal device interacts with the user through the desktop. The PC operating system in the terminal device discovers target devices that support screen projection in the near field through the service discovery protocol and displays a list of target devices on the desktop. Based on the user's selection, the projection device is determined, and a trusted dual-end channel is established through device authentication. The terminal device records and encodes its local screen and transmits the encoded video stream to the projection device through a wireless communication (WIFI) peer-to-peer lending (P2P) transmission channel. The graphics subsystem in the terminal device provides virtual dual-display capability. When the terminal device is in a folded state, the upper and lower halves of the screen are virtually converted into two logical screens. The graphics subsystem provides screen projection scene area encoding capability and frame rate adaptive capability. The projection device decodes the received video stream, renders it, and then projects it onto the local window or screen to complete the screen projection operation.

[0078] For ease of understanding, the following explanation will use the upper half of the terminal device's screen as the first display screen and the lower half as the second display screen as an example. Figure 7 The diagram shown is a schematic representation of the overall process of a screen projection method provided in an embodiment of this application, wherein:

[0079] In step 701: In response to the screen projection command, a first display corresponding to the first display screen is created, and a second display corresponding to the second display screen is created.

[0080] In this embodiment of the application, when a user has a screen casting requirement, a screen casting command can be triggered in the terminal device. Based on the screen casting command, the terminal device determines the current state. When it is determined that the terminal device is in a folded state, displays corresponding to the first display screen and the second display screen are created respectively.

[0081] In some possible embodiments, determining whether the terminal device is in a folded state can be specifically implemented by: determining the folding angle between the first display screen and the second display screen; and determining that the terminal device is in a folded state based on the folding angle.

[0082] In the embodiments of this application, such as Figure 8 As shown, the folding angle is the angle between the first display screen and the second display screen. The folding angle can be calculated by the sensor signal of the sensor in the terminal device. The specific calculation process is the same as in related technologies and will not be described in detail here.

[0083] After determining the folding angle, the terminal device can be determined to be in a folded state based on the relationship between the folding angle and a preset angle threshold. If the folding angle is greater than the first angle threshold and less than the second angle threshold, the terminal device is determined to be in a folded state. If the folding angle is greater than or equal to the second angle threshold and less than or equal to the third angle threshold, the terminal device is determined to be in an unfolded state. If the folding angle is less than or equal to the first angle threshold, the terminal device is determined to be in a closed state.

[0084] The first angle threshold can be set to 30 degrees (°), the second angle threshold can be set to 150°, and the third angle threshold can be set to 180°.

[0085] It should be noted that the values ​​of the first angle threshold, the second angle threshold, and the third angle threshold given above in this application are only for illustrative purposes and are not intended to limit the specific values ​​of the first angle threshold, the second angle threshold, and the third angle threshold. In specific implementation, the specific values ​​of the first angle threshold, the second angle threshold, and the third angle threshold can be set according to the implementation situation.

[0086] In some possible embodiments, determining whether to perform step 701 can be specifically implemented as follows: Figure 9 The process is as follows: First, an accelerometer is used to acquire sensor signals. Then, the sensor signals are processed by the Basic Input Output System (BIOS) and the Embedded Controller (EC). The folding angle is determined based on the processed sensor signals. If the folding angle is greater than a first angle threshold and less than a second angle threshold, the terminal device is determined to be in a folded state. At this time, a first display function corresponding to the first display screen and a second display function corresponding to the second display screen need to be created. If the folding angle is greater than the second angle threshold and less than or equal to a third angle threshold, the terminal is determined to be in an unfolded state. At this time, the first display screen and the second display screen correspond to one display.

[0087] In step 702: Display a screen mirroring prompt on the terminal device.

[0088] In this embodiment of the application, after the user triggers the screen casting command, the PC operating system in the terminal device discovers the target devices that support screen casting in the near field through the service discovery protocol, and displays a list of target devices on the desktop, as well as a list of screen casting areas on the desktop. The user can select the target projection device and the target display screen according to the list of target devices and the list of screen casting areas.

[0089] For example: Figure 10 As shown, the target device list and projection area list are displayed on the first display screen of the terminal device. The target device list includes: device 1, device 2, device 3, device 4 and device 5. The projection area list includes: upper half screen, lower half screen and full screen. If the user selects device 3 and the upper half screen, then device 3 will be used as the target projection device and the upper half screen will be used as the target display screen.

[0090] In step 703: the target projection device and the target display screen are determined based on the projection prompts.

[0091] In this embodiment of the application, when showing to the user Figure 10 After displaying the target device list and projection area list, the device and projection area selected by the user in the target device list and projection area list will be used as the target projection device and target display screen.

[0092] In step 704: The interface of the target display screen is projected onto the target projection device based on the display corresponding to the target display screen.

[0093] In this embodiment of the application, after the target display screen is determined, the interface of the target display screen can be sent to the target projection device based on the display corresponding to the target display screen, so that the target projection device can display it.

[0094] To facilitate further understanding, the process of creating the first display corresponding to the first display screen and the second display screen corresponding to the second display screen in step 701 above will be explained in detail below:

[0095] In some possible embodiments, a first display function corresponding to the first display screen is created, which can be implemented as follows: determining the first screen width of the first display screen; adjusting the height coordinate parameters in the initial display of the terminal device according to the first screen height to obtain the first display.

[0096] In this embodiment, the parameters of the initial display corresponding to the terminal device are adjusted, and the initial display with adjusted parameters is used as the first display corresponding to the first display screen. By adjusting the parameters of the initial display screen, it is ensured that the obtained first display screen can accurately represent the size of the first display screen, thus ensuring the accuracy of subsequent screen projection operations.

[0097] like Figure 11 As shown, when the terminal device is in the unfolded state, the screen size parameters in the initial display are the width (w) and height (h) of the entire screen of the terminal device. A two-dimensional coordinate system is established with the top left vertex of the screen as the origin. The coordinates of the top left vertex can be marked as (0, 0) and the coordinates of the bottom right vertex are (w, h). At this time, the screen size parameters in the initial display are (0, 0)(w, h). After the terminal device is adjusted to the folded state, the width of the first display is w and the height is half of the original height, i.e., h / 2. At this time, the coordinates of the top left vertex are (0, 0) and the coordinates of the bottom right vertex are (w, h / 2). The screen size parameters in the initial display are adjusted, and the screen size parameters in the first display are (0, 0) and (w, h / 2).

[0098] In some other possible embodiments, creating a second display corresponding to the second display screen can be specifically implemented as follows: determining the second screen height and the second screen width of the second display screen; generating a first coordinate parameter based on the second screen height and generating a second coordinate parameter based on the second screen width; and determining the second display screen based on the first coordinate parameter and the second coordinate parameter.

[0099] In this embodiment of the application, when it is determined that the screen is in a folded state, it is necessary to generate screen size parameters according to the screen size of the second display screen, and then obtain the second display. By creating the second display screen, each display screen corresponds to one display screen, thereby enabling subsequent screen projection operations to be performed only on the first display screen or the second display screen according to the user's selection.

[0100] like Figure 11 As shown, when the terminal device is in the unfolded state, the screen size parameters in the initial display are the width (w) and height (h) of the entire screen. A two-dimensional coordinate system is established with the top left vertex of the screen as the origin. The coordinates of the top left vertex can be marked as (0, 0), and the coordinates of the bottom right vertex are (w, h). At this time, the screen size parameters in the initial display are (0, 0)(w, h). After the terminal device is adjusted to the folded state, the width of the second display is w, and the height is half of the original height, i.e., h / 2. At this time, the coordinates of the top left vertex, i.e. the first coordinate parameter, are (0, h / 2), and the coordinates of the bottom right vertex, i.e. the second coordinate parameter, are (w, h). After obtaining the first coordinate parameter and the second coordinate parameter, the second display can be generated. The screen size parameters of the generated second display are (0, h / 2) and (w, h).

[0101] In some possible implementations, a partial refresh method is used to ensure the frame rate of the projection area, that is, after the implementation is complete... Figure 7 After the steps shown, the following can be performed: determine the presence of a refresh area in the target display screen based on the first frame data, and send the display data corresponding to the refresh area to the target projection device; the first frame data is the next frame data of the current frame data being displayed in the target display screen.

[0102] In this embodiment of the application, when it is determined that there is a refresh area in the target display screen, only the refresh area is sent to the target projection device, without sending the entire content of the target display screen to the target projection device. By using the partial refresh method, the load is reduced and the load of the projection area is guaranteed.

[0103] For example: Figure 12A As shown, the target display screen is the second display screen. The content of the second display screen is displayed on the target projection device. Based on the current frame data and the first frame data, the refresh area is determined to be area A, and then area A is sent to the target projection device; or, as... Figure 12B As shown, the target display screen is the first display screen. The content of the first display screen is displayed on the target projection device. The refresh area is determined to be area B based on the current frame data and the first frame data. Then, area B is sent to the target projection device.

[0104] In some possible embodiments, when the operating system graphics resources of the terminal device are limited, it is necessary to prioritize the frame rate of the target display screen to ensure clear image output. Therefore, the frame rate of non-target display screens can be appropriately reduced, i.e., Figure 13 As shown, the frame rate decision module first determines whether the load on the non-target display needs adjustment. If adjustment is needed, the frame rate adjustment module reduces the frame rate of the non-target display to ensure the frame rate of the target display. Specifically, this can be implemented as follows: Figure 14 The process shown is as follows:

[0105] In step 1401: Obtain the overall load of the terminal device.

[0106] In this embodiment of the application, when the overall load computing power of the terminal device is sufficient, there is no need to reduce the frame rate of the non-target display screen. The frame rate of the non-target display screen is only reduced when the computing power is insufficient. Therefore, it is necessary to first determine the current overall load of the terminal device.

[0107] In some possible embodiments, after obtaining the overall load, it can be determined whether the frame rate of the non-target display needs to be adjusted, and by what level, based on the overall load and a preset adjustment table.

[0108] For example: Preset gear adjustment table, such as Figure 15 As shown, if the current overall load is determined to be 80, then it can be determined that the computing power is insufficient at this time, and the load needs to be increased by 1 level.

[0109] In step 1402: Determine the frame rate adjustment required for non-target displays based on the overall load, and determine the frame rate percentage for non-target displays.

[0110] Wherein, if the target display screen is the first display screen, then the non-target display screen is the second display screen, and if the target display screen is the second display screen, then the non-target display screen is the first display screen.

[0111] In this application embodiment, considering that the distance between the human eye and the screen of the terminal device is relatively fixed (or fluctuates within a small range) during the screen projection operation of the user using the terminal device, the distance between the non-target display screen (focus window) and the target display screen (non-focus window) can be used to identify the human eye's viewing angle. Therefore, this application sets a frame rate percentage based on this, and determines the target frame rate of the non-target display screen according to the calculated frame rate percentage.

[0112] In some possible embodiments, the frame rate percentage of the non-target display screen is determined, specifically as follows: Figure 16 The steps shown are as follows:

[0113] In step 1601: Determine the focal distance between the first focal point of the target display screen and the second focal point of the non-target display screen.

[0114] In this embodiment of the application, the first focal point is the center point of the target display screen, and the second focal point is the center point of a non-target display screen. The focal distance between the first focal point and the second focal point can be obtained based on the coordinates of the first focal point and the second focal point, and the focal distance is denoted as x.

[0115] In step 1602: the physical distance between the first focal point and the second focal point is obtained based on the focal distance and pixel density.

[0116] In this embodiment of the application, the pixel density is the pixel density of the terminal device, and the ratio of the focal distance to the pixel density is used as the physical distance between the first focal point and the second focal point.

[0117] For example, if the focal distance is denoted as x and the pixel density is denoted as ppi, then the physical distance is x1 = x / ppi.

[0118] In step 1603: Determine the frame rate calculation function based on the physical distance and the preset distance threshold.

[0119] In this application, based on the relationship between non-target display screens, target display screens, and human eye attention, a system is set up as follows: Figure 17 The frame rate calculation function shown uses 3.44 as the dividing point between the two frame rate calculation functions. When x1 is less than 3.44, the frame rate calculation function is the first frame rate calculation function, and when x1 is greater than or equal to 3.44, the frame rate calculation function is the second frame rate calculation function.

[0120] For example, the first frame rate calculation function can be y = -0.11*x1 + 1.0038, and the second frame rate calculation function can be y = -0.0315*x1 + 0.7429.

[0121] It is understood that the specific data in the dividing point and frame rate function given above are only illustrative examples and are not intended to limit the dividing point and frame rate function.

[0122] In step 1604: the frame rate percentage is obtained based on the frame rate calculation function and the physical distance.

[0123] In this embodiment of the application, after obtaining the physical distance and the frame rate calculation function, the frame rate percentage can be obtained by substituting the physical distance into the corresponding frame rate calculation function.

[0124] In step 1403: Determine the target frame rate based on the frame rate percentage.

[0125] In this embodiment of the application, after obtaining the frame rate percentage, it can be based on Figure 18The table showing the relationship between the frame rate percentage and the target frame rate is used to determine the target frame rate.

[0126] For example: First, according to Figure 16 The preset load adjustment table shows that the current overall load is 80, indicating insufficient computing power. The load needs to be increased by 1 level. The current load is set to load 1, and the adjusted load is set to load 2. The frame rate percentage is set to 82%, indicating a target frame rate of 30 frames per second (FPS).

[0127] In step 1404: Adjust the frame rate of the non-target display to the target frame rate.

[0128] In this embodiment of the application, after obtaining the target frame rate, the frame rate of the non-target display screen is adjusted to the target frame rate, thereby reducing the frame rate of the non-target display screen and reducing the load on the operating system of the terminal device.

[0129] It is understood that the above description, which uses a terminal device including a first display screen and a second display screen as an example, does not limit the number of display screens on the terminal device. The screen projection method provided in the embodiments of this application is also applicable to terminal devices with multiple display screens. For example, in response to a screen projection command, a display corresponding to each display screen is created, and a screen projection prompt is displayed on the terminal device; the target projection device and the target display screen are determined based on the screen projection prompt; and the interface of the target display screen is projected onto the target projection device based on the display corresponding to the target display screen.

[0130] To facilitate further understanding, the overall process of a screen projection method provided in the embodiments of this application will be described below, such as... Figure 19 As shown, where:

[0131] In step 1901: In response to the screen mirroring command, a screen mirroring prompt is displayed on the terminal device.

[0132] The specific implementation method of this step is the same as that of step 702, and will not be described again here.

[0133] In step 1902: the target projection device and the target display screen are determined based on the projection prompts.

[0134] The specific implementation method of this step is the same as that of step 703, and will not be described again here.

[0135] In step 1903: Determine whether the target display screen is the first display screen or the second display screen. If the target display screen is determined to be the first display screen or the second display screen, proceed to step 1904; otherwise, proceed to step 1905.

[0136] In step 1904: Create the first display corresponding to the first display screen, and create the second display corresponding to the second display screen.

[0137] The specific implementation method of this step is the same as that of step 704, and will not be described again here.

[0138] In step 1905: The initial display based on the terminal device projects the display interface onto the target projection device.

[0139] In step 1906: Determine whether there is a refresh area in the terminal device based on the first frame data. If it is determined to exist, proceed to step 1907; otherwise, proceed to step 1905.

[0140] In step 1907: the display data corresponding to the refresh area is sent to the target projection device.

[0141] In step 1908: The interface of the target display screen is projected onto the target projection device based on the display corresponding to the target display screen.

[0142] The specific implementation method of this step is the same as that of step 701, and will not be described again here.

[0143] In step 1909: Determine whether there is a refresh area in the target display screen based on the first frame data. If there is, proceed to step 1910; otherwise, proceed to step 1908.

[0144] In step 1910: the display data corresponding to the refresh area is sent to the target projection device.

[0145] In step 1911: Obtain the overall load of the terminal device.

[0146] The specific implementation method of this step is the same as that of step 1401, and will not be described again here.

[0147] In step 1912: Determine whether the frame rate of non-target displays needs to be adjusted based on the overall load. If so, proceed to step 1913; otherwise, proceed to step 1908.

[0148] The specific implementation method of this step is the same as that of step 1401, and will not be described again here.

[0149] In step 1913: Determine the frame rate percentage of the non-target display.

[0150] The specific implementation method of this step is the same as that of step 1402, and will not be described again here.

[0151] In step 1914: Determine the target frame rate based on the frame rate percentage.

[0152] The specific implementation method of this step is the same as that of step 1403, and will not be described again here.

[0153] In step 1915: Adjust the frame rate of the non-target display to the target frame rate.

[0154] The specific implementation method of this step is the same as that of step 1404, and will not be described again here.

[0155] Based on the same inventive concept, this application also provides a screen projection device, such as... Figure 20 As shown, it is applied to a terminal device, which includes: a first display screen and a second display screen. The device includes:

[0156] The system comprises a response module 20001, a prompt module 20002, a confirmation module 20003, and a projection module 20004. The response module 20001 is used to execute the above-mentioned functions. Figure 7 Step 701 in Figure 19 Steps 1901 and 1904 in the above steps are prompted by the prompting module. Figure 7 Step 702 in Figure 19 Step 1902 in Figure 9 In step 1903, the module is determined to perform the above. Figure 7 In step 703, the projection module 20004 is used to perform the above... Figure 7 Step 704 in Figure 14 The steps shown and Figure 16 The steps shown Figure 19 Steps 1905, 1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, and 1915.

[0157] Corresponding to the above embodiments, this application also provides an electronic device. Figure 21 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 2100 may include a processor 2101, a memory 2102, and a communication unit 2103. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0158] The communication unit 2103 is used to establish a communication channel, enabling the electronic device to communicate with other devices. It receives user data from other devices or sends user data to other devices.

[0159] The processor 2101 serves as the control center of the electronic device, connecting various parts of the device via interfaces and lines. It executes software programs and / or modules stored in the memory 2102, and calls data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 2101 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0160] The memory 2102 is used to store the execution instructions of the processor 2101. The memory 2102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0161] When the execution instructions in memory 2102 are executed by processor 2101, the electronic device 2100 is able to perform operations. Figure 7 Some or all of the steps in the illustrated embodiments.

[0162] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the various embodiments of the screen projection method provided by the present invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0163] In some possible implementations, various aspects of the terminal device control method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in a screen projection method according to various exemplary embodiments of this application described above.

[0164] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0165] The program product for controlling a terminal device according to embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0166] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0167] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A screen projection method, characterized in that, Applied to a terminal device, the terminal device including: a first display screen and a second display screen, the method including: In response to the screen mirroring command, a first display function corresponding to the first display screen is created, and a second display function corresponding to the second display screen is created; A screen mirroring prompt is displayed on the terminal device; the screen mirroring prompt includes: a list of target devices and a list of screen mirroring areas; Based on the screen projection prompts, the target projection device and target display screen are determined; The interface of the target display screen is projected onto the target projection device based on the display corresponding to the target display screen.

2. The method according to claim 1, characterized in that, Creating the first display corresponding to the first display screen includes: Determine the first screen width of the first display screen; The height coordinate parameters in the initial display of the terminal device are adjusted according to the first screen height to obtain the first display.

3. The method according to claim 1, characterized in that, Creating the second display corresponding to the second display screen includes: Determine the second screen height and the second screen width of the second display screen; A first coordinate parameter is generated based on the height of the second screen, and a second coordinate parameter is generated based on the width of the second screen; The second display is determined based on the first coordinate parameter and the second coordinate parameter.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the overall load of the terminal device; Based on the overall load, it is determined that the frame rate of the non-target display needs to be adjusted, and the frame rate percentage of the non-target display is determined; wherein, if the target display is the first display, then the non-target display is the second display, and if the target display is the second display, then the non-target display is the first display. The target frame rate is determined based on the frame rate percentage. Adjust the frame rate of the non-target display to the target frame rate.

5. The method according to claim 4, characterized in that, Determining the frame rate percentage of the non-target display screen includes: Determine the focal distance between the first focal point of the target display screen and the second focal point of the non-target display screen; The physical distance between the first focal point and the second focal point is obtained based on the focal distance and pixel density; The frame rate calculation function is determined based on the physical distance and the preset distance threshold. The frame rate percentage is obtained based on the frame rate calculation function and the physical distance.

6. The method according to claim 1, characterized in that, After the display corresponding to the target display screen is projected onto the target projection device, the method further includes: Based on the first frame data, it is determined that there is a refresh area in the target display screen, and the display data corresponding to the refresh area is sent to the target projection device; the first frame data is the next frame data of the current frame data being displayed in the target display screen.

7. The method according to claim 1, characterized in that, Before creating the first display corresponding to the first display screen and creating the second display corresponding to the second display screen, the method further includes: Determine the folding angle between the first display screen and the second display screen; The terminal device is determined to be in a folded state based on the folding angle.

8. A screen projection device, characterized in that, Applied to a terminal device, the terminal device including: a first display screen and a second display screen, the device including: The response module is used to respond to the screen projection command, create a first display corresponding to the first display screen, and create a second display corresponding to the second display screen; The prompt module is used to display a screen projection prompt on the terminal device; The determination module is used to determine the target projection device and the target display screen based on the projection prompt; The screen projection module is used to project the interface of the target display screen to the target projection device based on the display value corresponding to the target display screen.

9. An electronic device, characterized in that, It includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of claims 1-7.

Images