Screen projection method, device and system

Abstract

The present disclosure provides a screen projection method, device and system. The screen projection method is applied to a screen projection device, including: obtaining screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device; obtaining matching parameters through rule matching based on the obtained screen projection parameters; encoding source data based on the matching parameters; and sending the projection data including the encoded source data to the screen projection receiving device. According to the method of the present disclosure, the purpose of reducing latency, reducing power consumption, improving heat generation and enhancing display effects can be achieved through the joint optimization and collaborative work of the screen projection device and the screen projection receiving device.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a screen projection method, device and system. Background Art

[0002] With the promotion and popularization of screen projection technology, more and more users hope to enjoy the big screen experience through screen projection technology while watching videos and playing games on terminals (for example, mobile phones). The current screen projection technology uses fixed parameters to transmit the projection resources to the projection receiving device. The projection device and the projection receiving device encode and decode the resources without making additional modifications. There are also differences between the hardware of the projection device and the projection receiving device. For some applications with high refresh rate requirements, such as games, the current screen projection technology has problems such as high latency, high power consumption, easy heat generation, and the resource display effect is not suitable for large screens. Summary of the invention

[0003] The present disclosure provides a screen projection method, device and system to at least solve the problems of high latency, high power consumption, easy heat generation and resource display effect not being suitable for large screens in the screen projection technology in the related art.

[0004] According to a first aspect of the present disclosure, a screen projection method is provided, which is applied to a screen projection device, including: obtaining screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device; obtaining matching parameters through rule matching based on the obtained screen projection parameters; encoding source data based on the matching parameters; and sending projection data including the encoded source data to the screen projection receiving device.

[0005] Preferably, obtaining the screen projection parameters includes: obtaining source data parameters and screen projection device parameters; establishing a handshake connection with the screen projection receiving device; and based on the establishment of the handshake connection, receiving the screen projection receiving device parameters sent by the screen projection receiving device.

[0006] Preferably, the source data parameters include source resolution, and the screen projection receiving device parameters include resolution gear information; the matching parameters are obtained based on the acquired screen projection parameters through rule matching, including: calculating the matching degree between each resolution gear in the resolution gear information and the source resolution; based on the calculated matching degree, selecting a resolution matching gear in the resolution gear information.

[0007] Preferably, encoding the source data based on the matching parameters includes: encoding the source data based on the resolution matching level; wherein the projection data includes the resolution matching level.

[0008] Preferably, the source data parameters include a target frame rate, and the projection device parameters include a supported refresh rate; the matching parameters are obtained based on the acquired projection parameters through rule matching, including: sorting the supported refresh rates; determining a matching refresh rate that matches the target frame rate based on the sorted supported refresh rates; and adjusting the screen refresh rate according to the matching refresh rate.

[0009] Preferably, the source data parameters include a target frame rate; the matching parameters are obtained based on the acquired screen projection parameters through rule matching, and also includes: setting the target frame rate as a matching frame rate; encoding the source data based on the matching parameters includes: encoding the source data based on the matching frame rate; wherein the screen projection data includes the matching frame rate.

[0010] Preferably, the screen projection device parameters include a first screen mode, and the screen projection receiving device parameters include a second screen mode; the matching parameters are obtained based on the acquired screen projection parameters through rule matching, including: determining the consistency between the first screen mode and the second screen mode, wherein the first screen mode and the second screen mode include a horizontal screen mode and a vertical screen mode; when the first screen mode is inconsistent with the second screen mode, determining the second screen mode as the matching screen mode.

[0011] Preferably, encoding the source data based on the matching parameters comprises: encoding the source data based on the matching screen mode.

[0012] According to a second aspect of the present disclosure, a screen projection method is provided, which is applied to a screen projection receiving device, including: sending screen projection receiving device parameters to the screen projection device; receiving screen projection data including encoded source data sent by the screen projection device; decoding the received encoded source data to obtain decoded data; and displaying on a screen based on the decoded data.

[0013] Preferably, sending the screen projection receiving device parameters to the screen projection device includes: establishing a handshake connection with the screen projection device; and sending the screen projection receiving device parameters to the screen projection device based on the establishment of the handshake connection.

[0014] Preferably, the screen projection data also includes matching parameters; the method also includes: adjusting the parameters of the screen according to the matching parameters.

[0015] Preferably, the matching parameters include a resolution matching gear; and adjusting the parameters of the screen according to the matching parameters includes: adjusting the resolution of the screen according to the resolution matching gear.

[0016] Preferably, the matching parameters include matching frame rate; adjusting the parameters of the screen according to the matching parameters includes: sorting supported refresh rates; determining a matching refresh rate that matches the matching frame rate based on the sorted supported refresh rates; and adjusting the refresh rate of the screen according to the matching refresh rate.

[0017] Preferably, the displaying on the screen based on the decoded data further comprises: performing display enhancement processing on the decoded data to obtain display-enhanced data; and displaying on the screen based on the display-enhanced data.

[0018] Preferably, the display enhancement processing includes at least one of frame interpolation processing, super-resolution processing and color processing.

[0019] Preferably, the interpolation processing includes: interpolating frames based on a motion compensation algorithm to obtain data for supplementary interpolation frames; the super-resolution processing includes: improving the resolution using a super-resolution model to obtain data with improved resolution; and the color processing includes: performing color processing through a high dynamic range HDR algorithm to obtain color-processed data.

[0020] According to a third aspect of the present disclosure, a screen projection device is provided, comprising: a parameter acquisition module, configured to acquire screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device; a matching parameter acquisition module, configured to acquire matching parameters through rule matching based on the acquired screen projection parameters; an encoding module, configured to encode source data based on the matching parameters; and a screen projection data sending module, configured to send projection data including the encoded source data to the screen projection receiving device.

[0021] Preferably, the parameter acquisition module is configured to: acquire source data parameters and screen projection device parameters; establish a handshake connection with the screen projection receiving device; and based on the establishment of the handshake connection, receive the screen projection receiving device parameters sent by the screen projection receiving device.

[0022] Preferably, the source data parameters include source resolution, and the projection receiving device parameters include resolution gear information; the matching parameter acquisition module is configured to: calculate the matching degree between each resolution gear in the resolution gear information and the source resolution; based on the calculated matching degree, select a resolution matching gear in the resolution gear information.

[0023] Preferably, the encoding module is configured to: encode the source data based on the resolution matching level; wherein the projection data includes the resolution matching level.

[0024] Preferably, the source data parameters include a target frame rate, and the projection device parameters include a supported refresh rate; the matching parameter obtaining module is configured to: sort the supported refresh rates; determine a matching refresh rate that matches the target frame rate based on the sorted supported refresh rates; and adjust the screen refresh rate according to the matching refresh rate.

[0025] Preferably, the source data parameters include a target frame rate; the matching parameter obtaining module is configured to: set the target frame rate as a matching frame rate; the encoding module is configured to: encode the source data based on the matching frame rate; wherein the projection data includes the matching frame rate.

[0026] Preferably, the screen projection device parameters include a first screen mode, and the screen projection receiving device parameters include a second screen mode; the matching parameter obtaining module is configured to: determine the consistency between the first screen mode and the second screen mode, wherein the first screen mode and the second screen mode include a horizontal screen mode and a vertical screen mode; when the first screen mode is inconsistent with the second screen mode, determine that the second screen mode is a matching screen mode.

[0027] Preferably, the encoding module is configured to: encode the source data based on the matching screen mode.

[0028] According to the fourth aspect of the present disclosure, a screen projection receiving device is provided, comprising: a parameter sending module, configured to send screen projection receiving device parameters to the screen projection device; a screen projection data receiving module, configured to receive screen projection data including encoded source data sent by the screen projection device; a decoding module, configured to decode the received encoded source data to obtain decoded data; and a display module, configured to display on a screen based on the decoded data.

[0029] Preferably, the parameter sending module is configured to: establish a handshake connection with the screen projection device; and based on the establishment of the handshake connection, send the screen projection receiving device parameters to the screen projection device.

[0030] Preferably, the screen projection data also includes matching parameters; the screen projection receiving device also includes: a parameter adjustment module configured to adjust the parameters of the screen according to the matching parameters.

[0031] Preferably, the matching parameters include a resolution matching level; and the parameter adjustment module is configured to: adjust the resolution of the screen according to the resolution matching level.

[0032] Preferably, the matching parameters include a matching frame rate; the parameter adjustment module is configured to: sort the supported refresh rates; determine a matching refresh rate that matches the matching frame rate based on the sorted supported refresh rates; and adjust the refresh rate of the screen according to the matching refresh rate.

[0033] Preferably, the display module is further configured to: perform display enhancement processing on the decoded data to obtain display-enhanced data; and display on a screen based on the display-enhanced data.

[0034] Preferably, the display enhancement processing includes at least one of frame interpolation processing, super-resolution processing and color processing.

[0035] Preferably, the interpolation processing includes: interpolating frames based on a motion compensation algorithm to obtain data for supplementary interpolation frames; the super-resolution processing includes: improving the resolution using a super-resolution model to obtain data with improved resolution; and the color processing includes: performing color processing through a high dynamic range HDR algorithm to obtain color-processed data.

[0036] According to a fifth aspect of the present disclosure, a screen projection method is provided, comprising: obtaining, by a screen projection device, screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by a screen projection receiving device; obtaining matching parameters through rule matching based on the acquired screen projection parameters; encoding source data based on the matching parameters; and sending the projection data including the encoded source data to the screen projection receiving device; the screen projection receiving device sending the projection receiving device parameters to the screen projection device; receiving the projection data including the encoded source data sent by the screen projection device; decoding the received encoded source data to obtain decoded data; and displaying on a screen based on the decoded data.

[0037] According to a sixth aspect of the present disclosure, a screen projection system is provided, comprising a screen projection device and a screen projection receiving device, wherein the screen projection device is configured to obtain screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device; based on the obtained screen projection parameters, matching parameters are obtained through rule matching; based on the matching parameters, source data is encoded; and the projection data including the encoded source data is sent to the screen projection receiving device; the screen projection receiving device is configured to send the projection receiving device parameters to the screen projection device; receive the projection data including the encoded source data sent by the screen projection device; decode the received encoded source data to obtain decoded data; and display on the screen based on the decoded data.

[0038] According to the seventh aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory storing a computer program, wherein when the computer program is executed by the processor, the screen projection method applied to the screen projection device and the screen projection method applied to the screen projection receiving device as described above are implemented.

[0039] According to an eighth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed, the screen projection method applied to the screen projection device and the screen projection method applied to the screen projection receiving device as described above are implemented.

[0040] The technical solution provided according to the embodiments of the present disclosure brings at least the following beneficial effects: Generally speaking, through the joint optimization and collaborative work of the screen projection device and the screen projection receiving device, the purpose of reducing latency, reducing power consumption, improving heat generation and enhancing display effects is achieved. Specifically, parameter matching is performed between the screen projection resource (source data) parameters, the screen projection device parameters and the screen projection receiving device parameters, and the screen projection resources are encoded by the matched parameters, which reduces latency and power consumption and improves the heat generation problem. The display parameters of the screen projection device and the screen projection receiving device are adjusted by matching parameters to further reduce power consumption and improve heat generation problems, thereby improving screen projection performance. Display enhancement processing is performed on the screen projection receiving device side, which alleviates the processing pressure on the screen projection device side, makes full use of the hardware resources of the screen projection receiving device (especially the large-screen display device), improves the display effect, and makes the display picture clearer and smoother, thereby improving the user experience.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0043] Figure 1 An example architecture of a screen projection system according to an exemplary embodiment is shown.

[0044] Figure 2 A flow chart of a screen projection method applied to a screen projection device according to an exemplary embodiment is shown.

[0045] Figure 3 A schematic diagram showing target frame rate acquisition of source data according to an exemplary embodiment is shown.

[0046] Figure 4 A schematic diagram of resolution matching according to an exemplary embodiment is shown.

[0047] Figure 5A schematic diagram of horizontal and vertical screen matching according to an exemplary embodiment is shown.

[0048] Figure 6 A flow chart of a screen projection method applied to a screen projection receiving device according to an exemplary embodiment is shown.

[0049] Figure 7 A schematic diagram of a display enhancement process according to an exemplary embodiment is shown.

[0050] Figure 8 A schematic flowchart of a screen projection method according to an exemplary embodiment is shown.

[0051] Fig. 9 A screen projection device adaptation module according to an exemplary embodiment is shown.

[0052] Fig.10 An intelligent display enhancement module of a screen projection receiving device according to an exemplary embodiment is shown.

[0053] Fig.11 A schematic diagram of screen projection from a terminal to a television according to an exemplary embodiment is shown.

[0054] Fig.12 A schematic diagram of screen projection for a landscape and portrait screen adaptation scenario according to an exemplary embodiment is shown.

[0055] Fig.13 A schematic diagram of screen projection of a multi-person conference scenario according to an exemplary embodiment is shown.

[0056] Fig.14 A schematic diagram showing screen projection of a multiplayer online game scene according to an exemplary embodiment.

[0057] Fig.15 A block diagram of a screen projection device according to an exemplary embodiment is shown.

[0058] Fig.16 A block diagram of a screen projection receiving device according to an exemplary embodiment is shown.

[0059] Fig.17 A screen projection system according to an exemplary embodiment is shown.

[0060] Fig.18 A block diagram of an electronic device according to an exemplary embodiment is shown. DETAILED DESCRIPTION

[0061] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0062] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0063] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0064] In the current screen projection technology, fixed parameters are used to transmit the projection resources to the projection receiving device. The projection device and the projected device encode and decode the resources without making additional modifications. However, more and more applications use high frame rates and other modes, with fast screen refreshes and high requirements for encoding, decoding and transmission, which leads to high latency, high power consumption, easy heat generation, and large transmission volume between the projection device and the projection receiving device. At the same time, the mismatch of parameters between the projection resources, the projection device, and the projection receiving device leads to low projection performance and poor projection display effects. In addition, since the processing pressure of the projection resources is on the projection device side, it further leads to high power consumption and easy heat generation problems on the projection device side. Since the hardware resources of the projection receiving device, especially the large-screen projection receiving device, have not been fully utilized, the display effect of the projection resources on the large screen is difficult to be satisfactory, and users cannot get the visual experience that the large screen should have.

[0065] In order to solve the above problems, the present disclosure provides a screen projection method, device and system. Generally speaking, the purpose of reducing latency, reducing power consumption, improving heat generation and enhancing display effects is achieved through the joint optimization and collaborative work of the screen projection device and the screen projection receiving device. Specifically, parameter matching is performed between the screen projection resource (source data) parameters, the screen projection device parameters and the screen projection receiving device parameters, and the screen projection resources are encoded by the matched parameters, which reduces latency and power consumption and improves the heat generation problem. The display parameters of the screen projection device and the screen projection receiving device are adjusted by matching parameters to further reduce power consumption and improve heat generation problems, thereby improving screen projection performance. Display enhancement processing is performed on the screen projection receiving device side, which alleviates the processing pressure on the screen projection device side, fully utilizes the hardware resources of the screen projection receiving device (especially large-screen display devices), improves the display effect, and makes the display picture clearer and smoother, thereby improving the user experience. Below, refer to Figures 1 to 18 The screen projection method, device and system according to the present disclosure are described in detail.

[0066] Figure 1 An example architecture of a screen projection system according to an exemplary embodiment is shown.

[0067] Reference Figure 1 The screen projection system 100 may include a screen projection device 110 and a screen projection receiving device 120, and the screen projection device 110 may be connected to the screen projection receiving device 120 in a wired or wireless (such as a WIFI wireless local area network) manner.

[0068] In the system 100, the screen projection device 110 can be a portable device on which the user can directly perform input operations (such as, but not limited to, a mobile phone, a tablet, a smart watch, a laptop computer, etc.), and the screen projection receiving device 120 can be a display device (such as a large-screen display device such as a television) on which the user usually does not directly perform input operations but instead performs input operations with the aid of, for example, but not limited to, a remote control device.

[0069] The screen projection device 110 can load and run various client applications or APPs, and can be configured to respond to a screen projection instruction (for example, a user clicks a screen projection button) and use a screen projection protocol (such as, but not limited to, DLNA protocol, airplay protocol, lelink protocol, etc.) to project the screen to the screen projection receiving device 120, so as to send the picture displayed on the screen of the screen projection device 110 (such as page resources such as web pages, videos, games, etc.) to the screen of the screen projection receiving device 120 for display.

[0070] Figure 2 A flow chart of a screen projection method applied to a screen projection device according to an exemplary embodiment is shown.

[0071] Reference Figure 2In step S210, screen projection parameters are obtained, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device. According to an exemplary embodiment of the present disclosure, the screen projection device can obtain source data parameters and screen projection device parameters, establish a handshake connection with the screen projection receiving device, and receive the screen projection receiving device parameters sent by the screen projection receiving device based on the establishment of the handshake connection.

[0072] In an embodiment of the present disclosure, the screen projection device may initiate a screen projection request to the screen projection receiving device. After initiating the screen projection request, the screen projection device may first obtain source data parameters and screen projection device parameters. The screen projection device screen projection request may be initiated, for example, in response to a screen projection instruction (for example, a user clicks a screen projection button). The source data here (i.e., screen projection resources or application resources) may include but are not limited to videos, games, images, and web pages, etc., and the source data parameters may include but are not limited to resolution (source resolution), frame rate (target frame rate), etc. The screen projection device parameters may include but are not limited to resolution gear information, current resolution, default resolution, supported refresh rate, current refresh rate, and horizontal and vertical screen modes (first screen mode), etc. Figure 3 FIG. 2 is a schematic diagram showing a target frame rate acquisition diagram of source data according to an exemplary embodiment. Figure 3 , the frame rate of the source data can be divided into two types: fixed frame rate and dynamic frame rate. The source data with a dynamic frame rate can be, for example, an application resource that can dynamically modify the frame rate configuration. If it is a fixed frame rate, the projection device can obtain the current frame rate of the source data as the target frame rate. If it is a dynamic frame rate, then if there is an integrated SDK, the target frame rate can be obtained through the SDK interface. If there is no SDK, the target frame rate can be predicted. The target frame rate can be predicted, for example, through the Google scoring algorithm. Each time a touch event is received, the frame rate change information is monitored to predict the target frame rate. The method for predicting the target frame rate here is not limited to this, and the present disclosure is not limited to this.

[0073] After the screen projection receiving device receives the screen projection request, the screen projection device can establish a handshake connection with the screen projection receiving device. During the handshake connection establishment process, the screen projection device can receive the screen projection receiving device parameters sent by the screen projection receiving device. The screen projection receiving device parameters may include but are not limited to resolution gear information, default resolution, supported refresh rate, and horizontal and vertical screen modes (second screen mode).

[0074] return Figure 2 In step S220, based on the acquired projection parameters, matching parameters are obtained through rule matching. In the embodiment of the present disclosure, after the projection device acquires the projection parameters, matching parameters can be obtained through rule matching, and rule matching may include: resolution rule matching, refresh rate and frame rate rule matching, and horizontal and vertical screen mode rule matching. The projection device can perform at least one of the above rule matching.

[0075] The following describes the specific contents of resolution rule matching, refresh rate and frame rate rule matching, and horizontal and vertical screen mode rule matching in turn:

[0076] 1. Resolution rule matching

[0077] According to an exemplary embodiment of the present disclosure, the source data parameters include source resolution, and the projection receiving device parameters include resolution gear information. The projection device may calculate the degree of match between each resolution gear in the resolution gear information and the source resolution, and select a resolution matching gear in the resolution gear information based on the calculated degree of match. In an embodiment of the present disclosure, the projection parameters acquired by the projection device may include source resolution (source data parameters) and resolution gear information (projection receiving device parameters), and the resolution matching gear is obtained by resolution rule matching based on the source resolution and the resolution gear information of the projection receiving device, wherein the resolution gear information is a plurality of resolutions supported by the projection receiving device, such as 540p, 720p, and 1080p, etc. Here, when the source data is displayed on the projection device, if the projection device adopts a resolution that fully matches the source resolution, the current resolution of the projection device is the source resolution.

[0078] Figure 4 FIG. 2 shows a schematic diagram of resolution rule matching according to an exemplary embodiment. Figure 4 , the acquired source resolution is represented as (x, y), and the resolution level information of the screen projection receiving device may include, for example, (x 1 ,y 1 ), (x 2 ,y 2 )…,(x n ,y n ), select the appropriate resolution level from the resolution level information of the screen projection receiving device through resolution rule matching.

[0079] The resolution rule matching algorithm is as follows: (1) Calculate the matching degree SR between each resolution level in the resolution level information and the source resolution.

[0080] 1) If the number of pixels of the resolution level information and the source resolution in both the x-direction and the y-direction are different, the matching degree SR is the product of the absolute value of the difference in the number of pixels in the x-direction and the absolute value of the difference in the number of pixels in the y-direction. That is, if x≠x n And y≠y n , SR=|x n -x||y n -y|.

[0081] 2) If the resolution level information and the source resolution differ in only one pixel number in the x direction and the y direction, the matching degree SR is the product of the absolute value of the difference in the number of pixels in the direction with different pixel numbers and the number of pixels in the direction with the same pixel number. That is, if x≠x n And y=y n , SR=|x n -x|*y; if x=x n And y≠y n , SR=x*|y n -y|.

[0082] 3) If the number of pixels in the x-direction and y-direction of the resolution level information and the source resolution is the same, the matching degree SR is 0, that is, if x=x n And y=y n , SR=0.

[0083] (2) The resolution level with the smallest calculated matching degree SR is selected as the resolution matching level in the resolution level information.

[0084] 2. Refresh rate and frame rate rule matching

[0085] According to an exemplary embodiment of the present disclosure, the source data parameters include a target frame rate, and the projection device parameters include a supported refresh rate. The projection device can sort the supported refresh rates, determine a matching refresh rate that matches the target frame rate based on the sorted supported refresh rates, and adjust the refresh rate of the screen according to the matching refresh rate.

[0086] In the embodiment of the present disclosure, the projection parameters acquired by the projection device may include a target frame rate (source data parameter) and a supported refresh rate (projection device parameter). Figure 3 After the method described above obtains the target frame rate of the source data, the matching refresh rate can be obtained through refresh rate rule matching based on the target frame rate and the supported refresh rate. The supported refresh rate here is the refresh rate supported by the hardware of the projection device.

[0087] The refresh rate rule matching is as follows: sort and save the supported refresh rates, and search for the matching refresh rate RR that matches the target frame rate in the sorted supported refresh rates. For example, there are s supported refresh rates in total, and they are sorted from small to large to RR n1 ,RR n2 …RR ns Here, the sorting method can also be from large to small.

[0088] 1) If the target frame rate is between the two adjacent supported refresh rates after sorting, the larger supported refresh rate of the two adjacent supported refresh rates is determined as the matching refresh rate. np < Target Frame Rate <RRnq , determine RR nq is RR, where RR np and RR nq Sort by the supported refresh rates of the two adjacent ones.

[0089] 2) If all supported refresh rates are greater than the target frame rate, the minimum supported refresh rate is determined to match the refresh rate, that is, if RR n1 ,RR n2 …RR ns > target frame rate, then determine RR n1 is RR.

[0090] 3) If all supported refresh rates are less than the target frame rate, the maximum supported refresh rate is determined to be the matching refresh rate, that is, if the target frame rate > RR n1 ,RR n2 …RR ns , then determine RR ns is RR.

[0091] 3) If the target frame rate is equal to one of the sorted supported refresh rates, then the supported refresh rate is determined to be the matching refresh rate, that is, if RR np = target frame rate, then determine RR np is RR, where RR np One of the sorted supported refresh rates.

[0092] After the matching refresh rate of the projection device is determined, the refresh rate of the projection device screen can be adjusted according to the matching refresh rate. By adjusting the refresh rate of the projection device, the target frame rate of the source data can be better matched, avoiding high power consumption and heat generation on the projection device side caused by too high refresh rate.

[0093] According to an exemplary embodiment of the present disclosure, the source data parameters include a target frame rate, and the screen projection device can set the target frame rate to match the frame rate.

[0094] In an embodiment of the present disclosure, the projection parameters acquired by the projection device may include a target frame rate (source data parameter), and a matching frame rate may be obtained based on the acquired target frame rate through frame rate rule matching. The frame rate rule matching here may be to match the frame rate during encoding so that the frame rate during encoding is equal to the target frame rate. Then, obtaining the matching frame rate through frame rate rule matching may be to set the target frame rate as the matching frame rate, or to use the target frame rate as the matching frame rate.

[0095] 3. Rules matching for horizontal and vertical screen modes

[0096] According to an exemplary embodiment of the present disclosure, the screen projection device parameters include a first screen mode, and the screen projection receiving device parameters include a second screen mode; the screen projection device can determine the consistency of the first screen mode and the second screen mode, wherein the first screen mode and the second screen mode include a horizontal screen mode and a vertical screen mode; when the first screen mode is inconsistent with the second screen mode, the second screen mode is determined to be a matching screen mode.

[0097] In an embodiment of the present disclosure, the projection parameters acquired by the projection device may include the horizontal and vertical screen modes of the projection device (first screen mode) and the horizontal and vertical screen modes of the projection receiving device (second screen mode). Based on the acquired horizontal and vertical screen modes of the projection device and the horizontal and vertical screen modes of the projection receiving device, the matching screen mode can be obtained by matching the horizontal and vertical screen mode rules. Specifically, the projection device can determine whether the horizontal and vertical screen modes of the projection device and the projection receiving device are consistent. In the case of consistency, the current screen mode is maintained. In the case of inconsistency, the projection device can, for example, enter the presentation mode (presentation mode) and determine that the horizontal and vertical screen modes of the projection receiving device are the matching screen modes. Through the presentation mode, different user interfaces (UIs) can be displayed on the projection device and the projection receiving device. After confirming the matching screen mode, the corresponding mode UI can be adapted, that is, the UI in the matching screen mode can be displayed on the projection receiving device. In addition, a projection UI module can also be set in the projection device. When the horizontal and vertical screen modes of the projection device and the projection receiving device are inconsistent, the projection UI module can provide a horizontal and vertical screen menu switch to better adapt to the display and touch of the projection device and the projection device in the projection mode. Figure 5 FIG. 2 shows a schematic diagram of horizontal and vertical screen matching according to an exemplary embodiment. Figure 5 After the screen projection request is initiated, the screen projection receiving device only displays the corresponding resources on the screen projection receiving device according to the horizontal and vertical screen modes of the screen projection device. If the screen projection receiving device is in horizontal mode and the projection device is in vertical mode, the screen projection receiving device will continue to display the vertical screen effect (current) after the screen projection. After determining the matching screen mode (horizontal mode) according to the horizontal and vertical screen mode rules (after updating), the screen projection device side encodes the source data according to the matching screen mode, and sends the encoded source data to the screen projection receiving device, so that it can be subsequently projected to the screen projection receiving device in horizontal mode.

[0098] return Figure 2 , in step S230, the source data is encoded based on the matching parameters.

[0099] According to an exemplary embodiment of the present disclosure, the screen projection device can encode the source data based on the resolution matching gear. In an embodiment of the present disclosure, after the resolution matching gear is obtained by resolution rule matching, the screen projection device uses the resolution matching gear (resolution) to encode the source data. The screen projection device uses the resolution matching gear to encode the source data, which can better match the source resolution of the source data, avoid the additional power consumption and heat caused by the use of a higher-end resolution for encoding, and also reduce the increase in delay caused by the increase in the amount of transmitted data due to the use of a higher-end resolution.

[0100] According to an exemplary embodiment of the present disclosure, the screen projection device may encode the source data based on a matching frame rate. In an embodiment of the present disclosure, after obtaining a matching frame rate through frame rate rule matching, the screen projection device encodes the source data using the matching frame rate. The screen projection device encodes the source data using a matching frame rate, which can better match the target frame rate (frame rate) of the source data, avoid the additional power consumption and heat caused by using a higher frame rate for encoding, and also reduce the increase in delay caused by the increased amount of transmitted data due to the use of a higher frame rate.

[0101] According to an exemplary embodiment of the present disclosure, the screen projection device can encode the source data based on the matching screen mode. In the embodiment of the present disclosure, after obtaining the matching screen mode through the horizontal and vertical screen mode rule matching, the screen projection device encodes the source data using the matching screen mode and sends the encoded source data to the screen projection receiving device, so that the screen projection receiving device can display the screen adapted to the horizontal and vertical screen modes of the screen receiving device, thereby improving the user's visual experience.

[0102] In step S240, the projection data including the encoded source data is sent to the projection receiving device. In an embodiment of the present disclosure, after encoding the source data, the projection device also sends the projection data including the encoded source data to the projection receiving device. Furthermore, the projection data may also include matching parameters.

[0103] According to an exemplary embodiment of the present disclosure, the screen projection data may include a resolution matching gear. In an embodiment of the present disclosure, after obtaining the resolution matching gear through resolution rule matching, the screen projection device may also send the resolution matching gear (or the encoded resolution matching gear) as screen projection data to the screen projection receiving device, so that the screen projection receiving device adjusts the resolution of its own screen according to the resolution matching gear.

[0104] According to an exemplary embodiment of the present disclosure, the screen projection data may include a matching frame rate. In an embodiment of the present disclosure, after obtaining a matching frame rate through frame rate rule matching, the screen projection device may also send the matching frame rate (or the encoded matching frame rate) as screen projection data to the screen projection receiving device, so that the screen projection receiving device determines the matching refresh rate on the screen projection receiving device side according to the matching frame rate, thereby adjusting the refresh rate of the screen projection receiving device.

[0105] As described above, according to the screen projection method applied to the screen projection device of the exemplary embodiment, parameter matching is performed between the screen projection resource (source data) parameters, the screen projection device parameters, and the screen projection receiving device parameters, and the screen projection resources are encoded by the matched parameters, which reduces the delay and power consumption and improves the heating problem. By adjusting the display parameters of the screen projection device through matching parameters, power consumption is further reduced and heating problems are improved, thereby improving the screen projection performance.

[0106] Figure 6 A flow chart of a screen projection method applied to a screen projection receiving device according to an exemplary embodiment is shown.

[0107] Reference Figure 6 In step S610, the screen projection receiving device parameters are sent to the screen projection device. According to an exemplary embodiment of the present disclosure, the screen projection receiving device can establish a handshake connection with the screen projection device, and based on the establishment of the handshake connection, the screen projection receiving device parameters are sent to the screen projection device.

[0108] In an embodiment of the present disclosure, the screen projection device sends a screen projection request to the screen projection receiving device. After the screen projection receiving device receives the screen projection request, the screen projection device can establish a handshake connection with the screen projection receiving device. During the handshake connection establishment process, the screen projection receiving device can send the screen projection receiving device parameters to the screen projection device. The screen projection receiving device parameters may include but are not limited to resolution gear information, default resolution, supported refresh rate, and horizontal and vertical screen modes (second screen mode).

[0109] In step S620, screen projection data including encoded source data sent by the screen projection device is received. In an embodiment of the present disclosure, after the screen projection receiving device sends the screen projection receiving device parameters to the screen projection device, the screen projection device can obtain matching parameters through rule matching based on the acquired screen projection parameters including the screen projection receiving device parameters, and then encode the source data based on the matching parameters, and then send the screen projection data including the encoded source data to the screen projection receiving device. Accordingly, the screen projection receiving device can receive the screen projection data including the encoded source data sent by the screen projection device.

[0110] According to an exemplary embodiment of the present disclosure, the screen projection data also includes matching parameters, and the screen projection method further includes step S650 (not shown), in which the parameters of the screen are adjusted according to the matching parameters. In an embodiment of the present disclosure, the screen projection device may send the screen projection data including the matching parameters to the screen projection receiving device, and accordingly, the screen projection receiving device may receive the screen projection data including the matching parameters.

[0111] According to an exemplary embodiment of the present disclosure, the matching parameters include a resolution matching gear, and the screen projection receiving device can adjust the resolution of the screen according to the resolution matching gear. In an embodiment of the present disclosure, the matching parameters can be the resolution matching gear obtained by the screen projection device through resolution rule matching. After the screen projection receiving device receives the projection data including the resolution matching gear (or the encoded resolution matching gear), the screen projection receiving device can adjust the resolution of the screen according to the resolution matching gear. By adjusting the resolution of the screen projection receiving device, the source resolution of the source data can be better matched, avoiding high power consumption and easy heating of the screen projection receiving device caused by too high resolution, or reduced display effect of the screen projection receiving device caused by too low resolution.

[0112] According to an exemplary embodiment of the present disclosure, the matching parameters include matching frame rates, and the screen projection receiving device may sort the supported refresh rates, determine the matching refresh rate that matches the matching frame rate based on the sorted supported refresh rates, and adjust the refresh rate of the screen according to the matching refresh rate. In an embodiment of the present disclosure, the matching parameter may be the matching frame rate obtained by the screen projection device through frame rate rule matching. After the screen projection receiving device receives the projection data including the matching frame rate (or the encoded matching frame rate), the screen projection receiving device may obtain the matching refresh rate of the screen projection receiving device through refresh rate rule matching based on the matching frame rate and the supported refresh rate (screen projection receiving device parameter). The supported refresh rate here is the refresh rate supported by the hardware of the screen projection receiving device.

[0113] The refresh rate rule matching here adopts the same rule matching as the refresh rate rule matching on the projection device side. It only needs to replace the target frame rate and the supported refresh rate of the projection device in the above refresh rate rule matching with the matching frame rate and the supported refresh rate of the projection receiving device respectively. This disclosure will not go into details about this.

[0114] After the screen projection receiving device obtains the matching refresh rate, it can adjust the screen refresh rate according to the matching refresh rate. By adjusting the refresh rate of the screen projection receiving device, the target frame rate of the source data can be better matched, avoiding high power consumption and heating of the screen projection receiving device caused by too high refresh rate, or reduced display effect of the screen projection receiving device caused by too low refresh rate.

[0115] In step S630, the received encoded source data is decoded to obtain decoded data. According to an embodiment of the present disclosure, the decoding process here is the inverse process of the encoding process of the above-mentioned projection device, and may correspond to the encoding process. Therefore, the projection receiving device may decode the encoded source data based on the matching parameters in the projection data to obtain decoded data. Specifically, the projection receiving device may decode the encoded source data based on the resolution matching gear, and the projection receiving device may decode the encoded source data based on the matching frame rate. In addition, the projection receiving device may decode the encoded source data based on the horizontal and vertical screen modes (projection receiving device parameters) of the projection receiving device.

[0116] In step S640, display is performed on a screen based on the decoded data.

[0117] According to an exemplary embodiment of the present disclosure, the screen projection receiving device can perform display enhancement processing on the decoded data to obtain display enhanced data, and display it on the screen based on the display enhanced data. The display enhancement processing includes at least one of interpolation processing, super-resolution processing and color processing.

[0118] In an embodiment of the present disclosure, the screen projection receiving device may adjust the parameters of the screen according to the received matching parameters included in the screen projection data. Specifically, the screen projection receiving device may adjust the resolution of the screen according to the received resolution matching gear. If the received resolution matching gear is low, for example, the resolution gear information of the screen projection receiving device includes 540p, 720p, 1080p, 2K and 4K, etc., and the resolution matching gear is 720p. Alternatively, the screen projection receiving device may also receive a matching frame rate, and determine a matching refresh rate that matches the matching frame rate, and adjust the refresh rate of the screen according to the matching refresh rate. If the matching frame rate is also low for the screen projection receiving device, for example, only 60fps, and the data frame rate that the screen projection receiving device can support includes 120fps, and the corresponding refresh rate is 120Hz. Alternatively, the resolution matching gear and matching frame rate received by the screen projection receiving device are both low. In the above case, the hardware resources of the screen projection receiving device are not fully utilized, and the effect of the screen projection is difficult to be satisfactory.

[0119] In an embodiment of the present disclosure, a screen projection receiving device, especially a large-screen screen projection receiving device, can perform display enhancement processing on the decoded data to obtain display enhanced data and display it on the screen based on the display enhanced data. Display enhancement processing includes at least one of frame insertion processing, super-resolution processing, and color processing. For example, if the matching frame rate is low (the decoded data frame rate is low), then frame insertion processing can be performed to increase the frame rate of the data. For another example, if the resolution matching gear is low, then super-resolution processing can be performed to improve the resolution. For another example, if the color effect of the picture is not good, then color processing can be performed to improve image performance. Of course, any combination of the above three processes can also be performed, such as frame insertion processing + super-resolution processing, or super-resolution processing + color processing, etc., to make full use of the hardware performance of the screen projection receiving device and enhance the user experience.

[0120] According to an exemplary embodiment of the present disclosure, interpolation processing may include: interpolation based on a motion compensation algorithm to obtain data for supplementary interpolation; super-resolution processing may include: resolution improvement using a super-resolution model to obtain resolution-improved data; color processing may include: color processing through a high dynamic range HDR algorithm to obtain color-processed data.

[0121] Figure 7 A schematic diagram of a display enhancement process according to an exemplary embodiment is shown.

[0122] Reference Figure 7 After the screen projection receiving device decodes the received encoded source data to obtain decoded data, it can input the decoded data into the display enhancement module for display enhancement processing. The display enhancement module includes three sub-modules, namely, the interpolation sub-module, the supramolecular module and the color processing sub-module (HDR), wherein the interpolation sub-module and the supramolecular module can be processed based on the NPU chip capability of the screen projection receiving device. Specifically, the interpolation sub-module can perform interpolation processing based on the motion compensation (MEMC) algorithm to obtain data for supplementary interpolation, wherein the motion compensation algorithm can also be implemented through deep learning. Figure 7 In the , after the interactive processing of the interpolation submodule and the NPU chip, the original frame data is added with interpolation data. The supramolecular module can use super-resolution models such as FSSR to increase the resolution and supplement the image details to obtain data with increased resolution. Figure 7 In the process, after interactive processing between the supramolecular module and the NPU chip, for example, HD / FHD data is processed into 4K / UHD data. The color processing module performs color processing through the high dynamic range HDR algorithm to obtain color-processed data, further improving the display effect.

[0123] In the embodiments of the present disclosure, not all decoded data must be processed by the three modules, but can be processed by at least one of the three modules as needed. Figure 7 The figure takes the processing of three modules as an example for explanation, and the positions of the interpolation submodule and the supramolecular module in the figure can be exchanged, but the color processing submodule needs to be after the interpolation submodule and the supramolecular module, that is, color processing is generally performed on the data after supplementary interpolation and / or resolution enhancement.

[0124] As described above, according to the screen projection method applied to the screen projection receiving device of the exemplary embodiment, on the one hand, the display parameters of the screen projection receiving device can be adjusted by matching parameters, further reducing power consumption and improving heating problems, thereby improving screen projection performance. On the other hand, in the current screen projection technology, the pressure of screen projection resource processing is all on the screen projection device, while the above-mentioned screen projection method performs display enhancement processing on the screen projection receiving device, which alleviates the processing pressure on the screen projection device side, fully utilizes the hardware resources of the screen projection receiving device (especially the large-screen display device), improves the display effect, and displays the picture more clearly and smoothly, thereby improving the user experience.

[0125] Figure 8 A schematic flowchart of a screen projection method according to an exemplary embodiment is shown.

[0126] Reference Figure 8 , in step S810, the screen projection device initiates a screen projection request to the screen projection receiving device. After the screen projection device initiates the screen projection request, the screen projection device obtains the screen projection device parameters and the source data parameters. In step S820, the screen projection device establishes a handshake connection with the screen projection receiving device. During the handshake connection process, the screen projection receiving device sends the screen projection receiving device parameters to the screen projection device, and correspondingly, the screen projection device receives the screen projection receiving device parameters sent by the screen projection receiving device. In step S830, the screen projection device automatically matches the rules. Here, the screen projection device obtains matching parameters through rule matching based on the screen projection parameters including the screen projection device parameters, the source data parameters and the screen projection receiving device parameters. The rule matching here includes: resolution rule matching, refresh rate and frame rate rule matching, and horizontal and vertical screen mode rule matching. Among them, the resolution matching gear can be obtained by the resolution rule matching, the matching refresh rate and frame rate rule matching can be obtained by the refresh rate and frame rate rule matching, and the matching screen mode can be obtained by the horizontal and vertical screen mode rule matching.

[0127] Fig. 9 FIG. 1 shows a screen projection device adaptive module according to an exemplary embodiment. The screen projection device may include a screen projection device adaptive module to implement the above steps S810-S830. Fig. 9, the projection device adaptive module 900 includes a parameter acquisition submodule 910, a projection handshake submodule 920 and an automatic matching submodule 930. Among them, the parameter acquisition submodule 910 obtains the source data parameters through the system services (service) such as SurfaceFlinger of the projection device, including the resolution, frame rate, and other information of the current application, and can also obtain the projection device parameters, including the supported refresh rate, horizontal and vertical screen mode, etc. The projection handshake submodule is mainly used for the interactive handshake between the projection device and the projection receiving device to obtain the parameters of the projection receiving device, including the resolution gear information, horizontal and vertical screen mode and other information. The automatic matching submodule 930 performs rule matching, and the rule matching here includes: resolution rule matching, refresh rate and frame rate rule matching, and horizontal and vertical screen mode rule matching.

[0128] return Figure 8 In step S840, the screen projection device encodes the source data. Here, the screen projection device can encode the source data based on the matching parameters (adjust the encoding parameters of the source data), and the matching parameters include resolution matching gear, matching frame rate and matching screen mode. Then, the screen projection device sends the projection data including the encoded source data to the screen projection receiving device. Correspondingly, the screen projection receiving device receives the projection data including the encoded source data. In step S850, the screen projection receiving device decodes the encoded source data to obtain decoded data. In step S860, the screen projection receiving device performs display enhancement processing on the decoded data, and displays it on the screen based on the display enhanced data.

[0129] Fig.10 FIG. 1 shows an intelligent display enhancement module of a screen projection receiving device according to an exemplary embodiment. The screen projection receiving device includes an intelligent display enhancement module of the screen projection receiving device to implement the above step S860. Fig.10 The intelligent display enhancement module 1000 of the projection receiving device includes three sub-modules, namely, the interpolation sub-module 1010, the supramolecular module 1020 and the HDR sub-module 1030. Among them, the interpolation sub-module 1010 uses the NPU chip of the projection receiving device to perform real-time interpolation based on deep learning to improve the frame rate of the data, making the projection picture smoother and the transition more natural. The supramolecular module 1020 uses the NPU chip of the projection receiving device and uses super-resolution models such as FSSR to perform image processing on data resources, supplement image details, and improve the quality of the projection picture. The HDR module 1030 performs image color processing on data resources through the HDR algorithm of the projection receiving device to improve image performance and make the picture clearer and more realistic.

[0130] Fig.11 A schematic diagram of screen projection from a terminal to a television according to an exemplary embodiment is shown.

[0131] Reference Fig.11, the screen projection device is a mobile terminal (such as a mobile phone), and the screen projection receiving device is a television (TV). Taking the screen projection resource as a game resource as an example, after the user's mobile terminal opens the game (for example, Honor of Kings), a screen projection request is initiated and a handshake connection is established with the TV. The screen projection parameters obtained by the mobile terminal may include: game resource resolution 720p, frame rate (target frame rate) 60fps; displayed as 720p on the mobile terminal, and displayed in landscape mode; the resolution gear information on the TV includes 540p, 720p, 1080p, 2K and 4K, and supports refresh rates including 60Hz, 90Hz and 120Hz, and is displayed in landscape mode. According to the screen projection parameters (including game resource parameters, mobile terminal parameters and TV parameters), the automatically matched resolution matching gear is 720p, the matching refresh rate is 60Hz, the matching frame rate is 60fps, and the matching screen mode is landscape mode.

[0132] After receiving the projection data, the TV side decodes it and uses interpolation processing based on the TV's NPU chip to increase the frame rate (matching the frame rate of 60fps) to 120fps, uses super-resolution processing to increase the resolution from 720p to 2K / 4K, and enhances the image quality through the HDR algorithm.

[0133] Fig.12 A schematic diagram of screen projection for a landscape and portrait screen adaptation scenario according to an exemplary embodiment is shown.

[0134] Reference Fig.12 , the screen projection device is a mobile terminal (such as a mobile phone), and the screen projection receiving device is a TV (TV). At this time, the mobile terminal is in portrait mode, and the TV is in landscape mode. When the landscape and portrait modes of the two are inconsistent, the landscape mode of the TV is used as the matching screen mode, and the corresponding UI effect (landscape mode) is displayed on the TV to improve the user experience.

[0135] Fig.13 A schematic diagram of screen projection of a multi-person conference scenario according to an exemplary embodiment is shown.

[0136] Reference Fig.13 Taking a multi-person conference as an example, all devices in the Metaverse can access the same virtual environment through networking, and can also conduct multi-person conferences by receiving source data and displaying it in a specific UI. For example, the resources on the terminal device in the conference are projected onto the conference UI. Since the terminal device can obtain matching parameters through rule matching and encode the source data based on the matching parameters, the source data can be adjusted according to the target device (conference UI) (for example, the encoding parameters are adjusted).

[0137] Fig.14 A schematic diagram showing screen projection of a multiplayer online game scene according to an exemplary embodiment.

[0138] Reference Fig.14Taking multiplayer online games as an example, it is necessary to interact with multiple source data in the virtual world. Different source data can be intelligently adaptively matched based on rule matching. According to the projection resources (source data) and the parameters of the projection receiving device, appropriate parameters can be selected and finally displayed in the virtual world.

[0139] Fig.15 A block diagram of a screen projection device according to an exemplary embodiment is shown.

[0140] Reference Fig.15 The screen projection device 1500 includes a parameter acquisition module 1510, a matching parameter acquisition module 1520, an encoding module 1530 and a screen projection data sending module 1540, wherein the parameter acquisition module 1510 can acquire screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device, the matching parameter acquisition module 1520 can obtain matching parameters through rule matching based on the acquired screen projection parameters, the encoding module 1530 can encode the source data based on the matching parameters, and the screen projection data sending module 1540 can send the projection data including the encoded source data to the screen projection receiving device.

[0141] According to an exemplary embodiment of the present disclosure, the parameter acquisition module 1510 can acquire source data parameters and screen projection device parameters, establish a handshake connection with the screen projection receiving device, and based on the establishment of the handshake connection, receive the screen projection receiving device parameters sent by the screen projection receiving device.

[0142] According to an exemplary embodiment of the present disclosure, the source data parameters include the source resolution, and the projection receiving device parameters include the resolution gear information. The matching parameter obtaining module 1520 can calculate the matching degree between each resolution gear in the resolution gear information and the source resolution, and select the resolution matching gear in the resolution gear information based on the calculated matching degree.

[0143] According to an exemplary embodiment of the present disclosure, the encoding module 1530 may encode the source data based on a resolution matching level, wherein the projection data includes a resolution matching level.

[0144] According to an exemplary embodiment of the present disclosure, the source data parameters include a target frame rate, the projection device parameters include a supported refresh rate, the matching parameter obtaining module 1520 can sort the supported refresh rates, determine a matching refresh rate that matches the target frame rate based on the sorted supported refresh rates, and adjust the refresh rate of the screen according to the matching refresh rate.

[0145] According to an exemplary embodiment of the present disclosure, the source data parameters include a target frame rate; the matching parameter obtaining module 1520 can set the target frame rate as a matching frame rate, and the encoding module 1530 can encode the source data based on the matching frame rate, wherein the projection data includes the matching frame rate.

[0146] According to an exemplary embodiment of the present disclosure, the screen projection device parameters include a first screen mode, the screen projection receiving device parameters include a second screen mode, and the matching parameter obtaining module 1520 can determine the consistency of the first screen mode and the second screen mode, wherein the first screen mode and the second screen mode include a horizontal screen mode and a vertical screen mode. When the first screen mode is inconsistent with the second screen mode, the second screen mode is determined to be the matching screen mode.

[0147] According to an exemplary embodiment of the present disclosure, the encoding module 1530 may encode the source data based on the matching screen mode.

[0148] As described above, according to the screen projection device of the exemplary embodiment, parameter matching is performed between the screen projection resource (source data) parameters, the screen projection device parameters, and the screen projection receiving device parameters, and the screen projection resources are encoded by the matched parameters, which reduces latency and power consumption and improves heating problems. By adjusting the display parameters of the screen projection device through matching parameters, power consumption is further reduced, heating problems are improved, and screen projection performance is improved.

[0149] Fig.16 A block diagram of a screen projection receiving device according to an exemplary embodiment is shown.

[0150] Reference Fig.16 , the screen projection receiving device includes: a parameter sending module 1610, a screen projection data receiving module 1620, a decoding module 1630 and a display module 1640. Among them, the parameter sending module 1610 can send the screen projection receiving device parameters to the screen projection device, the screen projection data receiving module 1620 can receive the screen projection data including the encoded source data sent by the screen projection device, the decoding module 1630 can decode the received encoded source data to obtain decoded data, and the display module 1640 can display on the screen based on the decoded data.

[0151] According to an exemplary embodiment of the present disclosure, the parameter sending module 1610 can establish a handshake connection with the screen projection device, and based on the establishment of the handshake connection, send the screen projection receiving device parameters to the screen projection device.

[0152] According to an exemplary embodiment of the present disclosure, the screen projection data also includes matching parameters, and the screen projection receiving device may further include: a parameter adjustment module 1650 (not shown) that can adjust the parameters of the screen according to the matching parameters.

[0153] According to an exemplary embodiment of the present disclosure, the matching parameter includes a resolution matching level, and the parameter adjustment module 1650 may adjust the resolution of the screen according to the resolution matching level.

[0154] According to an exemplary embodiment of the present disclosure, the matching parameters include a matching frame rate, and the parameter adjustment module 1650 can sort the supported refresh rates, determine a matching refresh rate that matches the matching frame rate based on the sorted supported refresh rates, and adjust the refresh rate of the screen according to the matching refresh rate.

[0155] According to an exemplary embodiment of the present disclosure, the display module 1640 may perform display enhancement processing on the decoded data to obtain display-enhanced data, and display on the screen based on the display-enhanced data.

[0156] According to an exemplary embodiment of the present disclosure, the display enhancement process includes at least one of a frame interpolation process, a super-resolution process, and a color process.

[0157] According to an exemplary embodiment of the present disclosure, interpolation processing includes: interpolation based on a motion compensation algorithm to obtain data for supplementary interpolation; super-resolution processing includes: resolution improvement using a super-resolution model to obtain resolution-improved data; and color processing includes: color processing through a high dynamic range HDR algorithm to obtain color-processed data.

[0158] As described above, according to the screen projection receiving device of the exemplary embodiment, on the one hand, the display parameters of the screen projection receiving device can be adjusted by matching parameters, further reducing power consumption and improving heating problems, thereby improving screen projection performance. On the other hand, in the current screen projection technology, the pressure of screen projection resource processing is on the screen projection device, and the above-mentioned screen projection method performs display enhancement processing on the screen projection receiving device, which alleviates the processing pressure on the screen projection device side, fully utilizes the hardware resources of the screen projection receiving device (especially large-screen display devices), improves the display effect, and displays the picture more clearly and smoothly, thereby improving the user experience.

[0159] Fig.17 A screen projection system according to an exemplary embodiment is shown.

[0160] Reference Fig.17 , the screen projection system 1700 includes a screen projection device 1710 and a screen projection receiving device 1720. The screen projection device 1710 can obtain screen projection parameters, wherein the screen projection parameters include screen projection receiving device parameters sent by the screen projection receiving device, obtain matching parameters through rule matching based on the obtained screen projection parameters, encode the source data based on the matching parameters, and send the projection data including the encoded source data to the screen projection receiving device. The screen projection receiving device can send the projection receiving device parameters to the screen projection device, receive the projection data including the encoded source data sent by the screen projection device, decode the received encoded source data to obtain decoded data, and display it on the screen based on the decoded data. Any relevant details involved in the operations performed by the screen projection device 1710 and the screen projection receiving device 1720 can be found in the description about Figure 2 and Figure 6 The relevant description will not be repeated here. Fig.17 The screen projection device 1710 in the screen projection system may also include a parameter acquisition module, a matching parameter acquisition module, an encoding module and a screen projection data sending module, and the screen projection receiving device 1720 may also include a parameter sending module, a screen projection data receiving module, a decoding module, a display module and a parameter adjustment module. Among them, any relevant details related to the specific operations of the corresponding modules included in the screen projection device 1710 and the screen projection receiving device 1720 can be referred to the relevant Fig.15 and Fig.16 The description is not repeated here.

[0161] Fig.18 1800 is a block diagram of an electronic device according to an exemplary embodiment. The electronic device 1800 may be a screen projection device or a screen projection receiving device according to an embodiment of the present disclosure. Fig.18 The electronic device 1800 may include at least one memory 1810 and at least one processor 1820, wherein the at least one memory stores a set of computer executable instructions. When the set of computer executable instructions is executed by the at least one processor, the screen projection method according to the embodiment of the present disclosure is executed.

[0162] Here, the electronic device is not necessarily a single electronic device, but may be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The electronic device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0163] In electronic devices, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0164] The processor can execute instructions or codes stored in the memory, wherein the memory can also store data. Instructions and data can also be sent and received through the network via the network interface device, wherein the network interface device can adopt any known transmission protocol.

[0165] The memory may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor or the like. In addition, the memory may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the memory.

[0166] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0167] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein a computer program is stored thereon, and when the program is executed, the screen projection method according to the present disclosure is implemented. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a terminal, a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0168] According to the screen projection method, device and system of the embodiments of the present disclosure, in general, the purpose of reducing latency, reducing power consumption, improving heat generation and enhancing display effects is achieved through the joint optimization and collaborative work of the screen projection device and the screen projection receiving device. Specifically, parameter matching is performed between the screen projection resource (source data) parameters, the screen projection device parameters and the screen projection receiving device parameters, and the screen projection resources are encoded by the matched parameters, which reduces latency and power consumption and improves the heat generation problem. The display parameters of the screen projection device and the screen projection receiving device are adjusted by matching parameters to further reduce power consumption and improve heat generation problems, thereby improving screen projection performance. Display enhancement processing is performed on the screen projection receiving device side, which alleviates the processing pressure on the screen projection device side, fully utilizes the hardware resources of the screen projection receiving device (especially the large-screen display device), improves the display effect, and displays the picture more clearly and smoothly, thereby improving the user experience.

[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A screen projection method, applied to a screen projection device, comprising: Obtaining screen projection parameters, wherein the screen projection parameters include source data parameters, screen projection device parameters, and screen projection receiving device parameters sent by the screen projection receiving device; Based on the acquired projection parameters, matching parameters are obtained through rule matching; encoding the source data based on the matching parameters; and Sending the screen projection data including the encoded source data to the screen projection receiving device, The source data parameters include source resolution, and the screen projection receiving device parameters include resolution level information. Wherein, obtaining matching parameters through rule matching based on the acquired projection parameters includes: Calculating a matching degree between each resolution level in the resolution level information and the source resolution; The resolution gear with the smallest calculated matching degree is selected from the resolution gear information as the resolution matching gear, and The step of calculating the matching degree between each resolution level and the source resolution includes: In the case where the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are different, the matching degree is calculated as the product of the absolute value of the difference in the number of pixels of the resolution level and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; In the case where the number of pixels in the x-direction and the y-direction differs between the resolution gear and the source resolution, the matching degree is calculated as the product of the absolute value of the difference in the number of pixels in the direction where the number of pixels in the resolution gear and the source resolution differ and the number of pixels in the direction where the number of pixels in the resolution gear and the source resolution differ; and When the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are the same, the matching degree is calculated as 0.

2. The method according to claim 1, wherein: The obtaining of screen projection parameters includes: Establishing a handshake connection with the screen projection receiving device; Based on the establishment of the handshake connection, the screen projection receiving device parameters sent by the screen projection receiving device are received.

3. The method according to claim 1, wherein: The encoding of the source data based on the matching parameter comprises: encoding the source data based on the resolution matching level; Wherein, the screen projection data includes the resolution matching level.

4. The method according to claim 1, wherein: The source data parameters include a target frame rate, and the projection device parameters include a supported refresh rate; The matching parameters are obtained based on the acquired projection parameters through rule matching, including: sorting the supported refresh rates; determining a matching refresh rate that matches the target frame rate based on the sorted supported refresh rates; and adjusting the refresh rate of the screen according to the matching refresh rate.

5. The method according to claim 1, wherein: The source data parameters include a target frame rate; The obtaining of matching parameters through rule matching based on the acquired projection parameters further includes: setting the target frame rate to a matching frame rate; The encoding of the source data based on the matching parameter comprises: encoding the source data based on the matching frame rate; Wherein, the screen projection data includes the matching frame rate.

6. The method according to claim 1, wherein: The screen projection device parameters include a first screen mode, and the screen projection receiving device parameters include a second screen mode; The obtaining of matching parameters through rule matching based on the acquired projection parameters includes: Determining consistency between a first screen mode and a second screen mode, wherein the first screen mode and the second screen mode include a horizontal screen mode and a vertical screen mode; When the first screen mode is inconsistent with the second screen mode, the second screen mode is determined to be a matching screen mode.

7. The method according to claim 6, encoding the source data based on the matching parameter comprises: The source data is encoded based on the matching screen mode.

8. A screen projection method, applied to a screen projection receiving device, comprising: Send the screen projection receiving device parameters to the screen projection device; Receiving screen projection data including encoded source data sent by the screen projection device; Decoding the received encoded source data to obtain decoded data; Displaying on a screen based on the decoded data, Wherein, the screen projection data also includes matching parameters, and the method further includes: adjusting screen parameters according to the matching parameters, The matching parameters include a resolution matching gear, and adjusting the parameters of the screen according to the matching parameters includes: adjusting the resolution of the screen according to the resolution matching gear, The projection receiving device parameters include resolution gear information, and the resolution matching gear is obtained by the projection device calculating the matching degree between each resolution gear in the resolution gear information and the source resolution, and selecting the resolution gear with the smallest calculated matching degree from the resolution gear information, and Among them, when the number of pixels between the resolution level and the source resolution in both the x-direction and the y-direction are different, the matching degree is calculated as the product of the absolute value of the difference between the number of pixels between the resolution level and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; when the number of pixels between the resolution level and the source resolution in only one direction is different in the x-direction and the y-direction, the matching degree is calculated as the product of the absolute value of the difference between the number of pixels between the resolution level and the source resolution in the directions where the number of pixels is different and the number of pixels in the directions where the number of pixels is the same; and when the number of pixels between the resolution level and the source resolution in both the x-direction and the y-direction are the same, the matching degree is calculated as 0.

9. The method according to claim 8, wherein: The step of sending the screen projection receiving device parameters to the screen projection device includes: Establishing a handshake connection with the screen projection device; Based on the establishment of the handshake connection, the screen projection receiving device parameters are sent to the screen projection device.

10. The method according to claim 8, wherein: The matching parameters include matching frame rate; The adjusting the parameters of the screen according to the matching parameters includes: sorting the supported refresh rates; determining a matching refresh rate that matches the matching frame rate according to the sorted supported refresh rates; and adjusting the refresh rate of the screen according to the matching refresh rate.

11. The method according to claim 8, wherein: The displaying on the screen based on the decoded data further comprises: Performing display enhancement processing on the decoded data to obtain display enhanced data; The enhanced data is displayed on the screen.

12. The method according to claim 11, wherein: The display enhancement processing includes at least one of frame insertion processing, super-resolution processing and color processing.

13. A screen projection device, comprising: A parameter acquisition module is configured to acquire screen projection parameters, wherein the screen projection parameters include source data parameters, screen projection device parameters, and screen projection receiving device parameters sent by the screen projection receiving device; A matching parameter obtaining module is configured to obtain matching parameters through rule matching based on the acquired screen projection parameters; an encoding module configured to encode the source data based on the matching parameters; and a screen projection data sending module, configured to send the screen projection data including the encoded source data to the screen projection receiving device, The source data parameters include source resolution, and the screen projection receiving device parameters include resolution level information. Wherein, the matching parameter obtaining module is configured as follows: Calculating a matching degree between each resolution level in the resolution level information and the source resolution; The resolution gear with the smallest calculated matching degree is selected from the resolution gear information as the resolution matching gear, and Wherein, the matching parameter obtaining module is further configured as follows: In the case where the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are different, the matching degree is calculated as the product of the absolute value of the difference in the number of pixels of the resolution level and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; In the case where the number of pixels in the x-direction and the y-direction differs between the resolution gear and the source resolution, the matching degree is calculated as the product of the absolute value of the difference in the number of pixels in the direction where the number of pixels in the resolution gear and the source resolution differ and the number of pixels in the direction where the number of pixels in the resolution gear and the source resolution differ; and When the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are the same, the matching degree is calculated as 0.

14. The device according to claim 13, wherein: The parameter acquisition module is configured as follows: Establishing a handshake connection with the screen projection receiving device; Based on the establishment of the handshake connection, the screen projection receiving device parameters sent by the screen projection receiving device are received.

15. A screen projection receiving device, comprising: A parameter sending module is configured to send the screen projection receiving device parameters to the screen projection device; A screen projection data receiving module, configured to receive the screen projection data including the encoded source data sent by the screen projection device; A decoding module, configured to decode the received encoded source data to obtain decoded data; A display module is configured to display the decoded data on a screen, The screen projection data further includes matching parameters, and the screen projection receiving device further includes: a parameter adjustment module configured to adjust the parameters of the screen according to the matching parameters, The matching parameters include a resolution matching level, and the parameter adjustment module is configured to: adjust the resolution of the screen according to the resolution matching level, The projection receiving device parameters include resolution gear information, and the resolution matching gear is obtained by the projection device calculating the matching degree between each resolution gear in the resolution gear information and the source resolution, and selecting the resolution gear with the smallest calculated matching degree from the resolution gear information, and Among them, when the number of pixels between the resolution level and the source resolution in both the x-direction and the y-direction are different, the matching degree is calculated as the product of the absolute value of the difference between the number of pixels between the resolution level and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; when the number of pixels between the resolution level and the source resolution in only one direction is different in the x-direction and the y-direction, the matching degree is calculated as the product of the absolute value of the difference between the number of pixels between the resolution level and the source resolution in the directions where the number of pixels is different and the number of pixels in the directions where the number of pixels is the same; and when the number of pixels between the resolution level and the source resolution in both the x-direction and the y-direction are the same, the matching degree is calculated as 0.

16. The device according to claim 15, wherein: The parameter sending module is configured as follows: Establishing a handshake connection with the screen projection device; Based on the establishment of the handshake connection, the screen projection receiving device parameters are sent to the screen projection device.

17. The apparatus according to claim 15, wherein: The display module is also configured as: Performing display enhancement processing on the decoded data to obtain display enhanced data; The enhanced data is displayed on the screen.

18. A screen projection method, comprising: The screen projection device obtains screen projection parameters, wherein the screen projection parameters include source data parameters, screen projection device parameters, and screen projection receiving device parameters sent by the screen projection receiving device; based on the obtained screen projection parameters, matching parameters are obtained through rule matching; the source data is encoded based on the matching parameters; and the screen projection data including the encoded source data is sent to the screen projection receiving device, wherein the source data parameters include source resolution, and the screen projection receiving device parameters include resolution gear information. Wherein, the matching parameters are obtained by rule matching based on the acquired projection parameters, including: calculating the matching degree between each resolution gear in the resolution gear information and the source resolution; selecting the resolution gear with the smallest calculated matching degree in the resolution gear information as the resolution matching gear, and Wherein, calculating the matching degree between each resolution level and the source resolution includes: when the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are different, calculating the matching degree as the product of the absolute value of the difference in the number of pixels between the resolution level and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; when the number of pixels of the resolution level and the source resolution in only one direction is different in the x-direction and the y-direction, calculating the matching degree as the product of the absolute value of the difference in the number of pixels between the resolution level and the source resolution in the directions where the number of pixels is different and the number of pixels in the directions where the number of pixels is the same; and when the number of pixels of the resolution level and the source resolution in both the x-direction and the y-direction are the same, calculating the matching degree as 0; The screen projection receiving device receives the projection data including the encoded source data sent by the screen projection device; decodes the received encoded source data to obtain decoded data; and displays on the screen based on the decoded data, wherein the projection data also includes matching parameters, and the screen projection receiving device also adjusts the parameters of the screen according to the matching parameters, wherein the matching parameters include resolution matching levels, and adjusting the parameters of the screen according to the matching parameters includes: adjusting the resolution of the screen according to the resolution matching level.

19. A screen projection system, comprising a screen projection device and a screen projection receiving device, wherein: The screen projection device is configured to obtain screen projection parameters, wherein the screen projection parameters include source data parameters, screen projection device parameters and screen projection receiving device parameters sent by the screen projection receiving device; obtain matching parameters through rule matching based on the acquired screen projection parameters; encode the source data based on the matching parameters; and send the screen projection data including the encoded source data to the screen projection receiving device, wherein the source data parameters include source resolution, and the screen projection receiving device parameters include resolution gear information, The projection device is configured to calculate the matching degree between each resolution level in the resolution level information and the source resolution; select the resolution level with the smallest matching degree calculated in the resolution level information as the resolution matching level, and Wherein, the projection device is further configured to, when the number of pixels of the resolution gear and the source resolution in both the x-direction and the y-direction are different, calculate the matching degree as the product of the absolute value of the difference in the number of pixels of the resolution gear and the source resolution in the x-direction and the absolute value of the difference in the number of pixels in the y-direction; when the number of pixels of the resolution gear and the source resolution in only one direction is different in the x-direction and the y-direction, calculate the matching degree as the product of the absolute value of the difference in the number of pixels of the resolution gear and the source resolution in the directions where the number of pixels is different and the number of pixels in the directions where the number of pixels is the same; and when the number of pixels of the resolution gear and the source resolution in both the x-direction and the y-direction are the same, calculate the matching degree as 0; The screen projection receiving device is configured to send the screen projection receiving device parameters to the screen projection device; receive the screen projection data including the encoded source data sent by the screen projection device; decode the received encoded source data to obtain decoded data; and display on the screen based on the decoded data, wherein the screen projection data also includes matching parameters, and the screen projection receiving device is configured to adjust the parameters of the screen according to the matching parameters, wherein the matching parameters include a resolution matching gear, and the screen projection receiving device is configured to: adjust the resolution of the screen according to the resolution matching gear.

20. An electronic device, comprising: processor; A memory storing a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

21. A computer-readable storage medium, wherein: A computer program is stored thereon, and when the program is executed, the method according to any one of claims 1 to 12 is implemented.

Images