Screen mirroring methods, devices, terminals and storage media

Abstract

This application discloses a screen mirroring method, apparatus, terminal, and storage medium, relating to the field of screen mirroring technology. It includes: when screen mirroring is required, determining a target compatible resolution from at least two compatible resolutions supported by the application based on the desired resolution of the destination device; determining a virtual screen resolution based on the target compatible resolution, wherein the aspect ratio of the virtual screen resolution is consistent with that of the target compatible resolution; determining a target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution, wherein the source encoding resolution is related to the encoding capability of the source device; performing video encoding on the application screen using an encoder with the target encoding resolution to obtain a screen mirroring data stream; and transmitting the screen mirroring data stream to the destination device so that the destination device can display the screen mirrored image based on the screen mirroring data stream. This application's solution helps reduce black borders appearing when the destination device displays the screen mirrored image.

Description

Technical Field

[0001] This application relates to the field of screen projection technology, and in particular to a screen projection method, device, terminal and storage medium. Background Technology

[0002] Screen mirroring is a common function in terminals, allowing users to redirect the screen displayed on a small terminal to a large terminal for display.

[0003] In related technologies, users can use the screen mirroring function provided by the application to mirror the currently displayed application screen to other terminals. Summary of the Invention

[0004] This application provides a screen mirroring method, device, terminal, and storage medium. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide a screen mirroring method, the method being used at the source end, the method comprising:

[0006] When there is a need for screen mirroring, the target adapted resolution is determined from at least two adapted resolutions supported by the application based on the desired resolution of the destination device. Different adapted resolutions are used to adapt to screens with different resolutions.

[0007] The virtual screen resolution is determined based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution.

[0008] Based on the virtual screen resolution, the source encoding resolution, and the desired resolution, the target encoding resolution is determined, wherein the source encoding resolution is related to the encoding capability of the source.

[0009] The application screen is video encoded using an encoder with the target encoding resolution to obtain a projection data stream;

[0010] The projection data stream is transmitted to the destination so that the destination can perform projection display based on the projection data stream.

[0011] On the other hand, embodiments of this application provide a screen projection device, the device comprising:

[0012] The processing module is used to determine the target adaptation resolution from at least two adaptation resolutions supported by the application based on the desired resolution of the destination device when there is a screen casting requirement. Different adaptation resolutions are used to adapt to screens with different resolutions.

[0013] The processing module is used to determine the virtual screen resolution of the virtual screen based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution.

[0014] The processing module is used to determine the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution, wherein the source encoding resolution is related to the encoding capability of the source.

[0015] The processing module is used to encode the application screen using an encoder with the target encoding resolution to obtain a projection data stream;

[0016] The transmission module is used to transmit the projection data stream to the destination terminal so that the destination terminal can perform projection display based on the projection data stream.

[0017] The transmission module is used to transmit the projected screen to the destination terminal so that the destination terminal can display the projected screen.

[0018] On the other hand, embodiments of this application provide a terminal, which includes dual SIM cards, a processor, and a memory; the memory stores at least one instruction, which is executed by the processor to implement the screen projection method as described above.

[0019] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the screen projection method as described above.

[0020] On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. The terminal's processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal to perform the screen projection method provided above.

[0021] In this embodiment, before the source end projects the screen to the destination end, it first determines the target adaptation resolution with the highest matching degree from a variety of adaptation resolutions supported by the application, based on the desired resolution of the destination end. Then, based on the target adaptation resolution, it determines the virtual screen resolution of the virtual screen carrying the application screen, which improves the matching degree between the virtual screen resolution and the desired resolution of the destination end and helps to reduce the black borders that appear when the destination end displays the projected screen. Furthermore, the source end combines the virtual screen resolution, the source end encoding resolution which characterizes the source end's encoding capability, and the desired resolution to determine the target encoding resolution. Based on the target encoding resolution, it encodes the application screen carried by the virtual screen through an encoder, thereby transmitting the encoded projected screen data stream to the destination end. This ensures that the projected screen process conforms to the performance of both ends and avoids the problem of projected screen abnormalities caused by exceeding the performance of the source end or the destination end. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0024] Figure 2 This is a flowchart of a screen mirroring method provided in an exemplary embodiment of this application;

[0025] Figure 3 This is a schematic diagram illustrating an exemplary embodiment of the target adaptation resolution determination process of this application;

[0026] Figure 4 This is a flowchart illustrating the virtual screen resolution determination process in an exemplary embodiment of this application;

[0027] Figure 5 This is a schematic diagram illustrating an exemplary embodiment of the virtual screen resolution determination process of this application;

[0028] Figure 6 This is a schematic diagram illustrating an exemplary embodiment of the target coding resolution determination process of this application;

[0029] Figure 7 This is a timing diagram illustrating the interaction process between the source and destination ends, as shown in an exemplary embodiment of this application.

[0030] Figure 8 This is a flowchart illustrating the reverse control command response process in an exemplary embodiment of this application;

[0031] Figure 9 This is a schematic diagram illustrating an exemplary embodiment of the coordinate transformation process of this application;

[0032] Figure 10 This is a structural block diagram of a screen projection device provided in an exemplary embodiment of this application;

[0033] Figure 11 This is a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] Please refer to Figure 1 This diagram illustrates an implementation environment provided by an exemplary embodiment of this application. This implementation environment may include a source end 110 and a destination end 120.

[0036] The source (110) is an electronic device with screen mirroring functionality. This screen mirroring function can be a native feature of the electronic device or a feature provided by an application installed on the device. The electronic device can be a smartphone, tablet, or personal computer, etc. Figure 1 (Taking the source device 110 as a smartphone as an example for illustration), the embodiments of this application do not limit the specific type of electronic device.

[0037] The sink 120 is an electronic device with screen projection display functionality. Specifically, the sink 120 acts as the receiving end of the projected image transmitted from the source 110, used to display the projected image. This screen projection display functionality can be a native function provided by the electronic device or a function provided by an application installed on the electronic device. The electronic device can be a smart TV, projector, tablet computer, personal computer, in-vehicle infotainment system, etc. Figure 1 (Taking a smart TV as an example for illustration, the embodiments of this application do not limit the specific type of electronic device. Optionally, the screen size of the destination 120 is larger than the screen size of the source 110, thereby achieving a better display effect during the screen projection process.)

[0038] A wired or wireless connection is established between the source end 110 and the destination end 120. Specifically, the source end 110 and the destination end 120 can establish a wired redirection connection via a data cable, or the source end 110 and the destination end 120 can establish a wireless redirection connection by accessing the same network (data exchange is performed through a wireless access point in the network).

[0039] In this embodiment of the application, the application installed in the source 110 has the function of adapting to electronic devices with different screen resolutions. That is, the application supports at least two adapted resolutions and can use the adapted resolution with the highest matching degree to display the application screen according to the screen resolution of the electronic device.

[0040] Leveraging the application's adaptation capabilities, the source device 110 negotiates resolution with the destination device 120 before screen projection. During this negotiation process, the source device 110 can select the target adaptation resolution with the highest matching degree from a variety of adaptation resolutions based on the desired resolution of the destination device 120, and create a virtual screen based on the target adaptation resolution, thereby reducing the black borders that appear when the destination device 120 displays the projected image.

[0041] In addition, the source end 110 will determine the target encoding resolution by combining its own performance and that of the destination end 120. Based on the target encoding resolution, it will encode the application screen carried by the virtual screen and transmit the encoded projection data stream to the destination end 120 for decoding and display. This ensures that the encoding process meets the performance of the source end and the display process meets the performance of the destination end, thereby improving the stability of the projection process.

[0042] Of course, during the screen mirroring process, in addition to transmitting screen mirroring data streams, including image data, to the destination 120, the source end 110 can also transmit audio data to the destination 120 so that the destination 120 can decode and play it.

[0043] In one possible implementation, a P2P connection is established between the source end 110 and the destination end 120, which includes an RTSP socket connection and an RTP socket connection. The RTSP socket connection is used for resolution negotiation between the source and destination ends, while the RTP socket connection is used for the transmission of image and audio data.

[0044] In some embodiments, the destination terminal 120 may further include an input device, which may be a touch screen, touchpad, keyboard, mouse, joystick, etc., and this embodiment is not limited thereto. During the display of the projected screen on the destination terminal 120, the user can control elements in the projected screen through the input device on the destination terminal 120. After receiving the control operation, the destination terminal 120 sends the reverse control command triggered by the input device to the source terminal 110 via the connection. The source terminal 110 then simulates the effect of the user controlling the screen through the source terminal 110 based on the reverse control command.

[0045] See Figure 2 , Figure 2 This is a flowchart of a screen mirroring method provided in an exemplary embodiment of this application. This embodiment uses this method for... Figure 1Taking the source end shown as an example, the method can include the following steps.

[0046] Step 202: When there is a need for screen mirroring, the target adapted resolution is determined from at least two adapted resolutions supported by the application based on the desired resolution of the destination device. Different adapted resolutions are used to adapt to screens with different resolutions.

[0047] In one possible implementation, upon receiving a screen mirroring command, the source end establishes a connection with the destination end, and the source end obtains the desired resolution from the destination end through this connection. Optionally, the desired resolution can be the screen resolution of the destination end's display.

[0048] In this embodiment, to achieve better display results on devices with different screen resolutions, the application supports at least two compatible resolutions. When the source device runs the application, the application adopts the most compatible resolution based on the source device's screen resolution.

[0049] Unlike related technologies that create virtual screens based on the source screen resolution, in this embodiment, by leveraging the application's adaptation capabilities, the source determines the target adaptation resolution that best matches the desired resolution from at least two adaptation resolutions supported by the application, so that the virtual screen resolution of the virtual screen can be determined subsequently based on this target adaptation resolution.

[0050] Because the target resolution matches the desired resolution to the highest degree, it can minimize the black borders around the projected image displayed on the target device.

[0051] The target adaptation resolution may be different from or the same as the adaptation resolution used by the application's source end.

[0052] In an illustrative example, the application indicates compatible resolutions including 1920×1080, 2560×1600, 1200×800, 1368×720, and 2572×1100. Since the source screen resolution is 1920×1080, the application uses this 1920×1080 compatible resolution when displayed on the source. When the desired resolution of the destination is 2560×1488, the source determines 2560×1600 as the target compatible resolution with the highest matching degree.

[0053] Step 204: Determine the virtual screen resolution based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution.

[0054] When casting, the application's screen is drawn in a created virtual display, which is not visible to the user. After determining the target resolution, the source device further determines the virtual screen resolution based on the target resolution, so that the virtual screen can be created subsequently based on this resolution.

[0055] To minimize black borders, the aspect ratio of the virtual screen resolution needs to be consistent with the target adapted resolution. Optionally, the virtual screen resolution can be obtained by scaling the width and height of the target adapted resolution proportionally.

[0056] For example, when the target adaptation resolution is 2560×1600, the determined virtual screen resolution is 2412×2132.

[0057] In some embodiments, the virtual screen resolution will be notified to the application so that the application can generate an application screen based on the virtual screen resolution.

[0058] Step 206: Determine the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution. The source encoding resolution is related to the encoding capability of the source.

[0059] During the screen mirroring process, the source end needs to encode the application screen displayed on the virtual screen and transmit the encoded screen mirroring data stream to the destination end. Since the encoding capability of the source end is limited and the display capability of the destination end is limited, in order to avoid the screen mirroring capability requiring the capabilities of the source end and the destination end, in one possible implementation, the source end needs to comprehensively consider the encoding resolution of the source end and the expected resolution of the destination end when determining the encoding resolution adopted by the encoder.

[0060] Among them, the source-end encoding resolution is the maximum resolution of the image input to the source encoder. The stronger the source-end encoding performance, the larger the source-end encoding resolution; conversely, the weaker the source-end encoding performance, the smaller the source-end encoding resolution.

[0061] Optionally, the determined target coding resolution is less than or equal to the source coding resolution and less than or equal to the desired resolution of the destination.

[0062] Step 208: The application screen is video encoded using an encoder with the target encoding resolution to obtain the projection data stream.

[0063] For each frame of the application screen displayed on the virtual screen, the source end encodes the application screen using an encoder to obtain the projection data stream corresponding to the continuous application screen.

[0064] In some embodiments, the resolution synthesized by SurfaceFlinger is the target encoding resolution, and correspondingly, the resolution of the application screen input to the encoder is the target encoding resolution.

[0065] In one possible implementation, the source end creates a virtual screen based on the virtual screen resolution and sets an encoder based on the target encoding resolution. The application screen generated by the subsequent application is displayed on the virtual screen and scaled to the target encoding resolution. The source end encodes the application screen at the target encoding resolution based on the encoder to obtain the projection data stream.

[0066] Step 210: Transmit the screen projection data stream to the destination so that the destination can perform screen projection display based on the screen projection data stream.

[0067] Ultimately, the source end transmits the projection data stream to the destination end through the connection between the source end and the destination end, which then performs projection display based on the received projection data stream.

[0068] In some embodiments, when the target device displays the screen, the screen image can be enlarged proportionally to further reduce the black borders. This embodiment does not limit this.

[0069] In one possible implementation, the source sends the target encoding resolution to the destination via a connection, so that the destination sets up a decoder based on the target encoding resolution and then uses the decoder to decode the projected data stream.

[0070] In summary, in this embodiment, before projecting the screen from the source end to the destination end, the target adaptation resolution with the highest matching degree is first determined from the various adaptation resolutions supported by the application, based on the desired resolution of the destination end. Then, the virtual screen resolution of the virtual screen carrying the application screen is determined based on this target adaptation resolution, improving the compatibility between the virtual screen resolution and the desired resolution of the destination end, and helping to reduce black borders when the projected screen is displayed on the destination end. Furthermore, the source end comprehensively considers the virtual screen resolution, the source end encoding resolution (characterizing the source end's encoding capability), and the desired resolution to determine the target encoding resolution. Based on the target encoding resolution, the application screen carried by the virtual screen is encoded using an encoder, thereby transmitting the encoded projected screen to the destination end. This ensures that the projection process conforms to the performance of both ends, avoiding projection anomalies caused by exceeding the performance limits of either the source or destination end.

[0071] Since the aspect ratio of the target adaptation resolution is the same as the aspect ratio of the final projected screen, the difference between the resolution of the projected screen and the desired resolution of the target device is the black border. Accordingly, the area of ​​the black border can be expressed as: (reference edge × aspect ratio difference) × reference edge.

[0072] In landscape mode, the "oil drum" mode (with black borders on the left and right sides of the screen) should be prioritized, while in portrait mode, the "envelope" mode (with black borders at the top and bottom of the screen) should be prioritized. Based on these considerations, the shorter side should be prioritized as the reference edge. Once the reference edge is determined, the black border area will be minimized when the aspect ratio of the adapted resolution is less than or equal to the aspect ratio of the desired resolution and the ratios are closest.

[0073] In one possible implementation, the source end may include the following steps when determining the target adaptation resolution:

[0074] 1. Determine the first aspect ratio of the desired resolution and the second aspect ratio of the adapted resolution.

[0075] In some embodiments, the aspect ratio of the resolution is the ratio of the longer side to the shorter side, that is, the aspect ratio is greater than or equal to 1.

[0076] In one possible implementation, the source determines the long and short sides in the resolution, and then determines the aspect ratio based on the long and short sides.

[0077] Indicative, such as Figure 3 As shown, the first aspect ratio of the desired resolution determined by the source is 1.72, and the second aspect ratios of the six adapted resolutions supported by the application are 1.778, 1.6, 1.5, 1.9, 2.5 and 1.778.

[0078] 2. In the case where there is a second aspect ratio that is less than or equal to the first aspect ratio, the second aspect ratio that is closest to the first aspect ratio among the second aspect ratios that are less than or equal to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution.

[0079] In one possible implementation, the source determines whether a second aspect ratio smaller than the first aspect ratio exists. If it does, the second aspect ratio that is smaller than and closest to the first aspect ratio is determined as the target aspect ratio, and the corresponding adaptation resolution is further determined as the target adaptation resolution. In this way, the area of ​​the black borders on both sides of the projected screen is minimized after adopting the target adaptation resolution.

[0080] Indicative, such as Figure 3 As shown, there are two adaptation resolutions with a second aspect ratio smaller than the desired resolution's first aspect ratio. Since the second aspect ratio of 1.6 is closest to the first aspect ratio of 1.72, the source end determines 2560×1600 as the target adaptation resolution.

[0081] 3. In the absence of a second aspect ratio that is less than or equal to the first aspect ratio, the second aspect ratio that is closest to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution.

[0082] In the absence of a second aspect ratio smaller than the first aspect ratio, the source end also uses the principle of closest approximation to determine the second aspect ratio closest to the first aspect ratio as the target aspect ratio.

[0083] In one possible implementation, before determining the target adaptation resolution based on the aspect ratio, the source end filters out adaptation resolutions that do not match the screen state represented by the adaptation resolution, based on the screen state represented by each adaptation resolution and the screen states represented by each adaptation resolution. When subsequently determining the target adaptation resolution, the search is only performed among the filtered adaptation resolutions.

[0084] In some embodiments, when the screen state represented by the adapted resolution is inconsistent with the screen state represented by the desired resolution, the source-side filtering adapts the resolution, and the screen state includes landscape state and portrait state.

[0085] Optionally, if the width of the resolution is greater than its height, the screen state is determined to be landscape mode; if the width of the resolution is less than its height, the screen state is determined to be portrait mode.

[0086] Indicative, such as Figure 3 As shown, since the desired resolution represents a landscape screen, while the adapted resolution of 1080×1920 represents a portrait screen, the source side filters out the adapted resolution.

[0087] In this embodiment, the source end determines the target adaptation resolution based on the aspect ratio of the resolution and selects the closest aspect ratio as the principle. This can minimize the black area on both sides of the projected screen and help improve the projected display effect.

[0088] In some embodiments, different adaptation resolutions correspond to their respective pixel densities, while the pixel density of the virtual screen created at the source end is consistent with the pixel density of the physical screen. If the target adaptation resolution is directly determined as the virtual screen resolution of the virtual screen, the inconsistent pixel densities may lead to image distortion (such as abnormal UI layout), affecting the projection display effect. Therefore, to ensure the projection effect, in one possible implementation, such as... Figure 4 As shown, determining the virtual screen resolution at the source end can include the following steps.

[0089] Step 204A: Determine the first pixel density corresponding to the target adaptation resolution and the second pixel density of the source screen.

[0090] Pixel density can be expressed in DPI (Dots Per Inch).

[0091] Optionally, each adaptation resolution corresponds to its own DPI. After the source determines the target adaptation resolution, the DPI corresponding to the target adaptation resolution is obtained.

[0092] Indicative, such as Figure 5 As shown, the source determines the DPI corresponding to the target adaptation resolution as 360 (i.e., the first pixel density), and the source screen's DPI is 480 (i.e., the second pixel density).

[0093] Step 204B: Determine the pixel density ratio of the first pixel density to the second pixel density.

[0094] Indicative, such as Figure 5 As shown, the source-side pixel density ratio is determined to be 480 / 360 = 1.33.

[0095] Step 204C: Scaling the target adaptation resolution based on the pixel density ratio to obtain the virtual screen resolution, where the pixel density of the virtual screen resolution is the second pixel density.

[0096] Based on the principle of maintaining the physical size, the source device scales the target adaptation resolution based on the pixel density ratio (scaling proportionally to ensure the aspect ratio remains unchanged) to obtain the virtual screen resolution. Specifically, when the first pixel density is greater than the second pixel density, the source device proportionally reduces the target adaptation resolution to lower the pixel density; when the first pixel density is less than the second pixel density, the source device proportionally enlarges the target adaptation resolution and increases the pixel density.

[0097] In some embodiments, the virtual screen resolution can be expressed as VirtualLong × VirtualShort, where VirtualLong = PadLong * SourceDPI / PadDPI, VirtualShort = PadShort * SourceDPI / PadDPI, PadLong is the width of the target adapted resolution, PadShort is the height of the target adapted resolution, SourceDPI is the pixel density of the source device, and PadDPI is the pixel density corresponding to the target adapted resolution.

[0098] Indicative, such as Figure 5 As shown, the source determines the width of the virtual screen resolution to be 2560×480÷360=3412, and the height of the virtual screen resolution to be 1600×480÷360=2132, ensuring that the DPI of the created virtual screen is 480.

[0099] In this embodiment, the source end scales the target adaptation resolution based on the target adaptation resolution and the pixel density corresponding to the source end screen to obtain the virtual screen resolution. This ensures that the virtual screen resolution has the same pixel density as the source end screen and the same aspect ratio as the target adaptation resolution, avoiding image distortion caused by inconsistent pixel density and helping to improve the display quality of the projected image.

[0100] When determining the target coding resolution by combining the performance of the source and destination ends, the following two situations may exist:

[0101] 1. The virtual screen resolution is smaller than the source encoding resolution, and the virtual screen resolution is smaller than the desired resolution;

[0102] 2. The virtual screen resolution is greater than the source encoding resolution, and / or the virtual screen resolution is greater than the expected resolution.

[0103] In this context, "first resolution" means that the width of the first resolution is greater than the width of the second resolution, and the height of the first resolution is greater than the height of the second resolution.

[0104] In the first scenario described above, when the application screen carried by the virtual screen is used as the projection screen, the performance requirements of both the source and destination ends can be met simultaneously. Therefore, in one possible implementation, when the virtual screen resolution is less than the source end's encoding resolution and the virtual screen resolution is less than the desired resolution, the source end determines the virtual screen resolution as the target encoding resolution.

[0105] In the second scenario described above, directly using the application screen displayed on the virtual screen as the projection screen may exceed the performance requirements of the source and / or destination ends. Therefore, in another possible implementation, when the virtual screen resolution is greater than the source encoding resolution and / or the virtual screen resolution is greater than the desired resolution, the source end determines the target encoding resolution based on the source encoding resolution and the desired resolution. The target encoding resolution is less than or equal to the source encoding resolution and less than or equal to the desired resolution.

[0106] Further discussing the second scenario above, in order to ensure that the determined target coding resolution can simultaneously meet the performance requirements of both the source and destination ends, the source end determines the target coding resolution based on the smaller value between the source coding resolution and the desired resolution.

[0107] The following exemplary embodiments will be used to describe different sub-cases in the second case.

[0108] First case: If the source coding resolution is greater than the desired resolution, the source determines the target coding resolution based on the desired resolution.

[0109] In this context, "source-end coding resolution greater than expected resolution" means that the width of the source-end coding resolution is greater than the width of the expected resolution, and the height of the source-end coding resolution is greater than the height of the expected resolution.

[0110] For example, when the source coding resolution is 3800×2160 and the desired resolution is 1920×1080, the source determines that the source coding resolution is greater than the desired resolution.

[0111] In one possible implementation, the source determines whether to use the height or width of the desired resolution as a reference based on the relationship between the aspect ratio of the target adapted resolution (i.e., the aspect ratio of the virtual screen resolution) and the aspect ratio of the desired resolution, and then determines the target encoding resolution based on the reference.

[0112] Optionally, if the aspect ratio of the target adaptation resolution is less than that of the desired resolution, the source end determines the height of the desired resolution as the target height, and determines the target width based on the target height and the aspect ratio of the target adaptation resolution, thereby determining the target encoding resolution based on the target width and the target height.

[0113] When the aspect ratio of the target adaptation resolution is smaller than that of the desired resolution, in order to ensure that only the left and right sides of the projected screen have black borders, while the top and bottom sides do not, the source end determines the height of the desired resolution as the target height, and uses the target height as a reference to determine the target encoding resolution with the same aspect ratio as the target adaptation resolution.

[0114] In an illustrative example, if both SourceCodecLong (width of the source encoding resolution) and SourceCodecShort (height of the source encoding resolution) are greater than RemoteLong (width of the desired resolution) and RemoteShort (height of the desired resolution), then if RemoteShort*PadLong (width of the target adapted resolution) / PadShort (height of the target adapted resolution) < RemoteLong, then FinalShort (target height) = RemoteShort, and FinalLong (target height) = RemoteShort*PadLong / PadShort.

[0115] Optionally, if the aspect ratio of the target adaptation resolution is greater than that of the desired resolution, the source end determines the width of the desired resolution as the target width, and determines the target height based on the target width and the aspect ratio of the target adaptation resolution, thereby determining the target coding resolution based on the target width and the target height.

[0116] When the aspect ratio of the target adaptation resolution is greater than that of the desired resolution, in order to ensure that only the top and bottom sides of the projected screen have black borders, while the left and right sides do not, the source end determines the width of the desired resolution as the target width, and uses the target width as a reference to determine the target encoding resolution that is consistent with the aspect ratio of the target adaptation resolution.

[0117] In an illustrative example, if both SourceCodecLong and SourceCodecShort are greater than RemoteLong and RemoteShort, and RemoteShort*PadLong / PadShort > RemoteLong, then FinalLong = RemoteLong and FinalShort = RemoteLong*PadShort / PadLong.

[0118] Second seed case: When the source coding resolution is less than the desired resolution, the target coding resolution is determined based on the source coding resolution.

[0119] In this context, "source-end coding resolution less than expected resolution" means that the width of the source-end coding resolution is less than the width of the expected resolution, and the height of the source-end coding resolution is less than the height of the expected resolution.

[0120] For example, when the source coding resolution is 1920×1080 and the desired resolution is 2550×1600, the source determines that the source coding resolution is less than the desired resolution.

[0121] In one possible implementation, the source determines whether to use the height or width of the source encoding resolution as a reference based on the relationship between the aspect ratio of the target adapted resolution (i.e., the aspect ratio of the virtual screen resolution) and the aspect ratio of the desired resolution, and then determines the target encoding resolution based on the reference.

[0122] Optionally, if the aspect ratio of the target adaptation resolution is less than that of the source coding resolution, the source end determines the height of the source coding resolution as the target height, and determines the target width based on the target height and the aspect ratio of the target adaptation resolution, thereby determining the target coding resolution based on the target width and the target height.

[0123] When the aspect ratio of the target adaptation resolution is smaller than that of the source encoding resolution, in order to minimize the black borders on the left and right sides of the projected screen, the source end determines the height of the source encoding resolution as the target height, and uses the target height as a reference to determine the target encoding resolution with the same aspect ratio as the target adaptation resolution.

[0124] In an illustrative example, when both SourceCodecLong and SourceCodecShort are less than RemoteLong and RemoteShort, if SourceCodecShort*PadLong / PadShort < SourceCodecLong, then FinalShort = SourceCodecShort, and FinalLong = SourceCodecShort*PadLong / PadShort.

[0125] Optionally, if the aspect ratio of the target adaptation resolution is greater than that of the source coding resolution, the source end determines the width of the source coding resolution as the target width, and determines the target height based on the target width and the aspect ratio of the target adaptation resolution, thereby determining the target coding resolution based on the target width and the target height.

[0126] When the aspect ratio of the target adaptation resolution is greater than that of the source encoding resolution, in order to minimize the black borders on the top and bottom of the projected screen, the source end determines the width of the source encoding resolution as the target width, and uses the target width as a reference to determine the target encoding resolution with the same aspect ratio as the target adaptation resolution.

[0127] In an illustrative example, if both SourceCodecLong and SourceCodecShort are less than RemoteLong and RemoteShort, and if SourceCodecShort*PadLong / PadShort > SourceCodecLong, then FinalLong = SourceCodecLong and FinalShort = SourceCodecLong*PadShort / PadLong.

[0128] The third seed case: When the source coding resolution is partially greater than the expected resolution and partially less than the expected resolution, the intermediate resolution is determined based on the source coding resolution and the expected resolution; the target coding resolution is then determined based on the intermediate resolution.

[0129] The situation where the source-end coding resolution is partially greater than the expected resolution and partially less than the expected resolution can include the following two cases: 1. The width of the source-end coding resolution is greater than the width of the expected resolution, and the height of the source-end coding resolution is less than the height of the expected resolution; 2. The width of the source-end coding resolution is less than the width of the expected resolution, and the height of the source-end coding resolution is greater than the height of the expected resolution.

[0130] To ensure that the final target coding resolution is less than or equal to the source coding resolution and the desired resolution, the source first needs to determine an intermediate resolution that is less than the source coding resolution and the desired coding resolution, so that the target coding resolution can be determined based on the intermediate resolution.

[0131] Regarding the method for determining the intermediate resolution, in one possible implementation, the source end determines the intermediate width as the smaller value between the width of the source-end encoded resolution and the width of the desired resolution, and determines the intermediate height as the smaller value between the height of the source-end encoded resolution and the height of the desired resolution, thereby determining the intermediate resolution based on the intermediate width and the intermediate height.

[0132] For example, when SourceCodecShort > RemoteShort and SourceCodecLong < RemoteLong, the determined middle height is RemoteShort, the middle width is SourceCodecLong, and the middle resolution is SourceCodecLong × RemoteShort.

[0133] After determining the intermediate resolution, similar to the first and second cases mentioned above, the source determines whether to use the height or width of the intermediate resolution as the benchmark based on the relationship between the aspect ratio of the target adaptation resolution and the aspect ratio of the intermediate resolution, and then determines the target coding resolution based on the benchmark.

[0134] Optionally, if the aspect ratio of the target adaptation resolution is less than that of the intermediate resolution, the source end determines the height of the intermediate resolution as the target height, and determines the target width based on the target height and the aspect ratio of the target adaptation resolution, thereby determining the target encoding resolution based on the target width and the target height.

[0135] When the aspect ratio of the target adaptation resolution is smaller than that of the intermediate resolution, in order to minimize the black borders on the left and right sides of the projected screen, the source end determines the height of the intermediate resolution as the target height, and uses the target height as a reference to determine the target encoding resolution with the same aspect ratio as the target adaptation resolution.

[0136] In an illustrative example, if TempShort (middle height) * PadLong / PadShort < TempLong (middle width), then FinalShort = TempShort, and FinalLong = TempShort * PadLong / PadShort.

[0137] Optionally, if the aspect ratio of the target adaptation resolution is greater than that of the intermediate resolution, the width of the intermediate resolution is determined as the target width, and the target height is determined based on the target width and the aspect ratio of the target adaptation resolution, thereby determining the target encoding resolution based on the target width and the target height.

[0138] When the aspect ratio of the target adaptation resolution is greater than that of the intermediate resolution, in order to minimize the black borders on the top and bottom of the projected screen, the source end determines the width of the intermediate resolution as the target width, and uses the target width as a reference to determine the target encoding resolution with the same aspect ratio as the target adaptation resolution.

[0139] In an illustrative example, if TempShort*PadLong / PadShort>TempLong, then FinalLong=TempLong, and FinalShort=TempLong*PadShort / PadLong.

[0140] exist Figure 3 , 5 Based on the example shown, such as Figure 6 As shown, since the source-end encoding resolution of 2376×1080 is less than the desired resolution of 2560×1488, the source end determines the target encoding resolution based on its source-end encoding resolution. Since the aspect ratio of the source-end encoding resolution (2.2) is greater than the aspect ratio of the target adaptation resolution (1.6), the source end uses the height of the source-end encoding resolution (1080) as the target height and determines the target width as 1728 based on the aspect ratio of 1.6, resulting in a target encoding resolution of 1728×1080. Furthermore, the source end encodes and transmits the application screen based on this target encoding resolution. When the destination end displays the projected screen, the width of the left and right black borders is 416, while the width of the top and bottom black borders is 204. Of course, the destination end can proportionally enlarge the projected screen to achieve a resolution of 2380×1488, thereby eliminating the top and bottom black borders and reducing the left and right black borders.

[0141] In this embodiment, the source end determines the target encoding resolution based on the smaller of the source end encoding resolution and the desired resolution, or based on the intermediate resolution determined based on the source end encoding resolution and the desired resolution, to ensure that the final projection resolution does not exceed the capability range of the source end and the destination end, thereby improving the stability of the projection process.

[0142] Please refer to Figure 7 It shows a timing diagram of the source-destination interaction process provided in an exemplary embodiment of this application.

[0143] Step 701: The destination end and the source end establish a screen projection link. The screen projection link may include a negotiation link and a data stream transmission link.

[0144] Step 702: The destination device notifies the source device of the desired resolution for screen projection.

[0145] Step 703: The source determines the target adaptation resolution from at least two adaptation resolutions based on the desired resolution.

[0146] Step 704: The source determines the virtual screen resolution based on the DPI corresponding to the target adaptation resolution and the DPI of the physical screen.

[0147] Step 705: The source determines the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution.

[0148] Step 706: The source end notifies the destination end of the target coding resolution so that the destination end can set the decoder based on the target coding resolution.

[0149] Step 707: Create a virtual screen on the source side and configure an encoder so that the application screen can be encoded by the encoder to obtain the projection data stream.

[0150] Step 708: The source end transmits the screen projection data stream to the destination end.

[0151] Step 709: The destination device decodes the screen projection data stream and displays the screen projection screen based on the decoding result.

[0152] Step 710: Disconnect the screen sharing link between the source and destination ends to terminate screen sharing.

[0153] As can be seen from the above embodiments, the resolution (i.e., the target encoding resolution) of the final projection data stream transmitted from the source end is not necessarily completely consistent with the virtual screen resolution of the created virtual screen. Therefore, when the destination end supports reverse control of the source end, in order to ensure the accuracy of reverse control, the source end needs to process the received reverse control commands based on the scaling ratio. Figure 8 As shown, the method may also include the following steps.

[0154] Step 801: Determine the scaling ratio based on the virtual screen resolution and the target encoding resolution.

[0155] Since the aspect ratios of both the virtual screen resolution and the target encoding resolution are consistent with the aspect ratio of the target adapted resolution, the source end can determine the scaling ratio as the ratio of the width or the height of the virtual screen resolution and the target encoding resolution.

[0156] Indicative, such as Figure 9As shown, when the virtual screen resolution is 3412×2132 and the target encoding resolution is 1728×1080, the scaling ratio determined by the source end is 1.974.

[0157] Step 802: Upon receiving a reverse control command from the destination, perform coordinate transformation on the operation coordinates contained in the reverse control command based on the scaling ratio.

[0158] In some embodiments, when the destination receives a reverse control operation, it generates a reverse control instruction based on the operation type and operation coordinates of the reverse control operation. The operation type can be a click operation, a long press operation, a swipe operation, a press operation, etc., and the corresponding operation coordinates can be the click position coordinates, the swipe trajectory coordinates, etc., which are not limited in this embodiment.

[0159] It should be noted that since there may be black borders around the screen displayed on the target device, the target device needs to determine the black border area based on the screen resolution and the target encoding resolution. When receiving the reverse control operation, the target device needs to convert the coordinates relative to the target device screen to the coordinates relative to the screen being projected based on the coordinates of the black border area. This will not be elaborated on in this embodiment.

[0160] The source end performs coordinate transformation on the operation coordinates contained in the instruction by scaling to obtain the operation coordinates within the virtual screen.

[0161] Indicative, such as Figure 9 As shown, when the operation coordinates in the reverse control command are (500, 200), the source end determines the converted coordinates to be (987, 395) based on the scaling ratio of 1.974.

[0162] Step 803: Response to instructions based on the transformed operation coordinates.

[0163] Furthermore, the source device responds to commands based on the converted operation coordinates and transmits the updated application screen's corresponding projection data stream to the destination device in real time. For example, the source device can simulate operations within the application screen of the virtual screen based on the converted operation coordinates and operation type.

[0164] In this embodiment, the source determines the scaling ratio based on the virtual screen resolution and the target encoding resolution, and uses the scaling ratio to perform coordinate transformation on the reverse control commands sent by the destination, that is, to transform the operation on the destination into the operation on the virtual screen, so as to ensure the accuracy of the reverse control operation.

[0165] See Figure 10 , Figure 10 This is a structural block diagram of a screen projection device provided in an exemplary embodiment of this application. The device includes:

[0166] The processing module 1010 is used to determine the target adaptation resolution from at least two adaptation resolutions supported by the application based on the expected resolution of the destination when there is a screen projection requirement. Different adaptation resolutions are used to adapt to screens with different resolutions.

[0167] The processing module 1010 is used to determine the virtual screen resolution of the virtual screen based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution.

[0168] The processing module 1010 is used to determine the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution, wherein the source encoding resolution is related to the encoding capability of the source.

[0169] The processing module 1010 is used to perform video encoding on the application screen using an encoder with the target encoding resolution to obtain a projection data stream;

[0170] The transmission module 1020 is used to transmit the screen projection data stream to the destination terminal so that the destination terminal can perform screen projection display based on the screen projection data stream.

[0171] Optionally, the processing module 1010 includes:

[0172] The adaptation resolution determination unit is used to determine the first aspect ratio of the desired resolution and the second aspect ratio of the adaptation resolution;

[0173] In the case where there is a second aspect ratio that is less than or equal to the first aspect ratio, the second aspect ratio that is closest to the first aspect ratio among the second aspect ratios that are less than or equal to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution;

[0174] If there is no second aspect ratio less than or equal to the first aspect ratio, the second aspect ratio closest to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution.

[0175] Optionally, the adaptation resolution determination unit is used for:

[0176] If the screen state represented by the adapted resolution is inconsistent with the screen state represented by the desired resolution, the adapted resolution is filtered out. The screen state includes landscape mode and portrait mode.

[0177] Optionally, the processing module 1010 includes:

[0178] The virtual screen resolution determination unit is used to determine the first pixel density corresponding to the target adaptation resolution and the second pixel density of the source screen.

[0179] Determine the pixel density ratio of the first pixel density to the second pixel density;

[0180] The target adaptation resolution is scaled based on the pixel density ratio to obtain the virtual screen resolution, where the pixel density of the virtual screen resolution is the second pixel density.

[0181] Optionally, the processing module 1010 includes:

[0182] The encoding resolution determination unit is used to determine the virtual screen resolution as the target encoding resolution when the virtual screen resolution is less than the source encoding resolution and the virtual screen resolution is less than the desired resolution.

[0183] If the virtual screen resolution is greater than the source-end encoding resolution, and / or the virtual screen resolution is greater than the desired resolution, the target encoding resolution is determined based on the source-end encoding resolution and the desired resolution.

[0184] Optionally, an encoding resolution determination unit is used for:

[0185] If the source-end coding resolution is greater than the desired resolution, the target coding resolution is determined based on the desired resolution.

[0186] If the source-end coding resolution is less than the desired resolution, the target coding resolution is determined based on the source-end coding resolution.

[0187] If the source-end coding resolution is partially greater than the desired resolution and partially less than the desired resolution, an intermediate resolution is determined based on the source-end coding resolution and the desired resolution; and the target coding resolution is determined based on the intermediate resolution.

[0188] Optionally, an encoding resolution determination unit is used for:

[0189] If the aspect ratio of the target adapted resolution is less than the aspect ratio of the desired resolution, the height of the desired resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adapted resolution; and the target encoding resolution is determined based on the target width and the target height.

[0190] If the aspect ratio of the target adaptation resolution is greater than the aspect ratio of the desired resolution, the width of the desired resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

[0191] Optionally, an encoding resolution determination unit is used for:

[0192] If the aspect ratio of the target adaptation resolution is less than the aspect ratio of the source coding resolution, the height of the source coding resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adaptation resolution; and the target coding resolution is determined based on the target width and the target height.

[0193] If the aspect ratio of the target adaptation resolution is greater than the aspect ratio of the source encoding resolution, the width of the source encoding resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

[0194] Optionally, an encoding resolution determination unit is used for:

[0195] The smaller of the width of the source-end coding resolution and the width of the desired resolution is determined as the intermediate width, and the smaller of the height of the source-end coding resolution and the height of the desired resolution is determined as the intermediate height.

[0196] The intermediate resolution is determined based on the intermediate width and the intermediate height;

[0197] If the aspect ratio of the target adaptation resolution is less than that of the intermediate resolution, the height of the intermediate resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

[0198] If the aspect ratio of the target adaptation resolution is greater than that of the intermediate resolution, the width of the intermediate resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

[0199] Optionally, the processing module further includes:

[0200] A conversion unit is used to determine a scaling ratio based on the virtual screen resolution and the target encoding resolution;

[0201] Upon receiving a reverse control command sent by the destination, the operation coordinates contained in the reverse control command are transformed based on the scaling ratio;

[0202] Command response is based on the transformed operation coordinates.

[0203] Optionally, the device further includes:

[0204] A configuration module is used to create the virtual screen based on the virtual screen resolution and to set the encoder based on the target encoding resolution;

[0205] The transmission module 1020 is further configured to transmit the target encoding resolution to the destination end so that the destination end can set the decoder based on the target encoding resolution.

[0206] In summary, in this embodiment, before projecting the screen from the source end to the destination end, the source end first determines the target adaptation resolution with the highest matching degree from a variety of adaptation resolutions supported by the application, based on the desired resolution of the destination end. Then, based on the target adaptation resolution, the virtual screen resolution of the virtual screen carrying the application screen is determined, which improves the matching degree between the virtual screen resolution and the desired resolution of the destination end and helps to reduce the black borders that appear when the projected screen is displayed on the destination end. Furthermore, the source end combines the virtual screen resolution, the source end encoding resolution which characterizes the source end's encoding capability, and the desired resolution to determine the target encoding resolution. Based on the target encoding resolution, the application screen carried by the virtual screen is encoded by an encoder, thereby transmitting the encoded projection data stream to the destination end. This ensures that the projection process conforms to the performance of both ends and avoids projection abnormalities caused by exceeding the performance of the source end or the destination end.

[0207] See Figure 11 , Figure 11 This is a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application. This terminal can be implemented as the source terminal in the above embodiments. The terminal can be a smartphone, tablet computer, smart wearable device, etc. The terminal may also include one or more of the following components: a processor 1110 and a memory 1120.

[0208] Optionally, the processor 1110 connects various parts within the electronic device using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1120, and by calling data stored in the memory 1120. Optionally, the processor 1110 can be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1110 can integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and baseband chip. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for the touchscreen display; the NPU is used to implement Artificial Intelligence (AI) functions; and the baseband chip is used for wireless communication. It is understandable that the aforementioned baseband chip may not be integrated into the processor 1110, but may be implemented using a separate chip.

[0209] The memory 1120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1120 may include a non-transitory computer-readable storage medium. The memory 1120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, phone book, etc.).

[0210] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0211] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method described in the above embodiments. Optionally, the computer-readable storage medium may include ROM, RAM, solid-state drives (SSDs), or optical discs, etc. The RAM may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0212] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the screen projection method provided in the various optional implementations of the above aspects.

[0213] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A screen projection method, characterized in that, The method is used at the source end, and the method includes: When there is a need for screen mirroring, the target adapted resolution is determined from at least two adapted resolutions supported by the application based on the desired resolution of the destination device. Different adapted resolutions are used to adapt to screens with different resolutions. The virtual screen resolution is determined based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution. Based on the virtual screen resolution, the source encoding resolution, and the desired resolution, the target encoding resolution is determined, wherein the source encoding resolution is related to the encoding capability of the source. The application screen is video encoded using an encoder with the target encoding resolution to obtain a projection data stream; The projection data stream is transmitted to the destination so that the destination can perform projection display based on the projection data stream.

2. The method according to claim 1, characterized in that, The determination of the target adaptation resolution based on the desired resolution of the destination device, from at least two adaptation resolutions supported by the application, includes: Determine the first aspect ratio of the desired resolution and the second aspect ratio of the adapted resolution; In the case where there is a second aspect ratio that is less than or equal to the first aspect ratio, the second aspect ratio that is closest to the first aspect ratio among the second aspect ratios that are less than or equal to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution; If there is no second aspect ratio less than or equal to the first aspect ratio, the second aspect ratio closest to the first aspect ratio is determined as the target aspect ratio, and the adaptation resolution corresponding to the target aspect ratio is determined as the target adaptation resolution.

3. The method according to claim 2, characterized in that, Before determining the first aspect ratio of the desired resolution and the second aspect ratio of each adapted resolution, the method further includes: If the screen state represented by the adapted resolution is inconsistent with the screen state represented by the desired resolution, the adapted resolution is filtered out. The screen state includes landscape mode and portrait mode.

4. The method according to claim 1, characterized in that, Determining the virtual screen resolution based on the target adaptive resolution includes: Determine the first pixel density corresponding to the target adaptation resolution and the second pixel density of the source screen; Determine the pixel density ratio of the first pixel density to the second pixel density; The target adaptation resolution is scaled based on the pixel density ratio to obtain the virtual screen resolution, where the pixel density of the virtual screen resolution is the second pixel density.

5. The method according to claim 1, characterized in that, Determining the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution includes: If the virtual screen resolution is less than the source encoding resolution and the virtual screen resolution is less than the desired resolution, the virtual screen resolution is determined as the target encoding resolution. If the virtual screen resolution is greater than the source-end encoding resolution, and / or the virtual screen resolution is greater than the desired resolution, the target encoding resolution is determined based on the source-end encoding resolution and the desired resolution.

6. The method according to claim 5, characterized in that, Determining the target coding resolution based on the source coding resolution and the desired resolution includes: If the source-end coding resolution is greater than the desired resolution, the target coding resolution is determined based on the desired resolution. If the source-end coding resolution is less than the desired resolution, the target coding resolution is determined based on the source-end coding resolution. If the source-end coding resolution is partially greater than the desired resolution and partially less than the desired resolution, an intermediate resolution is determined based on the source-end coding resolution and the desired resolution; and the target coding resolution is determined based on the intermediate resolution.

7. The method according to claim 6, characterized in that, Determining the target coding resolution based on the desired resolution includes: If the aspect ratio of the target adapted resolution is less than the aspect ratio of the desired resolution, the height of the desired resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adapted resolution; and the target encoding resolution is determined based on the target width and the target height. If the aspect ratio of the target adaptation resolution is greater than the aspect ratio of the desired resolution, the width of the desired resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

8. The method according to claim 6, characterized in that, Determining the target coding resolution based on the source-end coding resolution includes: If the aspect ratio of the target adaptation resolution is less than the aspect ratio of the source coding resolution, the height of the source coding resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adaptation resolution; and the target coding resolution is determined based on the target width and the target height. If the aspect ratio of the target adaptation resolution is greater than the aspect ratio of the source encoding resolution, the width of the source encoding resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

9. The method according to claim 6, characterized in that, The step of determining the intermediate resolution based on the source-end coding resolution and the desired resolution includes: The smaller of the width of the source-end coding resolution and the width of the desired resolution is determined as the intermediate width, and the smaller of the height of the source-end coding resolution and the height of the desired resolution is determined as the intermediate height. The intermediate resolution is determined based on the intermediate width and the intermediate height; Determining the target coding resolution based on the intermediate resolution includes: If the aspect ratio of the target adaptation resolution is less than that of the intermediate resolution, the height of the intermediate resolution is determined as the target height; the target width is determined based on the target height and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height. If the aspect ratio of the target adaptation resolution is greater than that of the intermediate resolution, the width of the intermediate resolution is determined as the target width; the target height is determined based on the target width and the aspect ratio of the target adaptation resolution; and the target encoding resolution is determined based on the target width and the target height.

10. The method according to claim 1, characterized in that, The method further includes: The scaling ratio is determined based on the virtual screen resolution and the target encoding resolution; Upon receiving a reverse control command sent by the destination, the operation coordinates contained in the reverse control command are transformed based on the scaling ratio; Command response is based on the transformed operation coordinates.

11. The method according to claim 1, characterized in that, Before obtaining the projection data stream by encoding the application screen using an encoder with the target encoding resolution, the method further includes: The virtual screen is created based on the virtual screen resolution, and the encoder is set based on the target encoding resolution; The target encoding resolution is transmitted to the destination so that the destination can set up a decoder based on the target encoding resolution.

12. A screen projection device, characterized in that, The device includes: The processing module is used to determine the target adaptation resolution from at least two adaptation resolutions supported by the application based on the desired resolution of the destination device when there is a screen casting requirement. Different adaptation resolutions are used to adapt to screens with different resolutions. The processing module is used to determine the virtual screen resolution of the virtual screen based on the target adaptation resolution. The virtual screen is used to carry the application screen of the application, and the aspect ratio of the virtual screen resolution is consistent with the aspect ratio of the target adaptation resolution. The processing module is used to determine the target encoding resolution based on the virtual screen resolution, the source encoding resolution, and the desired resolution, wherein the source encoding resolution is related to the encoding capability of the source. The processing module is used to encode the application screen using an encoder with the target encoding resolution to obtain a projection data stream; The transmission module is used to transmit the projection data stream to the destination terminal so that the destination terminal can perform projection display based on the projection data stream.

13. A terminal, characterized in that, The terminal includes a processor and a memory; the memory stores at least one instruction, which is executed by the processor to implement the screen projection method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is executed by a processor to implement the screen projection method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the screen projection method as described in any one of claims 1 to 11.

Images