Screen projection method and device, terminal equipment and computer readable storage medium

Abstract

Embodiments of the present application disclose a screen projection method and device, a terminal device and a computer readable storage medium. The method is applied to a first terminal device, and the method comprises: obtaining a screen projection parameter corresponding to a second terminal device, wherein the screen projection parameter comprises a first resolution corresponding to a screen projection window of the second terminal device and a coding and decoding capability corresponding to the second terminal device; determining a target resolution which is proportional to the first resolution according to the screen projection parameter; and sending video stream data matched with the target resolution to the second terminal device, wherein the video stream data is used to play a screen projection picture in the screen projection window of the second terminal device. The screen projection method, device, terminal device and computer readable storage medium can ensure that the screen projection picture will not appear black edges or picture stretching and distortion when playing in the screen projection window, and improve the screen projection effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a screen projection method and device, terminal equipment and computer readable storage medium. BACKGROUND

[0002] In the current screen projection technology, the source terminal device directly captures the image frame displayed on the screen and then directly sends it to the opposite terminal device for screen projection display. When the opposite terminal device displays the screen projection frame sent by the source terminal device, black edges or frame stretching and distortion problems may occur, affecting the screen projection effect. SUMMARY

[0003] The embodiments of the present application disclose a screen projection method, device, terminal equipment and computer readable storage medium, which can ensure that the screen projection frame does not have black edges or frame stretching and distortion problems when playing in the screen projection window, and improves the screen projection effect.

[0004] The embodiments of the present application disclose a screen projection method applied to a first terminal device, and the method comprises the following steps:

[0005] Obtaining screen projection parameters corresponding to a second terminal device, wherein the screen projection parameters comprise a first resolution corresponding to a screen projection window of the second terminal device and a coding and decoding capability corresponding to the second terminal device;

[0006] Determining a target resolution equal in proportion to the first resolution according to the screen projection parameters;

[0007] Sending video stream data matched with the target resolution to the second terminal device, wherein the video stream data is used to play a screen projection frame in the screen projection window of the second terminal device.

[0008] The embodiments of the present application disclose a screen projection method applied to a second terminal device, and the method comprises the following steps:

[0009] Receiving video stream data matched with a target resolution sent by a first terminal device, wherein the target resolution is determined by the first terminal device according to screen projection parameters corresponding to the second terminal device; the screen projection parameters comprise a first resolution corresponding to a screen projection window of the second terminal device and a coding and decoding capability corresponding to the second terminal device, and the target resolution is equal in proportion to the first resolution;

[0010] Playing a screen projection frame in the screen projection window according to the video stream data.

[0011] The embodiments of the present application disclose a screen projection device applied to a first terminal device, and the device comprises the following steps:

[0012] The parameter obtaining module is configured to obtain a screen projection parameter corresponding to the second terminal device, the screen projection parameter comprising a first resolution corresponding to a screen projection window of the second terminal device and a codec capability;

[0013] The resolution determining module is configured to determine a target resolution that is proportional to the first resolution according to the screen projection parameter.

[0014] The data sending module is configured to send video stream data matching the target resolution to the second terminal device, the video stream data being used to play a screen projection picture in the screen projection window of the second terminal device.

[0015] Embodiments of the present application disclose a screen projection device applied to a second terminal device, the device comprising:

[0016] The data receiving module is configured to receive video stream data matching a target resolution sent by a first terminal device, the target resolution being determined by the first terminal device according to a screen projection parameter corresponding to the second terminal device; the screen projection parameter comprising a first resolution corresponding to a screen projection window of the second terminal device and a codec capability, the target resolution being proportional to the first resolution.

[0017] The playing module is configured to play a screen projection picture in the screen projection window according to the video stream data.

[0018] Embodiments of the present application disclose a terminal device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor implement the method as described above.

[0019] Embodiments of the present application disclose a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method as described above.

[0020] The screen projection method and device, the terminal device and the computer readable storage medium disclosed by the embodiments of the present application are as follows: a first terminal device acquires a screen projection parameter corresponding to a second terminal device, the screen projection parameter including a first resolution corresponding to a screen projection window of the second terminal device and a coding and decoding capability corresponding to the second terminal device, a target resolution equal in proportion to the first resolution is determined according to the screen projection parameter, and video stream data matched with the target resolution is sent to the second terminal device, so that the second terminal device plays a screen projection picture in the screen projection window according to the video stream data. The first terminal device (i.e. the source terminal device in screen projection) can adaptively adjust the resolution of the transmitted video stream data according to the first resolution corresponding to the screen projection window of the second terminal device (i.e. the opposite terminal device in screen projection) and the coding and decoding capability corresponding to the second terminal device, which can not only ensure that the second terminal device normally decodes the video stream data, but also ensure that the screen projection picture does not have the problems of black edges or picture stretching and distortion when played in the screen projection window, thereby improving the screen projection effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1A A display diagram of a screen projection picture in the related art;

[0023] Figure 1B An application scenario diagram of a screen projection method in an embodiment;

[0024] Figure 2 A flowchart of a screen projection method in an embodiment;

[0025] Figure 3 A flowchart of a screen projection method in another embodiment;

[0026] Figure 4 A diagram for determining a target resolution in an embodiment;

[0027] Figure 5 A flowchart of a screen projection method in another embodiment;

[0028] Figure 6 A diagram for background screen projection of multiple screen projection windows in an embodiment;

[0029] Figure 7 A timing diagram of a screen projection method in an embodiment;

[0030] Figure 8Flow chart of the screen projection method in another embodiment;

[0031] Figure 9 Block diagram of the screen projection device in an embodiment;

[0032] Figure 10 Block diagram of the screen projection device in another embodiment;

[0033] Figure 11 Structural block diagram of the terminal device in an embodiment. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0035] It should be noted that the terms “include” and “have” and any variations thereof in the embodiments of the present application and the drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed or optionally further includes other steps or units inherent to the process, method, product or device.

[0036] It can be understood that the terms “first”, “second” and the like used in the present application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the present application, the first terminal device can be referred to as the second terminal device, and similarly, the second terminal device can be referred to as the first terminal device. The first terminal device and the second terminal device are both terminal devices, but they are not the same terminal device. The term “a plurality of” used in the present application means two or more. The term “and / or” used in the present application means one of the options, or any combination of a plurality of options.

[0037] In the current screen projection technology, the source device directly captures the image frame displayed on the screen, and then directly sends it to the opposite device for screen projection display. The resolution of the screen projection frame displayed by the opposite device is based on the resolution of the image frame captured by the source device, so it is easy to have black edges or distorted screen, which affects the screen projection effect. For example, as shown in FIG. 1, the screen projection frame displayed by the opposite device is not the same as the image frame captured by the source device, which causes the screen projection frame to have black edges and be distorted. Figure 1AAs shown in the figure, the source terminal device is a mobile phone, and the opposite terminal device is a computer. When the mobile phone performs screen projection to the computer, because the resolution of the mobile phone is different from the resolution of the computer, directly displaying the screen projection picture in the resolution of the mobile phone will cause the screen projection picture projected to the computer to have a black border when displayed.

[0038] The embodiment of the present application provides a screen projection method and device, a terminal device and a computer readable storage medium, which can ensure that the screen projection picture does not have a black border or picture stretching and distortion when played in a screen projection window, and improve the screen projection effect.

[0039] Figure 1B The figure shows the application scenario of the screen projection method in one embodiment. As shown in the figure, Figure 1B The first terminal device 110 and the second terminal device 120 can establish a communication connection. The first terminal device 110 can be used as a source terminal device in a screen projection scenario, and the second terminal device 120 can be used as an opposite terminal device in the screen projection scenario. The source terminal device can transmit multimedia data to the opposite terminal device to perform screen projection in the opposite terminal device. The first terminal device 110 and the second terminal device 120 can include but are not limited to a mobile phone, a wearable device, a PC (Personal Computer), a tablet computer, a notebook computer, a vehicle-mounted terminal and the like. The communication connection between the first terminal device 110 and the second terminal device 120 can include a wireless communication connection such as Bluetooth and Wi-Fi, and can also include a wired communication connection realized through a USB (Universal Serial Bus) port, and the like. The embodiment of the present application does not limit this.

[0040] The first terminal device 110 can obtain the screen projection parameter corresponding to the second terminal device 120. The screen projection parameter can include a first resolution corresponding to a screen projection window of the second terminal device 120 and a coding and decoding capability of the second terminal device 120. The first terminal device 110 can determine a target resolution that is proportional to the first resolution according to the screen projection parameter corresponding to the second terminal device 120, and send video stream data matched with the target resolution to the second terminal device 120. The second terminal device 120 receives the video stream data matched with the target resolution sent by the first terminal device 110, and can play a screen projection picture in the screen projection window according to the video stream data. Because the resolution of the video stream data is the target resolution, which is proportional to the first resolution of the screen projection window, when the screen projection window displays the screen projection picture, there is no problem of black border or picture stretching and distortion, and the screen projection effect is improved.

[0041] It should be noted that the screen projection method provided by the embodiments of the present application can also be applied to a screen projection scenario in which one first terminal device 110 projects to multiple second terminal devices 120, or a screen projection scenario in which multiple first terminal devices 110 project to one second terminal device 120. The terminal device can serve as the first terminal device (i.e., the source device) or the second terminal device (i.e., the target device), and the embodiments of the present application do not limit the specific screen projection scenario.

[0042] As shown in FIG. 1, in one embodiment, a screen projection method is provided, which can be applied to the first terminal device, and can include the following steps: Figure 2

[0043] In step 210, the screen projection parameters corresponding to the second terminal device are acquired, and the screen projection parameters include the first resolution corresponding to the screen projection window of the second terminal device and the coding and decoding capability corresponding to the second terminal device.

[0044] The first terminal device and the second terminal device can first establish a communication connection, which can include but is not limited to a wireless communication connection or a wired communication connection. Alternatively, a screen projection link between the first terminal device and the second terminal device can be established on the basis of the communication connection, and the screen projection link can be used to transmit multimedia data in the screen projection process and information related to screen projection, etc. The multimedia data can include video stream data and audio stream data, etc., and the information related to screen projection can include one or more of the resolution information corresponding to the video stream data, the volume of the audio stream data, the sound channel, etc.

[0045] In some embodiments, the user can initiate a request for establishing the screen projection link on the first terminal device or the second terminal device. After the first terminal device and the second terminal device establish the communication connection, a screen projection control can be displayed on the first terminal device and / or the second terminal device. The user can trigger the screen projection control. For example, the user triggers the screen projection control displayed on the first terminal device, and the first terminal device responds to the triggering operation of the screen projection control and sends an establishment request for establishing the screen projection link to the second terminal device. The second terminal device can respond to the establishment request and establish the screen projection link with the first terminal device. For another example, the user triggers the screen projection control displayed on the second terminal device, and the second terminal device responds to the triggering operation of the screen projection control and sends an establishment request for establishing the screen projection link to the first terminal device. The first terminal device can respond to the establishment request and establish the screen projection link with the second terminal device.

[0046] ​Further, the user can select the screen projection relationship between the first terminal device and the second terminal device when triggering the screen projection control, such as selecting the screen projection relationship of the first terminal device projecting to the second terminal device. Alternatively, the screen projection relationship between the first terminal device and the second terminal device can also be a default setting, such as the screen projection relationship of a computer projecting to a television, the screen projection relationship of a mobile phone projecting to a computer, and the like, but is not limited thereto. The user can initiate the establishment of the screen projection link at any terminal device, and can determine the screen projection relationship according to actual needs, thereby meeting different needs of the user.

[0047] After the screen projection link is established between the first terminal device and the second terminal device, the first terminal device can obtain the screen projection parameter corresponding to the second terminal device, which can include the first resolution corresponding to the screen projection window of the second terminal device and the coding and decoding capability corresponding to the second terminal device. The second terminal device can be provided with a screen projection window, which is used to display the screen projection picture of the first terminal device. Alternatively, the screen projection window can be a full-screen window, so that the first terminal device can play the screen projection picture in full screen through the screen projection window. The screen projection window can also be a window of a custom size, such as a window occupying one half of the screen or a window occupying three fourths of the screen. The first resolution corresponding to the screen projection window can refer to the number of pixels that can be displayed in the screen projection window, which can be represented by the horizontal pixel number * the vertical pixel number. The coding and decoding capability corresponding to the second terminal device can include one or more parameters such as the video format supported by the second terminal device for coding and decoding, the maximum video width and height parameter supported by the second terminal device for coding and decoding, the frame rate, the code rate, and the like.

[0048] In step 220, a target resolution equal in proportion to the first resolution is determined according to the screen projection parameter.

[0049] The first terminal device can determine the target resolution equal in proportion to the first resolution according to the screen projection parameter corresponding to the second terminal device. As an implementation manner, the coding and decoding capability corresponding to the second terminal device can affect the maximum resolution supported by the second terminal device for coding and decoding, so that the resolution range supported by the second terminal device for coding and decoding can be determined according to the coding and decoding capability corresponding to the second terminal device, and the target resolution equal in proportion to the first resolution can be determined within the resolution range. The target resolution is equal in proportion to the first resolution, and the ratio of the horizontal pixel number to the vertical pixel number of the target resolution is the same as the ratio of the horizontal pixel number to the vertical pixel number of the first resolution, such as the first resolution being 1000*2000 and the target resolution being 500*1000. The target resolution can be less than, greater than, or equal to the first resolution, which is not limited herein. Since the target resolution is determined within the resolution range supported by the second terminal device for coding and decoding, it can be ensured that the video stream data transmitted by the first terminal device to the second terminal device can be normally coded and decoded.

[0050] Step 230: Send video stream data that matches the target resolution to the second terminal device, where the video stream data is used to play the projection screen in the projection window of the second terminal device.

[0051] After the first terminal device determines the target resolution, it can generate video stream data that matches the target resolution, that is, the resolution of the video stream data is the target resolution, and transmit the generated video stream data to the second terminal device. After the second terminal device receives the video stream data sent by the first terminal device, it can play the projected screen in the set projection window according to the video stream data. Since the resolution of the video stream data is the target resolution, which is proportional to the first resolution corresponding to the projection window, the projection screen can fill the projection window for display without causing black edges or problems of stretched or distorted images. For example, if the projection window is a full-screen window, the second terminal device can display the projection screen in full screen without causing black edges or problems of stretched or distorted images.

[0052] Furthermore, if the target resolution is the same as the first resolution, the second terminal device can directly decode the video stream data to generate a projection screen and display it in the projection window; if the target resolution is different from the first resolution, since the target resolution is proportional to the first resolution corresponding to the projection window, the video stream data can be decoded and then scaled to obtain a projection screen with the first resolution and displayed in the projection window. For example, if the first resolution is 1000*2000 and the target resolution is 500*1000, after decoding the video stream data, the decoded data is magnified by 2 times to obtain a projection screen with the first resolution. Since it is a proportional scaling, the projection screen will not be stretched or distorted, and the projection screen obtained after scaling is consistent with the resolution of the projection window, and can fill the projection window for display without black edges in the projection window.

[0053] In an embodiment of the present application, the first terminal device (i.e., the source device in the screen projection) can adaptively adjust the resolution of the transmitted video stream data according to the first resolution corresponding to the screen projection window of the second terminal device (i.e., the counterpart device in the screen projection) and the encoding and decoding capability corresponding to the second terminal device. This can not only ensure that the second terminal device decodes the video stream data normally, but also ensure that the projected screen will not have black edges or stretched or distorted images when played in the screen projection window, thereby improving the screen projection effect.

[0054] like Figure 3 As shown, in another embodiment, a screen projection method is provided, which can be applied to the first terminal device mentioned above, and the method may include the following steps:

[0055] At step 302, the second terminal device corresponding to the screen projection parameter is obtained, and the screen projection parameter includes the first resolution corresponding to the screen projection window of the second terminal device and the coding and decoding capability corresponding to the second terminal device.

[0056] After the first terminal device and the second terminal device establish the screen projection link, the first terminal device can obtain the screen projection parameter corresponding to the second terminal device, and the manner of obtaining the screen projection parameter corresponding to the second terminal device can include but is not limited to any one of the following manners:

[0057] Manner one, the first terminal device receives the screen projection parameter sent by the second terminal device. After the first terminal device and the second terminal device establish the screen projection link, the second terminal device can directly send the screen projection parameter to the first terminal device. Alternatively, the second terminal device can send the first resolution corresponding to the screen projection window and the coding and decoding capability corresponding to the second terminal device to the first terminal device at the same time, or send them respectively. For example, the second terminal device can first send the coding and decoding capability corresponding to the second terminal device to the first terminal device, the second terminal device can set the screen projection window according to the actual screen projection demand, and after determining the first resolution corresponding to the screen projection window, the first resolution corresponding to the screen projection window is sent to the second terminal device.

[0058] Manner two, the first terminal device receives the device information sent by the second terminal device, and queries the screen projection parameter corresponding to the device information. After the first terminal device and the second terminal device establish the screen projection link, the second terminal device can send the device information to the first terminal device, and the device information can include but is not limited to one or more of the device identification, device model, factory batch, manufacturer information, etc. of the second terminal device.

[0059] As an implementation manner, the first terminal device can store a white list, and the white list can include the device information and the corresponding screen projection parameter of each terminal device capable of screen projection. After the first terminal device receives the device information sent by the second terminal device, the screen projection parameter corresponding to the device information of the second terminal device can be found in the white list. As another implementation manner, the first terminal device can upload the device information sent by the second terminal device to a server, find the screen projection parameter corresponding to the device information of the second terminal device through the server, and receive the screen projection parameter corresponding to the second terminal device returned by the server.

[0060] Alternatively, since the first resolution corresponding to the screen projection window in the second terminal device can be set according to the actual demand, rather than fixed, the first terminal device can only find the corresponding coding and decoding capability according to the device information of the terminal device, and after the second terminal device sets the screen projection window according to the actual screen projection demand and determines the first resolution corresponding to the screen projection window, the first resolution corresponding to the screen projection window is sent to the second terminal device.

[0061] Alternatively, the first terminal device can find the corresponding codec capability according to the device information of the terminal device, and the first resolution corresponding to the full-screen window, which can be understood as the screen display resolution of the first terminal device. If the first terminal device does not re-receive the first resolution corresponding to the screen projection window sent by the second terminal device before sending the video stream data to the second terminal device, it indicates that the second terminal device does not re-set a new screen projection window, and the full-screen window can be directly determined as the screen projection window of the second terminal device, and the target resolution proportional to the first resolution corresponding to the full-screen window is determined.

[0062] In some embodiments, the screen projection link established between the first terminal device and the second terminal device can include a first transmission channel and a second transmission channel. The first transmission channel can be a bidirectional transmission channel, that is, through the first transmission channel, the first terminal device and the second terminal device can transmit data to each other. The second transmission channel can be a unidirectional transmission channel, and the second terminal device can only receive the data sent by the first terminal device through the second transmission channel, and cannot send data to the first terminal device through the second transmission channel.

[0063] The first terminal device can receive the screen projection parameters sent by the second terminal device through the first transmission channel established between the first terminal device and the second terminal device. The first transmission channel can be used for transmission of various parameters between the first terminal device and the second terminal device, such as codec capability negotiation. The first terminal device and the second terminal device can send their respective codec capabilities to each other through the first transmission channel. Alternatively, the second terminal device sends the first resolution of the screen projection window to the first terminal device through the first transmission channel, and the first terminal device sends the resolution information (i.e. target resolution) of the video stream data to the second terminal device through the second transmission channel.

[0064] The first terminal device can send the video stream data matched with the target resolution to the second terminal device through the second transmission channel established between the first terminal device and the second terminal device. The second transmission channel can be used for transmission of multimedia data (including video stream data and audio stream data, etc.).

[0065] As a specific embodiment, the screen projection link between the first terminal device and the second terminal device can include a P2P (Peer to Peer) transmission channel, which can include an RTSP (Real Time Streaming Protocol) socket channel and an RTP (Real-time Transport Protocol) socket channel. The RTSP socket channel (i.e., the first transmission channel described above) can be used as a transmission channel for various parameters, and the RTP socket channel (i.e., the second transmission channel described above) can be used as a transmission channel for multimedia data. By using the RTCP socket channel and the RTP socket channel together, the transmission efficiency can be optimized with effective feedback and minimal overhead, and the data transmission efficiency during screen projection can be improved.

[0066] In the embodiments of the present application, the first terminal device can accurately obtain the screen projection parameters corresponding to the second terminal device, so that the resolution of the video stream data sent subsequently can match the coding and decoding capability of the second terminal device and the first resolution of the screen projection window, and the accuracy of screen projection can be improved.

[0067] In step 304, the coding and decoding parameters are determined according to the coding and decoding capability of the second terminal device and the coding and decoding capability of the first terminal device.

[0068] The first terminal device can perform coding and decoding capability negotiation with the second terminal device to determine the coding and decoding parameters that can be simultaneously supported by the first terminal device and the second terminal device. The coding and decoding parameters can include, but are not limited to, one or more of a video format, a video width and height parameter, a frame rate, a code rate, etc. The frame rate refers to the number of image frames transmitted per unit time (e.g., 1 second), and the code rate refers to the number of data bits transmitted per unit time. The first terminal device and the second terminal device can perform coding and decoding processing and data transmission during screen projection based on the coding and decoding parameters.

[0069] In some embodiments, the first terminal device can compare its codec capability with the codec capability corresponding to the second terminal device, and determine the codec parameter according to the comparison result, so as to ensure that the determined codec parameter can be supported by both the first terminal device and the second terminal device. The intersection range of the codec capability of the first terminal device and the codec capability corresponding to the second terminal device can be selected, and the codec parameter can be determined from the intersection range. Taking the frame rate as an example, the frame rate range supported by the first terminal device can be compared with the frame rate range supported by the second terminal device, the intersection range of the two frame rate ranges is obtained, and the target frame rate in the final screen projection process is selected from the intersection range. Further, the maximum parameter value can be determined as the codec parameter from the intersection range of the codec capability of the first terminal device and the codec capability corresponding to the second terminal device. For example, the maximum frame rate can be selected as the target frame rate from the intersection range of the two frame rate ranges. By determining the maximum parameter value as the codec parameter from the intersection range of the codec capability of the first terminal device and the codec capability corresponding to the second terminal device, it can not only ensure normal data transmission and codec processing in the screen projection process, but also improve the performance and efficiency of data transmission and codec processing as much as possible.

[0070] As a specific implementation, the first terminal device can compare the first maximum video width-height parameter supported by the first terminal device with the second maximum video width-height parameter supported by the second terminal device, and select the smaller maximum video width-height parameter as the target video width-height parameter, which is the maximum video width-height parameter that can be supported by both the first terminal device and the second terminal device for codec. For example, the first maximum video width-height parameter supported by the first terminal device is 320*210, and the second maximum video width-height parameter supported by the second terminal device is 280*160, and the target video width-height parameter can be 280*160. The target resolution corresponding to the first resolution of the screen projection window in the second terminal device can be determined within the resolution range corresponding to the target video width-height parameter, so that the first terminal device and the second terminal device can normally perform codec.

[0071] Step 306, determining a target resolution corresponding to the first resolution in proportion according to the codec parameter and the first resolution corresponding to the screen projection window of the second terminal device.

[0072] After determining the target video width-height parameter, as an implementation, the first terminal device can determine the maximum resolution corresponding to the first resolution of the screen projection window in proportion within the resolution range corresponding to the target video width-height parameter, and take the maximum resolution as the target resolution. For example, Figure 4 A schematic diagram for determining the target resolution in an embodiment. As shown in Figure 4As shown, the first terminal device supports a first maximum video width-height parameter of 320*210, the second terminal device supports a second maximum video width-height parameter of 280*160, and the smaller maximum video width-height parameter is 280*160. Therefore, the target video width-height parameter can be determined as 280*160, the corresponding resolution range can be less than or equal to 280*160, the first resolution corresponding to the screen projection window of the second terminal device is 100*100, the maximum resolution equal to 100*100 in the resolution range less than or equal to 280*160 is 160*160, and the target resolution can be determined as 160*160. In the resolution range corresponding to the target video width-height parameter, the maximum resolution equal to the first resolution is used as the target resolution, the screen projection picture can be displayed in the optimal quality under the condition that both the terminal devices can be normally encoded and decoded, and the display effect of the screen projection picture is further improved.

[0073] As another implementation, the target resolution equal to the first resolution can be determined according to the network transmission parameter and / or the performance parameter of the second terminal device in the resolution range corresponding to the target video width-height parameter.

[0074] The network transmission parameter can be used to represent the network condition between the first terminal device and the second terminal device, and the network transmission parameter can include, but is not limited to, one or more of the transmission rate, the transmission delay, the transmission packet loss rate, etc. In the case of determining the target resolution according to the network transmission parameter only, the better the network condition represented by the network transmission parameter, the larger the target resolution determined. In the case of the network condition represented by the network transmission parameter being poor, a smaller target resolution can be determined, so that the fluency and accuracy of the video stream transmission can be improved, and the packet retransmission or the lag can be avoided.

[0075] The performance parameter of the second terminal device can be used to represent the performance of the second terminal device in processing data, and the performance parameter can include, but is not limited to, one or more of the memory parameter, the CPU (central processing unit) parameter, the GPU (graphics processing unit) parameter, the heat parameter, etc. of the second terminal device. In the case of determining the target resolution according to the performance parameter of the second terminal device only, the better the performance represented by the performance parameter, the larger the target resolution determined. In the case of the performance represented by the performance parameter being poor, a smaller target resolution can be determined, so that the processing burden of the second terminal device can be reduced, and the screen projection picture can be normally and fluently displayed.

[0076] Optionally, the target resolution can be determined based on both the network transmission parameters and the performance parameters of the second terminal device. Different weight values ​​can be assigned to the network transmission parameters and the performance parameters of the second terminal device, and weighted and calculated based on the weight values ​​corresponding to the network transmission parameters and the performance parameters of the second terminal device, respectively, to determine the final influencing parameter, and then determine the target resolution based on the influencing parameter. For example, if the influencing parameter is larger, it means that the network condition is better and the performance of the second terminal device is better. The target resolution can be the product of the maximum resolution proportional to the first resolution corresponding to the projection window within the resolution range corresponding to the width and height parameters of the target video and the influencing parameter. The appropriate target resolution can be selected based on the network transmission conditions and the performance of the second terminal device, thereby improving the accuracy of video stream data transmission, further improving the display effect of the projection screen, and avoiding freezes and other situations.

[0077] It should be noted that the method of determining the target resolution is not limited to the above-mentioned methods, and other methods can also be used. For example, within the resolution range corresponding to the width and height parameters of the target video, various resolutions that are proportional to the first resolution corresponding to the projection window can be determined, and a resolution can be randomly selected as the target resolution.

[0078] Step 308: Send video stream data that matches the target resolution to the second terminal device. The video stream data is used to play the projection screen in the projection window of the second terminal device.

[0079] The description of step 308 may refer to the relevant description of step 230 in the above embodiment, and will not be repeated here.

[0080] In an embodiment of the present application, the first terminal device can combine its own encoding and decoding capabilities and the corresponding encoding and decoding capabilities of the second terminal device to determine the resolution range that both can support, and determine a target resolution in this resolution range that is proportional to the first resolution corresponding to the projection window of the second terminal device, to ensure that the first terminal device and the second terminal device can both perform encoding and decoding processing normally during the projection process, and to ensure that the projection screen will not have black edges or stretched and distorted images when played in the projection window, thereby improving the projection effect.

[0081] like Figure 5 As shown, in another embodiment, a screen projection method is provided, which can be applied to the first terminal device mentioned above, and the method may include the following steps:

[0082] Step 502: Obtain the screen projection parameters corresponding to the second terminal device, where the screen projection parameters include the first resolution corresponding to the screen projection window of the second terminal device and the encoding and decoding capabilities corresponding to the second terminal device.

[0083] At step 504, the coding parameter is determined according to the coding and decoding capability corresponding to the second terminal device and the coding and decoding capability corresponding to the first terminal device.

[0084] At step 506, a target resolution equal to the first resolution is determined according to the coding parameter and the first resolution corresponding to the screen projection window of the second terminal device.

[0085] The description of steps 502-506 can refer to the related description in the above embodiments, which will not be repeated here.

[0086] At step 508, a virtual screen is created according to the target resolution.

[0087] In the embodiments of the present application, the screen projection between the first terminal device and the second terminal device can be background screen projection, which means that the target application program in the first terminal device for screen projection can run in the background and project the corresponding interface content to the second terminal device without occupying the display screen of the first terminal device. In some embodiments, after the first terminal device and the second terminal device establish a screen projection link, the second terminal device can send a background screen projection request to the first terminal device, which can carry the first resolution corresponding to the screen projection window. After receiving the background screen projection request sent by the first terminal device, the second terminal device can determine a target resolution equal to the first resolution according to the previously obtained coding and decoding capability corresponding to the second terminal device, the coding and decoding capability of itself and the first resolution corresponding to the screen projection window.

[0088] The first terminal device can create a virtual screen according to the determined target resolution. The virtual screen is different from the physical screen, and the interface displayed on the virtual screen is invisible to the user, which can be understood as that the virtual screen is displayed in the background of the system. The virtual screen with the target resolution can be created by writing the target resolution in a virtual screen creation interface provided by the system, so that the resolution of the interface content displayed in the virtual screen is the target resolution.

[0089] At step 510, the interface content of the target application program is output to the virtual screen.

[0090] The target application refers to an application that needs to be projected to the second terminal device. Optionally, the user can select the target application that needs to be projected on the first terminal device or the second terminal device. For example, the user selects the target application that needs to be projected on the first terminal device. The first terminal device can display an application list containing various applications currently running, and the user can select the target application in the application list. The first terminal device can determine the target application selected by the user in response to the selection operation of the user on the application list. The interface of each application running on the first terminal device can also be provided with a projection control. The user can trigger the projection control to make the application project. If any application running on the first terminal device detects a triggering operation on the projection control, the application can be determined as the target application.

[0091] For another example, the user selects the target application that needs to be projected on the second terminal device. The first terminal device can send an application list to the second terminal device. The application list can include various applications currently running on the first terminal device. The user can select the target application in the application list. The second terminal device can determine the target application selected by the user in response to the selection operation of the user on the application list, and send application information of the target application to the first terminal device. The application information can include, but is not limited to, the application name, the package name, and other application identifiers of the target application. The first terminal device determines the target application that needs to be projected according to the application information sent by the second terminal device. The user can select the target application that needs to be projected on the first terminal device or the second terminal device, which is more flexible and meets different needs of the user.

[0092] The first terminal device can output the interface content of the target application to the virtual screen for display, instead of displaying it on the physical screen. Since the resolution of the virtual screen is the target resolution, the interface content of the target application can be output to the virtual screen according to the target resolution, so that the resolution of the interface content of the target application displayed on the virtual screen is the target resolution. Optionally, if the target application runs in the foreground, the target application can be switched to run in the background, and the interface content of the target application can be output to the virtual screen. This can avoid the target application for projection occupying the physical screen and affecting the user's normal use of other applications during the projection process, thereby improving the user experience.

[0093] In step 512, the interface content of the target application is grabbed from the virtual screen as projection data. Video stream data matching the target resolution is generated according to the projection data, and the video stream data is sent to the second terminal device. The video stream data is used to play the projection picture in the projection window of the second terminal device.

[0094] In some embodiments, the first terminal device can capture the interface content of the target application from the virtual screen as the screen projection data at a first time interval, and the first time interval can be determined according to a frame rate (which can be determined according to the coding and decoding capability of the second terminal device and the coding and decoding capability of the first terminal device), and the first time interval can be a ratio of a unit time to the frame rate.

[0095] In some embodiments, the first terminal device can also identify the interface content displayed on the virtual screen and capture a key frame of the interface content as the screen projection data. Alternatively, the key frame can be interface content with a content difference from a previous frame of interface content greater than a difference threshold. The content difference between each frame of interface content and the previous frame of interface content can be determined, and the content difference can be represented by the number of pixels that are different, or the content difference can be represented by the content displacement from the previous frame of interface content, which is not limited in the embodiments of the present application. It can be determined whether the content difference between the current frame of interface content and the previous frame of interface content is greater than the difference threshold. If it is greater, the current frame of interface content can be determined as the key frame, and the current frame of interface content can be captured as the screen projection data. By capturing the key frame of the target application for screen projection, the smoothness of the screen projection can be improved, the accuracy of the screen projection can be improved, and the screen projection effect is further improved.

[0096] As an implementation, after determining the target resolution, the first terminal device can create an encoder according to the target resolution, and the encoder can be used to encode video data with the target resolution. The first terminal device can also send the target resolution to the second terminal device, so that the second terminal device creates a decoder according to the target resolution, and the decoder can be used to decode video data with the target resolution. Further, the first terminal device can send the target resolution to the second terminal device through the first transmission channel.

[0097] After the first terminal device captures the interface content of the target application from the virtual screen as the screen projection data, the first terminal device can encode the screen projection data through the encoder to obtain video stream data matching the target resolution, and then send the video stream data matching the target resolution to the second terminal device. After the second terminal device receives the video stream data matching the target resolution, the second terminal device can decode the video stream data through the decoder to obtain the screen projection data, and perform scaling processing on the screen projection data according to the first resolution and the target resolution, so as to play the screen projection picture in the screen projection window of the second terminal device.

[0098] If the first resolution is less than the target resolution, the second terminal device can perform a scaling-down processing on the screen projection data according to a ratio of the first resolution to the target resolution, to play a screen projection picture with the first resolution in the screen projection window of the second terminal device. For example, the first resolution is 1000*2000, and the target resolution is 1500*3000. The ratio of the first resolution to the target resolution is 1:1.5. Therefore, the screen projection data can be scaled down by 1 / 3. Similarly, if the first resolution is greater than the target resolution, the second terminal device can perform a scaling-up processing on the screen projection data according to the ratio of the first resolution to the target resolution, to play a screen projection picture with the first resolution in the screen projection window of the second terminal device. For example, the first resolution is 1000*2000, and the target resolution is 500*1000. The ratio of the first resolution to the target resolution is 2:1. Therefore, the screen projection data can be scaled up by 2 times. The second terminal device can obtain a screen projection picture with the first resolution through simple scaling processing, so that the screen projection picture displayed in the screen projection window will not have black edges or picture distortion.

[0099] In some embodiments, a plurality of screen projection windows can be set in the second terminal device to play screen projection pictures. The first resolutions corresponding to the screen projection windows can be the same or different. The background screen projection request sent by the second terminal device to the first terminal device can carry the first resolutions corresponding to the screen projection windows. After receiving the background screen projection request, the first terminal device can determine target resolutions proportional to the first resolutions corresponding to the screen projection windows, according to the coding and decoding capabilities of the second terminal device and the first resolutions corresponding to the screen projection windows. It should be noted that the determination of the target resolutions proportional to the first resolutions corresponding to each screen projection window can refer to the related descriptions in the above embodiments, which will not be repeated here.

[0100] After determining the target resolutions proportional to the first resolutions corresponding to the screen projection windows, the first terminal device can create virtual screens corresponding to the screen projection windows according to the target resolutions proportional to the first resolutions. The number of the virtual screens created by the first terminal device is consistent with the number of the screen projection windows set by the second terminal device, and one-to-one correspondence exists between the virtual screens and the screen projection windows. The user can select target application programs corresponding to the screen projection windows according to actual needs. The screen projection windows can be used to play screen projection pictures of the target application programs. The first terminal device can output interface contents of the target application programs corresponding to the screen projection windows to the virtual screens corresponding to the screen projection windows, respectively, to generate video stream data corresponding to the screen projection windows.

[0101] As a specific embodiment, taking the first projection window as an example, the first projection window can be any one of a plurality of projection windows set in the second terminal device, the first terminal device can determine a first target resolution which is proportional to a first resolution corresponding to the first projection window, and create a first virtual screen according to the first target resolution, and can also create a first encoder according to the first target resolution, and the second terminal device can create a first decoder according to the first target resolution. The first terminal device can output the interface content of the first target application corresponding to the first projection window to the first virtual screen, and capture the interface content of the first target application from the first virtual screen as projection data corresponding to the first projection window, and encode the projection data corresponding to the first projection window through the first encoder to obtain video stream data matched with the first target resolution. After receiving the video stream data matched with the first target resolution, the second terminal device can decode the video stream data through the first decoder to obtain the projection data corresponding to the first projection window, and perform scaling processing on the projection data according to the first resolution corresponding to the first projection window and the first target resolution, so as to play the projection picture in the first projection window. The first terminal device generates video stream data corresponding to each projection window, and plays the projection picture in the corresponding projection window of the second terminal device in a similar manner as described in the above embodiments.

[0102] Exemplarily, Figure 6 A schematic diagram for background projection of a plurality of projection windows in an embodiment. As Figure 6 shown, the second terminal device can be provided with three different projection windows: projection window 1, projection window 2 and projection window 3, and the first terminal device can create corresponding three virtual screens: virtual screen 1, virtual screen 2 and virtual screen 3, wherein the resolution of the virtual screen 1 is proportional to the resolution of the projection window 1, the resolution of the virtual screen 2 is proportional to the resolution of the projection window 2, and the resolution of the virtual screen 3 is proportional to the resolution of the projection window 3. The first terminal device can output the interface content of the application A to the virtual screen 1, and generate the video stream data corresponding to the projection window 1 by capturing the interface content of the application A displayed in the virtual screen 1, and send the video stream data to the second terminal device, so as to play the projection picture corresponding to the application A in the projection window 1 of the second terminal device. Similarly, the projection window 2 can play the projection picture corresponding to the application B, and the projection window 3 can play the projection picture corresponding to the application C. In the embodiments of the present application, it can be realized that a plurality of projection windows are set in the second terminal device to play different projection pictures respectively, and it can be ensured that there is no black border or picture stretching and distortion in each projection window, which improves the projection effect and meets the different needs of users.

[0103] In the embodiments of the present application, the first terminal device can perform screen projection on the second terminal device in the background, which can avoid the situation that screen projection affects the normal use of the first terminal device, and can ensure that the first terminal device and the second terminal device can normally perform encoding and decoding processing in the screen projection process, and the screen projection picture will not have the problems of black edges or picture stretching and distortion when playing in the screen projection window, thereby improving the screen projection effect.

[0104] In some embodiments, after the first terminal device and the second terminal device establish the screen projection link, before the second terminal device sends the first resolution corresponding to the screen projection window to the first terminal device, the first terminal device can perform screen projection in the second terminal device according to the resolution of the first terminal device. The first terminal device can determine the initial resolution proportional to the second resolution according to the encoding and decoding capability corresponding to the second terminal device, the encoding and decoding capability corresponding to the first terminal device, and the second resolution corresponding to the first terminal device.

[0105] The second resolution corresponding to the first terminal device can be the resolution when the first terminal device is full-screen displayed, that is, the resolution of the physical screen of the first terminal device. Alternatively, the second resolution can also be the display resolution of the target application program that needs to be projected in the first terminal device on the physical screen. The first terminal device and the second terminal device can perform encoding and decoding capability negotiation, determine the encoding and decoding parameters according to the encoding and decoding capability corresponding to the second terminal device and the encoding and decoding capability corresponding to the first terminal device, and determine the initial resolution proportional to the second resolution according to the encoding and decoding parameters and the second resolution corresponding to the first terminal device.

[0106] As a specific implementation, the first maximum video width and height parameter supported by the first terminal device can be compared with the second maximum video width and height parameter supported by the second terminal device, and the smaller maximum video width and height parameter is selected as the target video width and height parameter, and the initial resolution proportional to the second resolution is determined within the resolution range corresponding to the target video width and height parameter. For example, the second resolution is 300*200, and the target video width and height parameter is 280*160, and the initial resolution proportional to 300*200 can be determined within the range of 280*160, such as 240*160.

[0107] It should be noted that the way of determining the initial resolution can be similar to the way of determining the target resolution described in the above embodiments, except that the initial resolution is proportional to the second resolution, and the target resolution is proportional to the first resolution. The embodiments of the present application will not repeat the way of determining the initial resolution, and the related description of determining the first resolution in the above embodiments can be referred to.

[0108] The first terminal device can send video stream data matching the initial resolution to the second terminal device. After the first terminal device receives the first resolution corresponding to the screen projection window sent by the second terminal device, the target resolution equal to the first resolution can be determined, and the video stream data matching the target resolution can be sent to the second terminal device. If the first terminal device receives the first end request sent by the second terminal device, the first end request can be used to indicate that the screen projection in the screen projection window is ended. Then, the first terminal device can send the video stream data matching the initial resolution to the second terminal device according to the first end request. That is, after the second terminal device ends the screen projection in the screen projection window, the first terminal device can return to the mode of screen projection in the second terminal device based on the initial resolution. Further, the first end request can include an end background screen projection request, which is used to request to end the background screen projection. Then, the target application can return to the mode of foreground screen projection, and the screen projection in the second terminal device based on the initial resolution.

[0109] Exemplarily, Figure 7 A timing diagram of a screen projection method in an embodiment is shown in FIG. 1. As shown in FIG. 1, the screen projection method can include the following steps: Figure 7

[0110] Step 1: The second terminal device can request the first terminal device to establish a screen projection link. The first terminal device and the second terminal device successfully establish the screen projection link.

[0111] Step 2: The first terminal device and the second terminal device negotiate the coding and decoding capability and resolution, etc.

[0112] The second terminal device can send its coding and decoding capability to the first terminal device. The first terminal device determines the coding and decoding parameter according to the coding and decoding capability of the second terminal device and the coding and decoding capability of the first terminal device, and determines the initial resolution equal to the second resolution according to the coding and decoding parameter and the second resolution corresponding to the first terminal device.

[0113] The first terminal device can create an encoder according to the initial resolution, and grab the screen projection data. The first terminal device can also send the initial resolution to the second terminal device. The second terminal device creates a decoder according to the initial resolution.

[0114] Step 3: The first terminal device pushes the audio and video stream to the second terminal device. The resolution of the video stream data is the initial resolution.

[0115] ​The first terminal device can encode the captured projection data using an encoder with a resolution equal to the initial resolution to obtain video stream data that matches the initial resolution, and push the audio stream data and the video stream data to the second terminal device. The second terminal device can decode the received video stream data using a decoder with a resolution equal to the initial resolution and play the corresponding projection screen.

[0116] Step 4: The second terminal device requests background screen projection from the first terminal device, and the request may carry the first resolution corresponding to the projection window.

[0117] Step 5: The first terminal device determines a target resolution that is proportional to the first resolution based on the negotiated codec capability.

[0118] The first terminal device can determine a target resolution that is proportional to the first resolution based on the encoding and decoding parameters determined in step 2 and the first resolution.

[0119] Step 6: The first terminal device creates a virtual screen and encoder according to the target resolution.

[0120] Step 7: The first terminal device receives the target application to the virtual screen, outputs the interface content of the target application to the virtual screen, and captures the projection data from the virtual screen.

[0121] Step 8: The first terminal device sends the target resolution to the second terminal device.

[0122] Step 9: The second terminal device creates a decoder according to the target resolution.

[0123] Step 10: The first terminal device pushes the audio and video stream to the second terminal device, wherein the resolution of the video stream data is the target resolution.

[0124] The first terminal device can encode the projection data captured from the virtual screen through an encoder with a resolution of the target resolution to obtain video stream data matching the target resolution, and push the audio stream data and the video stream data to the second terminal device.

[0125] Step 11: The second terminal device scales the video stream data that matches the target resolution, and plays the projection image without black edges and without stretching or distortion in the projection window.

[0126] The second terminal device can decode the received video stream data through a decoder with a resolution of the target resolution to obtain screen projection data, and scale the screen projection data.

[0127] Step 12: The second terminal device requests the first terminal device to end background screen projection.

[0128] Step 13: The first terminal device pulls the target application to the physical screen and outputs the interface content of the target application to the physical screen.

[0129] Step 14: The first terminal device creates an encoder according to the target resolution and captures the projection data.

[0130] Step 15: The first terminal device sends the initial resolution to the second terminal device.

[0131] Step 16: The second terminal device creates a new decoder according to the initial resolution.

[0132] Step 17: The first terminal device pushes the audio and video stream to the second terminal device, wherein the resolution of the video stream data is the initial resolution.

[0133] The first terminal device can encode the captured projection data using an encoder with a resolution equal to the initial resolution to obtain video stream data that matches the initial resolution, and push the audio stream data and the video stream data to the second terminal device. The second terminal device can decode the received video stream data using a decoder with a resolution equal to the initial resolution and play the corresponding projection screen.

[0134] If the screen projection is to be ended, the screen projection link between the first terminal device and the second terminal device can be disconnected. The screen projection link can be disconnected by the first terminal device or by the second terminal device.

[0135] In an embodiment of the present application, before and after background screen projection, screen projection is performed on the second terminal device based on the second resolution corresponding to the first terminal device, and the foreground screen projection and the background screen projection are distinguished, which can give the user a prompt of the change, and avoid the user being unclear about when to start and end the background screen projection during the projection process, thereby meeting user needs and improving user experience.

[0136] like Figure 8 As shown, in one embodiment, a screen projection method is provided, which can be applied to the above-mentioned second terminal device. The screen projection method may include the following steps:

[0137] Step 810: Receive video stream data matching the target resolution sent by the first terminal device. The target resolution is determined by the first terminal device based on the projection parameters corresponding to the second terminal device; the projection parameters include the first resolution corresponding to the projection window of the second terminal device and the encoding and decoding capabilities corresponding to the second terminal device. The target resolution is proportional to the first resolution.

[0138] In one embodiment, before step 810, the method further includes: sending the projection parameters corresponding to the second terminal device to the first terminal device; or, sending the device information of the second terminal device to the first terminal device, so that the first terminal device queries the projection parameters corresponding to the device information.

[0139] Step 820: Play the projection image in the projection window according to the video stream data.

[0140] In one embodiment, the step of sending the projection parameters corresponding to the second terminal device to the first terminal device may include: sending the projection parameters corresponding to the second terminal device to the first terminal device through a first transmission channel established between the first terminal device and the first terminal device; step 820 may include: receiving video stream data matching the target resolution sent by the first terminal device through a second transmission channel established between the first terminal device and the first terminal device; wherein the first transmission channel is a bidirectional transmission channel, and the second transmission channel is a unidirectional transmission channel.

[0141] In one embodiment, before step 810, the method also includes: receiving a target resolution sent by the first terminal device; creating a decoder according to the target resolution; step 820 may include: decoding the video stream data through the decoder to obtain projection data; scaling the projection data according to the first resolution and the target resolution, and outputting the scaled projection data to the projection window to play the projection image in the projection window.

[0142] It should be noted that the screen projection method applied to the second terminal device provided in the embodiments of the present application can refer to the relevant description of the screen projection method applied to the first terminal device provided in the above embodiments, and will not be repeated here.

[0143] In an embodiment of the present application, the first terminal device (i.e., the source device in the screen projection) can adaptively adjust the resolution of the transmitted video stream data according to the first resolution corresponding to the screen projection window of the second terminal device (i.e., the counterpart device in the screen projection) and the encoding and decoding capability corresponding to the second terminal device. This can not only ensure that the second terminal device decodes the video stream data normally, but also ensure that the projected screen will not have black edges or stretched or distorted images when played in the screen projection window, thereby improving the screen projection effect.

[0144] like Figure 9 As shown, in one embodiment, a screen projection device 900 is provided, which can be applied to the above-mentioned first terminal device. The screen projection device 900 may include a parameter acquisition module 910, a resolution determination module 920 and a data sending module 930.

[0145] The parameter obtaining module 910 is configured to obtain a screen projection parameter corresponding to the second terminal device, the screen projection parameter comprising a first resolution corresponding to a screen projection window of the second terminal device and a coding and decoding capability.

[0146] The resolution determining module 920 is configured to determine a target resolution that is proportional to the first resolution according to the screen projection parameter.

[0147] The data sending module 930 is configured to send video stream data matching the target resolution to the second terminal device, the video stream data being used to play a screen projection picture in the screen projection window of the second terminal device.

[0148] In the embodiments of the present application, the first terminal device (i.e., the source terminal device in screen projection) can adaptively adjust the resolution of the transmitted video stream data according to the first resolution corresponding to the screen projection window of the second terminal device (i.e., the opposite terminal device in screen projection) and the coding and decoding capability corresponding to the second terminal device, which can ensure that the second terminal device normally decodes the video stream data and can ensure that the screen projection picture does not have black edges or picture stretching and distortion when played in the screen projection window, thereby improving the screen projection effect.

[0149] In one embodiment, the parameter obtaining module 910 is further configured to receive the screen projection parameter sent by the second terminal device, or receive device information sent by the second terminal device and query the screen projection parameter corresponding to the device information.

[0150] In one embodiment, the parameter obtaining module 910 is further configured to receive the screen projection parameter sent by the second terminal device through a first transmission channel established between the first terminal device and the second terminal device.

[0151] The data sending module 930 is further configured to send the video stream data matching the target resolution to the second terminal device through a second transmission channel established between the first terminal device and the second terminal device.

[0152] The first transmission channel is a bidirectional transmission channel, and the second transmission channel is a unidirectional transmission channel.

[0153] In one embodiment, the resolution determining module 920 comprises a coding and decoding parameter determining unit and a resolution determining unit.

[0154] The coding and decoding parameter determining unit is configured to determine a coding and decoding parameter according to the coding and decoding capability corresponding to the second terminal device and the coding and decoding capability corresponding to the first terminal device.

[0155] The resolution determining unit is configured to determine a target resolution that is proportional to the first resolution according to the coding and decoding parameter and the first resolution corresponding to the screen projection window of the second terminal device.

[0156] In one embodiment, the codec capability comprises a video width-height parameter; the codec parameter determination unit is further configured to compare a first maximum video width-height parameter supported by the first terminal device with a second maximum video width-height parameter supported by the second terminal device, and select a smaller maximum video width-height parameter as a target video width-height parameter.

[0157] The resolution determination unit is further configured to determine a target resolution that is proportional to the first resolution within a resolution range corresponding to the target video width-height parameter.

[0158] In one embodiment, the resolution determination unit is further configured to determine a maximum resolution that is proportional to the first resolution within a resolution range corresponding to the target video width-height parameter, and select the maximum resolution as the target resolution.

[0159] In one embodiment, the resolution determination unit is further configured to determine a target resolution that is proportional to the first resolution within a resolution range corresponding to the target video width-height parameter according to a network transmission parameter and / or a performance parameter of the second terminal device.

[0160] In the embodiments of the present application, the first terminal device can determine a resolution range that can be supported by both the first terminal device and the second terminal device in combination with the codec capability of the first terminal device and the codec capability corresponding to the second terminal device, and determine a target resolution that is proportional to the first resolution corresponding to the screen projection window of the second terminal device within the resolution range, so as to ensure that the first terminal device and the second terminal device can normally perform codec processing in the screen projection process, and can ensure that there is no black edge or picture stretching and distortion when the screen projection picture is played in the screen projection window, thereby improving the screen projection effect.

[0161] In one embodiment, the screen projection apparatus 900 described above further comprises a virtual screen creation module, an output module in addition to the parameter acquisition module 910, the resolution determination module 920 and the data sending module 930.

[0162] The virtual screen creation module is configured to create a virtual screen according to the target resolution.

[0163] The output module is configured to output the interface content of the target application program to the virtual screen.

[0164] The data sending module 930 is further configured to capture the interface content of the target application program from the virtual screen as screen projection data, generate video stream data matching the target resolution according to the screen projection data, and send the video stream data to the second terminal device.

[0165] In one embodiment, the screen projection apparatus 900 described above further comprises an encoder creation module and a parameter sending module.

[0166] The encoder creation module is configured to create an encoder according to the target resolution.

[0167] The parameter sending module is configured to send the target resolution to the second terminal device, so that the second terminal device creates a decoder according to the target resolution.

[0168] The data sending module 930 is further configured to encode the screen projection data by using the encoder to obtain video stream data matching the target resolution, and send the video stream data to the second terminal device; the video stream data is used for the second terminal device to decode the video stream data by using the decoder to obtain the screen projection data, and perform scaling processing on the screen projection data according to the first resolution and the target resolution, so as to play the screen projection picture in the screen projection window of the second terminal device.

[0169] In one embodiment, the screen projection parameters further include a plurality of first resolutions respectively corresponding to a plurality of screen projection windows in the second terminal device; the resolution determining module 920 is further configured to respectively determine a target resolution proportional to each of the first resolutions according to the coding and decoding capability corresponding to the second terminal device and the first resolutions respectively corresponding to the screen projection windows.

[0170] The virtual screen creating module is further configured to respectively create a virtual screen corresponding to each of the screen projection windows according to the target resolution proportional to each of the first resolutions.

[0171] In the embodiments of the present application, the first terminal device can perform screen projection on the second terminal device in the background, which can avoid the situation that the screen projection affects the normal use of the first terminal device by the user, and can ensure that the first terminal device and the second terminal device can normally perform coding and decoding processing during the screen projection process, so that the screen projection picture will not appear black edges or picture stretching and distortion when played in the screen projection window, thereby improving the screen projection effect.

[0172] In one embodiment, the resolution determining module 920 is further configured to, before determining the target resolution proportional to the first resolution according to the screen projection parameters, determine an initial resolution proportional to a second resolution according to the coding and decoding capability corresponding to the second terminal device, the coding and decoding capability corresponding to the first terminal device, and the second resolution corresponding to the first terminal device.

[0173] The data sending module 930 is further configured to send video stream data matching the initial resolution to the second terminal device.

[0174] The data sending module 930 is further configured to receive a first end request sent by the second terminal device; and send the video stream data matching the initial resolution to the second terminal device according to the first end request.

[0175] In the embodiments of the present application, the screen projection is performed in the second terminal device based on the second resolution corresponding to the first terminal device before and after the background screen projection, the foreground screen projection is distinguished from the background screen projection, the user can be prompted to change, the user can be avoided from being unclear about when to start and end the background screen projection in the screen projection process, the user demand is met, and the user experience is improved.

[0176] As shown in Figure 10 In one embodiment, a screen projection device 1000 is provided, which can be applied to the second terminal device described above, and the screen projection device 1000 can include a data receiving module 1010 and a playing module 1020.

[0177] The data receiving module 1010 is configured to receive video stream data matching a target resolution sent by the first terminal device, and the target resolution is determined by the first terminal device according to screen projection parameters corresponding to the second terminal device; the screen projection parameters include a first resolution corresponding to a screen projection window of the second terminal device and a codec capability, and the target resolution is proportional to the first resolution.

[0178] In one embodiment, the screen projection device 1000 further includes a sending module.

[0179] The sending module is configured to send the screen projection parameters corresponding to the second terminal device to the first terminal device, or send device information of the second terminal device to the first terminal device, so that the first terminal device queries the screen projection parameters corresponding to the device information.

[0180] In one embodiment, the sending module is further configured to send the screen projection parameters corresponding to the second terminal device to the first terminal device through a first transmission channel established between the first terminal device and the second terminal device.

[0181] The data receiving module 1010 is further configured to receive the video stream data matching the target resolution sent by the first terminal device through a second transmission channel established between the first terminal device and the second terminal device.

[0182] The first transmission channel is a bidirectional transmission channel, and the second transmission channel is a unidirectional transmission channel.

[0183] The playing module 1020 is configured to play a screen projection picture in a screen projection window according to the video stream data.

[0184] In one embodiment, the screen projection device 1000 further includes a parameter receiving module and a decoder creating module.

[0185] The parameter receiving module is configured to receive the target resolution sent by the first terminal device.

[0186] The decoder creating module is configured to create a decoder according to the target resolution.

[0187] The data receiving module 1010 is further configured to decode the video stream data by a decoder to obtain the screen projection data, perform scaling processing on the screen projection data according to the first resolution and the target resolution, and output the screen projection data after the scaling processing to the screen projection window to play the screen projection picture in the screen projection window.

[0188] In the embodiments of the present application, the first terminal device (i.e., the source device in the screen projection) can adaptively adjust the resolution of the transmitted video stream data according to the first resolution corresponding to the screen projection window of the second terminal device (i.e., the opposite device in the screen projection) and the coding and decoding capability of the second terminal device, which can not only ensure that the second terminal device can normally decode the video stream data, but also ensure that the screen projection picture will not appear black edges or picture stretching and distortion when played in the screen projection window, thereby improving the screen projection effect.

[0189] Figure 11 FIG. 11 is a structural block diagram of a terminal device according to an embodiment of the present application. Figure 11 As shown in FIG. 11, the terminal device 1100 can include one or more of the following components: a processor 1110, and a memory 1120 coupled to the processor 1110, wherein the memory 1120 can store one or more computer programs, and the one or more computer programs can be configured to be executed by the one or more processors 1110 to implement the screen projection method applied to the first terminal device as described in the above embodiments.

[0190] The processor 1110 can include one or more processing cores. The processor 1110 connects various parts within the terminal device 1100 by various interfaces and lines, and performs various functions of the terminal device 1100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1120, and calling data stored in the memory 1120. Alternatively, the processor 1110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1110 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU is mainly used to process operating systems, user interfaces, and application programs; the GPU is used to render and draw display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1110, but can be realized by a separate communication chip.

[0191] The memory 1120 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 1120 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1120 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can also store data created by the terminal device 1100 in use, etc.

[0192] It can be understood that the terminal device 1100 can include more or fewer structural elements than those in the above structural block diagram, for example, including a power module, a physical key, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., which are not limited herein.

[0193] The embodiments of the present application disclose a terminal device, including a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor implement the screen projection method applied to a second terminal device as described in the above embodiments.

[0194] The embodiment of the present application discloses a computer readable storage medium which stores a computer program, wherein the computer program is executed by a processor to realize the screen projection method applied to the first terminal device as described in the above embodiments.

[0195] The embodiment of the present application discloses a computer readable storage medium which stores a computer program, wherein the computer program is executed by a processor to realize the screen projection method applied to the second terminal device as described in the above embodiments.

[0196] The embodiment of the present application discloses a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program can be executed by a processor to realize the screen projection method applied to the first terminal device as described in the above embodiments.

[0197] The embodiment of the present application discloses a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program can be executed by a processor to realize the screen projection method applied to the second terminal device as described in the above embodiments.

[0198] Those skilled in the art can understand that all or part of the flow of the above-mentioned embodiment method can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, the program can include the flow of the above-mentioned embodiment method. The storage medium can be a magnetic disc, an optical disc, a ROM, etc.

[0199] As used herein, any reference to memory, storage, a database or other medium can include non-volatile and / or volatile storage. Suitable non-volatile storage can include ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), or flash memory. Volatile storage can include random access memory (RAM), which acts as external cache. By way of illustration and not limitation, RAM can be SRAM, DRAM, synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and the like.

[0200] It is to be understood that the terminology "one embodiment" or "an embodiment" used throughout this specification can denote a particular feature, structure, or characteristic included in at least one embodiment of the application. Thus, the appearing of "in one embodiment" or "in an embodiment" at various places throughout the specification are not necessarily referring to the same embodiment. Furthermore, the various features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is also to be understood that the embodiments described herein are optional embodiments, and the acts and modules involved are not necessarily required by the application.

[0201] In various embodiments of the present application, it should be understood that the magnitude of the serial number of the above-mentioned processes does not mean the inevitable sequence of execution order, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0202] The units described as separate components above can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0203] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0204] Each technical feature of the above-described embodiments can be combined arbitrarily, and in order to make the description concise, all possible combinations of each technical feature in the above-described embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0205] The above describes in detail a screen projection method, device, terminal equipment and computer readable storage medium disclosed by the embodiments of the present application, the principle and implementation mode of the present application are described in this paper by applying specific examples, the above embodiment is only used to help understand the method and its core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A screen projection method, characterized in that: Applied to a first terminal device, the method includes: Obtaining screen projection parameters corresponding to the second terminal device, including: obtaining codec capabilities corresponding to the second terminal device; receiving a background screen projection request sent by the second terminal, the background screen projection request including first resolutions corresponding to multiple screen projection windows in the second terminal device; a target application for screen projection in the first terminal device is running in the background; Determining codec parameters according to the codec capability corresponding to the second terminal device and the codec capability corresponding to the first terminal device; Determine, according to the encoding and decoding parameters and the first resolution corresponding to each of the projection windows, a target resolution that is proportional to the first resolution corresponding to each of the projection windows; Send video stream data that matches the target resolution that is proportional to each of the first resolutions to the second terminal device, and the video stream data is used to play the projected screen in the corresponding projection window in the second terminal device; wherein, the video stream data corresponding to the first projection window is generated based on the interface content of the target application corresponding to the first projection window, and the video stream data corresponding to the first projection window is used to play the projected screen in the first projection window in the second terminal device, and the first projection window is any of the projection windows.

2. The method according to claim 1, characterized in that The codec capability includes video width and height parameters; and determining the codec parameters according to the codec capability corresponding to the second terminal device and the codec capability corresponding to the first terminal device includes: comparing a first maximum video width and height parameter supported by the first terminal device with a second maximum video width and height parameter supported by the second terminal device, and selecting the smaller maximum video width and height parameter as the target video width and height parameter; The determining, based on the encoding and decoding parameters and the first resolution corresponding to each of the projection windows, a target resolution proportional to the first resolution corresponding to each of the projection windows, includes: Within a resolution range corresponding to the width and height parameters of the target video, a target resolution that is proportional to the first resolution is determined.

3. The method according to claim 2, characterized in that Determining a target resolution that is proportional to the first resolution within a resolution range corresponding to the width and height parameters of the target video includes: Within the resolution range corresponding to the width and height parameters of the target video, a maximum resolution that is proportional to the first resolution is determined, and the maximum resolution is used as the target resolution.

4. The method according to claim 2, characterized in that Determining a target resolution that is proportional to the first resolution within a resolution range corresponding to the width and height parameters of the target video includes: Within the resolution range corresponding to the target video width and height parameters, a target resolution that is proportional to the first resolution is determined according to network transmission parameters and / or performance parameters of the second terminal device.

5. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the projection parameters corresponding to the second terminal device includes: Receive the projection parameters sent by the second terminal device; or, Receive the device information sent by the second terminal device and query the screen projection parameters corresponding to the device information.

6. The method according to claim 5, characterized in that The receiving of the screen projection parameters sent by the second terminal device includes: Receiving, via the first transmission channel established with the second terminal device, screen projection parameters sent by the second terminal device; The sending the video stream data matching the target resolution to the second terminal device includes: sending video stream data matching the target resolution to the second terminal device through a second transmission channel established with the second terminal device; The first transmission channel is a bidirectional transmission channel, and the second transmission channel is a unidirectional transmission channel.

7. The method according to any one of claims 1 to 4, characterized in that: After determining, based on the encoding and decoding parameters and the first resolution corresponding to each of the projection windows, a target resolution that is proportional to the first resolution corresponding to each of the projection windows, the method further includes: Creating virtual screens corresponding to the respective projection windows according to target resolutions that are proportional to the respective first resolutions; Outputting the interface content of the target application corresponding to each of the projection windows to the virtual screen corresponding to each of the projection windows; The sending, to the second terminal device, video stream data matching a target resolution that is proportional to each of the first resolutions, includes: From the virtual screen corresponding to the first projection window, capture the interface content of the target application corresponding to the first projection window as the projection data corresponding to the first projection window, generate video stream data that matches the target resolution that is proportional to the first resolution corresponding to the first projection window based on the projection data corresponding to the first projection window, and send the video stream data to the second terminal device; the first projection window is any of the projection windows.

8. The method according to claim 7, characterized in that After determining, based on the encoding and decoding parameters and the first resolution corresponding to each of the projection windows, a target resolution that is proportional to the first resolution corresponding to each of the projection windows, the method further includes: Creating an encoder corresponding to each of the projection windows according to a target resolution proportional to the first resolution corresponding to each of the projection windows; Sending a target resolution proportional to the first resolution corresponding to each of the projection windows to the second terminal device, so that the second terminal device creates a decoder corresponding to each of the projection windows according to the target resolution proportional to the first resolution corresponding to each of the projection windows; The generating, according to the projection data corresponding to the first projection window, video stream data matching a target resolution that is proportional to the first resolution corresponding to the first projection window, and sending the video stream data to the second terminal device, includes: The projection data corresponding to the first projection window is encoded by the encoder corresponding to the first projection window to obtain video stream data that matches the target resolution that is proportional to the first resolution corresponding to the first projection window, and the video stream data is sent to the second terminal device; the video stream data is used by the second terminal device to decode the video stream data by the decoder corresponding to the first projection window to obtain the projection data corresponding to the first projection window, and the projection data is scaled according to the first resolution corresponding to the first projection window and the target resolution that is proportional to the first resolution corresponding to the first projection window, so as to play the projection screen in the first projection window of the second terminal device.

9. The method according to any one of claims 1 to 4, characterized in that: Before determining a target resolution that is proportional to the first resolution according to the screen projection parameters, the method further includes: Determining an initial resolution proportional to the second resolution based on the codec capability corresponding to the second terminal device, the codec capability corresponding to the first terminal device, and the second resolution corresponding to the first terminal device; Sending video stream data matching the initial resolution to the second terminal device; After sending the video stream data matching the target resolution to the second terminal device, the method further includes: receiving a first end request sent by the second terminal device; Video stream data matching the initial resolution is sent to the second terminal device according to the first end request.

10. A screen projection method, characterized in that: Applied to a second terminal device, the method includes: A background screen projection request is sent to the first terminal device, where the background screen projection request carries the first resolutions corresponding to the multiple projection windows in the second terminal device, so that the first terminal device determines codec parameters according to the codec capabilities corresponding to the second terminal device and the codec capabilities corresponding to the first terminal device, and determines a target resolution proportional to the first resolution corresponding to each projection window according to the codec parameters and the first resolution corresponding to each projection window; a target application for projection in the first terminal device runs in the background; Receive video stream data sent by the first terminal device that matches a target resolution that is proportional to each of the first resolutions; wherein the video stream data corresponding to the first projection window is generated by the first terminal device based on the interface content of the target application corresponding to the first projection window, and the video stream data corresponding to the first projection window is used to play the projection screen in the first projection window in the second terminal device, and the first projection window is any of the projection windows; The projection screen is played in each projection window according to the video stream data that matches the target resolution in proportion to each of the first resolutions.

11. The method according to claim 10, characterized in that Before receiving the video stream data sent by the first terminal device and matching the target resolutions that are proportional to each of the first resolutions, the method further includes: Sending the projection parameters corresponding to the second terminal device to the first terminal device; or, Send the device information of the second terminal device to the first terminal device, so that the first terminal device queries the screen projection parameters corresponding to the device information.

12. The method according to claim 11, characterized in that The sending the screen projection parameters corresponding to the second terminal device to the first terminal device includes: Sending the screen projection parameters corresponding to the second terminal device to the first terminal device through the first transmission channel established with the first terminal device; The receiving video stream data matching the target resolution sent by the first terminal device includes: receiving, via a second transmission channel established with the first terminal device, video stream data matching a target resolution and sent by the first terminal device; The first transmission channel is a bidirectional transmission channel, and the second transmission channel is a unidirectional transmission channel.

13. The method according to claim 10, characterized in that Before receiving the video stream data sent by the first terminal device and matching the target resolutions that are proportional to each of the first resolutions, the method further includes: Receive a target resolution proportional to the first resolution corresponding to each projection window sent by the first terminal device; Creating a decoder corresponding to each of the projection windows according to a target resolution proportional to the first resolution corresponding to each of the projection windows; Playing the projection screen in each projection window according to the video stream data matched with the target resolution in proportion to each of the first resolutions includes: Decoding the video stream data corresponding to the first projection window through a decoder corresponding to the first projection window to obtain projection data corresponding to the first projection window; the first projection window is any of the projection windows, and the video stream data corresponding to the first projection window is video stream data that matches the target resolution proportional to the first resolution corresponding to the first projection window; According to the first resolution corresponding to the first projection window and the target resolution proportional to the first resolution corresponding to the first projection window, the projection data corresponding to the first projection window is scaled, and the scaled projection data is output to the first projection window to play the projection screen in the first projection window.

14. A screen projection device, characterized in that: Applied to a first terminal device, the apparatus includes: The parameter acquisition module includes: obtaining the codec capability corresponding to the second terminal device; receiving a background screen projection request sent by the second terminal, the background screen projection request carrying a first resolution corresponding to the projection window of the second terminal device; the projection window includes a full-screen window or a window of a custom size; the background screen projection request includes the first resolutions corresponding to the multiple projection windows in the second terminal device; the target application for projection in the first terminal device runs in the background; a codec parameter determination unit, configured to determine codec parameters according to the codec capability corresponding to the second terminal device and the codec capability corresponding to the first terminal device; a resolution determination module, configured to determine a target resolution proportional to the first resolution corresponding to each of the projection windows based on the encoding and decoding parameters and the first resolution corresponding to each of the projection windows; A data sending module is used to send video stream data that matches the target resolution that is proportional to each of the first resolutions to the second terminal device, and the video stream data is used to play the projected screen in the corresponding projection window in the second terminal device; wherein, the video stream data corresponding to the first projection window is generated based on the interface content of the target application corresponding to the first projection window, and the video stream data corresponding to the first projection window is used to play the projected screen in the first projection window in the second terminal device, and the first projection window is any of the projection windows.

15. A screen projection device, characterized in that: Applied to a second terminal device, the apparatus includes: A sending module is used to send a background screen projection request to the first terminal device, wherein the background screen projection request carries the first resolutions corresponding to the multiple screen projection windows in the second terminal device, so that the first terminal device determines the codec parameters according to the codec capabilities corresponding to the second terminal device and the codec capabilities corresponding to the first terminal device, and determines a target resolution proportional to the first resolution corresponding to each of the screen projection windows according to the codec parameters and the first resolution corresponding to each of the screen projection windows; the target application for screen projection in the first terminal device runs in the background; a data receiving module, configured to receive video stream data sent by the first terminal device that matches a target resolution that is proportional to each of the first resolutions; wherein the video stream data corresponding to the first projection window is generated by the first terminal device based on the interface content of the target application corresponding to the first projection window, and the video stream data corresponding to the first projection window is used to play the projection screen in the first projection window in the second terminal device, and the first projection window is any of the projection windows; The playback module is used to play the projection screen in the projection window corresponding to each first resolution according to the video stream data matched with the target resolution in proportion to each first resolution.

16. A terminal device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 9.

17. A terminal device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 10 to 13.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 10 to 13 is implemented.

Images