Screen projection code stream adjustment method and system

Abstract

The present invention discloses a method and system for adjusting screen projection code stream. After receiving and storing data sent by a sending end, the redundant data length and the remaining length of the receiving end container are obtained according to the current received data length and the total length of the receiving end container, and then it is determined whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and corresponding instructions are sent to the sending end according to the situation that the redundant data length is greater than the preset redundant length, so that the sending end adjusts the data code stream according to the corresponding instructions, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, the client is notified of the dynamic change of the data code stream through the margin of data storage, which can improve the current high latency problem caused by data packet redundancy and improve the stability of screen projection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for adjusting screen projection code stream. Background Art

[0002] Video screen projection refers to the process of synchronizing the video played on a smart device such as a mobile phone, tablet, or computer to another device for playback and viewing.

[0003] The existing screen projection solutions cannot dynamically adjust the current image quality according to the current network conditions. When the network delay is high, a series of problems such as screen freeze and screen distortion caused by decoding may occur, making the screen projection unstable and with high latency, affecting the screen projection experience. Summary of the invention

[0004] The present invention provides a method for adjusting screen projection code stream, which improves the stability of screen projection by dynamically adjusting the screen projection data code stream according to the margin of data storage.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a method for adjusting a screen projection code stream, including:

[0006] Receive first data sent by the sending end, store the first data into a receiving end container and obtain a current first data length and current first data information, wherein the first data information includes a current first data frame width, a current first data code stream, and a current first data frame rate;

[0007] Obtaining the current first data redundant length and the remaining length of the receiving end container according to the current first data length and the total length of the receiving end container;

[0008] It is determined whether the redundant length of the current first data is greater than the preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the receiving end container is sent to the sending end, so that the sending end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, and then sends the pre-processed data to the receiving end. If not, the remaining length of the receiving end container and the current first data information are returned to the sending end.

[0009] In the implementation of the embodiment of the present invention, data sent by the sending end is received, and the receiving end stores one data when receiving one data. After obtaining the redundant data length and the remaining length of the receiving end container according to the current received data length and the total length of the receiving end container, it is determined whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and the remaining capacity of the receiving end container is sent to the sending end, so that the sending end adjusts the data code stream according to the remaining capacity of the receiving end container, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, it is possible to realize the dynamic change of the data code stream by notifying the client through the remaining amount of data storage, thereby improving the stability of screen projection.

[0010] As a preferred solution, the first data sent by the sending end is received, the first data is stored in the receiving end container, and the current first data length and current first data information are obtained, specifically:

[0011] The first data is sent by the sending end through the TCP connection communication receiving end;

[0012] The first data is stored in a storage container using a queue and a current first data redundancy length value and current first data information are obtained.

[0013] As a preferred solution, it is determined whether the current redundant length of the first data is greater than the preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the container at the receiving end is sent to the sending end, specifically:

[0014] If the current redundant length of the first data is greater than the first preset redundant length, the redundant data of the current first data is cleared and the first remaining length of the receiving end container is sent to the sending end;

[0015] If the current redundant length of the first data is greater than the second preset redundant length, the redundant data of the current first data is cleared and the second remaining length of the receiving end container is sent to the sending end.

[0016] As a preferred solution, after the sending end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, the pre-processed data is sent to the receiving end, specifically:

[0017] If the sending end receives the first remaining length of the container at the receiving end, reducing the code stream of the first data to obtain pre-processed first data and sending the pre-processed first data to the receiving end;

[0018] If the sending end receives the second remaining length of the container at the receiving end, the sending end generates an I frame and the first data carrying the SPS and PPS information as the first detection data, and sends the first detection data to the receiving end.

[0019] In order to solve the same technical problem, the present invention also provides a screen projection code stream adjustment system, comprising:

[0020] A data receiving module, used for receiving first data sent by a transmitting end, storing the first data into a receiving end container and obtaining a current first data length and current first data information, wherein the first data information includes a current first data frame width, a current first data code stream, and a current first data frame rate;

[0021] A data calculation module, used to obtain a current first data redundant length and a remaining length of a receiving end container according to a current first data length and a total length of a receiving end container;

[0022] The data judgment module is used to judge whether the redundant length of the current first data is greater than the preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the receiving end container is sent to the sending end, so that the sending end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, and then sends the pre-processed data to the receiving end. If not, the remaining length of the receiving end container and the current first data information are returned to the sending end.

[0023] As a preferred solution, the data receiving module includes a receiving unit and a computing unit;

[0024] The receiving unit is used to receive the first data sent by the sending end by building a connection communication using TCP;

[0025] The calculation unit is used for storing the first data in a storage container using a queue and obtaining the current first data redundant length and current first data information.

[0026] As a preferred solution, the data judgment module includes a processing unit and a control unit;

[0027] The processing unit is configured to, if the current redundant length of the first data is greater than the first preset redundant length, clear the redundant data of the current first data and send the first remaining length of the receiving end container to the sending end; if the current redundant length of the first data is greater than the second preset redundant length, clear the redundant data of the current first data and send the second remaining length of the receiving end container to the sending end;

[0028] The control unit is used for reducing the code stream of the first data to obtain preprocessed first data and sending the preprocessed first data to the receiving end if the transmitting end receives the first remaining length of the receiving end container, and when the transmitting end receives the second remaining length of the receiving end container, the transmitting end generates an I frame and the first data carrying SPS and PPS information as the first detection data, and sends the first detection data to the receiving end.

[0029] As a preferred solution, in order to solve the same technical problem, an embodiment of the present invention further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the screen projection code stream adjustment method as described in the embodiment of the present invention is implemented.

[0030] As a preferred solution, in order to solve the same technical problem, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer executable program. The computer executable program is used to enable a computer to execute the screen projection code stream adjustment method as described in the embodiment of the present invention.

[0031] The present invention provides a method for adjusting screen projection code stream. After receiving data sent by a sending end, the receiving end stores one data after receiving one data, obtains the redundant data length and the remaining length of the receiving end container according to the current received data length and the total length of the receiving end container, and then determines whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and the corresponding instructions are sent to the sending end according to the situation that the redundant data length is greater than the preset redundant length, so that the sending end adjusts the data code stream according to the corresponding instructions, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, the client is notified of the dynamic change of the data code stream through the data storage margin, which can improve the current high latency problem caused by data packet redundancy and improve the stability of screen projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 : A flowchart of a method for adjusting screen projection code stream provided by an embodiment of the present invention;

[0033] Figure 2 : Another flowchart of the method for adjusting screen projection code stream provided by one embodiment of the present invention;

[0034] Figure 3 : A structural diagram of a screen projection code stream adjustment system provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0037] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0038] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0039] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0040] Embodiment 1

[0041] Please refer to Figure 1 , is a method for adjusting screen projection code stream provided by an embodiment of the present invention, the method includes steps 101 to 103, each step is specifically as follows:

[0042] Step 101: Receive first data sent by a transmitting end, store the first data into a receiving end container and obtain current first data length and current first data information, wherein the first data information includes current first data frame width, current first data code stream, and current first data frame rate.

[0043] Optionally, receiving the first data sent by the sending end, storing the first data in the receiving end container and obtaining the current first data length and the current first data information is specifically:

[0044] The first data is sent by the sending end through the TCP connection communication receiving end;

[0045] The first data is stored in a storage container using a queue and a current first data redundancy length value and current first data information are obtained.

[0046] In this embodiment, the screen projection receiving end can receive a video data packet sent from different types of senders such as the screen projection ios sender, the screen projection Android sender, the screen projection macOS sender, and the screen projection winOS sender. The sender type is not limited to the sender type mentioned above, and based on the C / S mode, TCP is used to establish a connection communication to transmit the data to the receiving end to ensure stable data transmission. After the receiving end receives a video data, it uses a queue storage to store a video data in the client's data container, and then uses a decoder to decode a video detection data stored in the data container and render an image. After decoding the video detection data, a storage space is released and the current data redundant length value, the remaining capacity of the current container and the current video data detection data information are obtained. The current video data detection data information includes the current frame width and height, bit rate, and frame rate.

[0047] Step 102: Obtain the current first data redundant length and the remaining length of the receiving end container according to the current first data length and the total length of the receiving end container.

[0048] In this embodiment, the redundant length value of the current data container and the remaining capacity of the receiving end container are obtained according to the length of the video data and the total length of the receiving end container. After decoding a video frame data, the current data redundant length value, the remaining capacity of the current container and the current video data detection data information are returned to the sending end.

[0049] Step 103: determine whether the current redundant length of the first data is greater than the preset redundant length. If so, clear the redundant data of the current first data and send the corresponding remaining length of the receiving end container to the sending end according to the situation that the current redundant length of the first data is greater than the preset redundant length, so that the sending end adjusts the code stream of the first data according to the remaining length of the receiving end container to obtain pre-processed data, and then sends the pre-processed data to the receiving end. If not, return the remaining length of the receiving end container and the current first data information to the sending end.

[0050] In this embodiment, the video data is processed in two echelons according to the current remaining capacity of the container. When the number of redundant data packets at the receiving end exceeds 5 and when the number of redundant data packets at the receiving end exceeds 10, the data is cleared and instructions are resent so that the sending end adjusts the video data bitstream according to the excess of redundant packets of the video data, and then sends the data after the adjusted video data bitstream to the receiving end. If there is no excess, the remaining capacity of the container at the receiving end and the current video data information are returned to the sending end after decoding a video data.

[0051] Optionally, it is determined whether the current redundant length of the first data is greater than a preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the container at the receiving end is sent to the sending end, specifically:

[0052] If the current redundant length of the first data is greater than the first preset redundant length, the redundant data of the current first data is cleared and the first remaining length of the receiving end container is sent to the sending end;

[0053] If the current redundant length of the first data is greater than the second preset redundant length, the redundant data of the current first data is cleared and the second remaining length of the receiving end container is sent to the sending end.

[0054] Optionally, after the sending end performs code stream adjustment on the first data according to the remaining length of the container at the receiving end to obtain preprocessed data, the preprocessed data is sent to the receiving end, specifically:

[0055] If the sending end receives the first remaining length of the container at the receiving end, reducing the code stream of the first data to obtain pre-processed first data and sending the pre-processed first data to the receiving end;

[0056] If the sending end receives the second remaining length of the container at the receiving end, the sending end generates an I frame and the first data carrying the SPS and PPS information as the first detection data, and sends the first detection data to the receiving end.

[0057] In this embodiment, if Figure 2 As shown, the receiving end initializes the Socket server and the queue storage container ArrayBlockingQueue to ensure the consistency of the data sequence. When the projection receiving end receives a video data from the sending end, it stores the video data in a queue storage manner, and uses a decoder to decode the video data stored in the data container and render an image. After decoding the video data, a storage space is released, and the remaining capacity of the current container is returned to the sending end. The information returned by the receiving end to the sending end includes the current frame width and height, bit rate, frame rate and current data redundancy length value. When the number of redundant data packets at the receiving end exceeds 5, all redundant data packets are cleared and the remaining number of the current container is sent to the sending end. After receiving the remaining number of the current container, the sending end downcodes the data. When the number of redundant data packets at the receiving end exceeds 10, all redundant data packets are cleared and the remaining number of the current receiving container is sent to the sending end. After receiving the remaining capacity of the receiving container, the sending end reduces the bit rate and the frame rate. If the standard frame rate is 30FPS, it is reduced to 20FPS, and the key frame data is resent to the projection receiving end, that is, the data packet that generates an I frame and carries SPS and PPS information is sent to the receiving end as the first frame data, thereby improving the current high latency and data cleaning caused by data packet redundancy.

[0058] As an example of this embodiment, when the redundancy of data packets exceeds 5, the bit rate is reduced. For example, in the case of 1080P, the bit rate is 1000k, then consider reducing the bit rate to 800k, and so on, the minimum is 400k. When the redundancy of data packets exceeds 10, the bit rate is reduced, the frame rate is reduced, and the key frame data is resent. For example, the standard is 30FPS, which is reduced to 20FPS.

[0059] In the implementation of the embodiment of the present invention, after receiving the data sent by the sending end, the redundant data length and the remaining length of the receiving end container are obtained according to the current received data length and the total length of the receiving end container, and then it is determined whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and the corresponding instructions are sent to the sending end according to the situation that the redundant data length is greater than the preset redundant length, so that the sending end adjusts the data bit stream according to the corresponding instructions, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, the client is notified of the dynamic change of the data bit stream through the margin of data storage, which can improve the current high latency problem caused by data packet redundancy and improve the stability of screen projection.

[0060] Embodiment 2

[0061] Accordingly, see Figure 3 , Figure 3 Schematic diagram of the structure of a screen projection code stream adjustment system provided by the present invention. Figure 3 As shown, the screen projection code stream adjustment system includes:

[0062] The data receiving module 301 is used to receive the first data sent by the sending end, store the first data into the receiving end container and obtain the current first data length and the current first data information, wherein the first data information includes the current first data frame width, the current first data code stream, and the current first data frame rate;

[0063] A data calculation module 302, configured to obtain a current first data redundant length and a remaining length of a receiving end container according to a current first data length and a total length of a receiving end container;

[0064] The data judgment module 303 is used to judge whether the redundant length of the current first data is greater than the preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the receiving end container is sent to the sending end, so that the sending end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, and then sends the pre-processed data to the receiving end. If not, the remaining length of the receiving end container and the current first data information are returned to the sending end.

[0065] In this embodiment, the data receiving module 301 includes a receiving unit 3011 and a calculating unit 3012;

[0066] The receiving unit 3011 is used to receive the first data sent by the sending end by using TCP to establish a connection communication;

[0067] The calculation unit 3012 is used to store the first data in a storage container using the queue and obtain the current first data redundant length and the current first data information.

[0068] The data judgment module 303 includes a processing unit 3031 and a control unit 3032;

[0069] The processing unit 3031 is configured to, if the current redundant length of the first data is greater than the first preset redundant length, clear the redundant data of the current first data and send the first remaining length of the receiving end container to the sending end; if the current redundant length of the first data is greater than the second preset redundant length, clear the redundant data of the current first data and send the second remaining length of the receiving end container to the sending end;

[0070] The control unit 3032 is used to reduce the code stream of the first data to obtain preprocessed first data and send the preprocessed first data to the receiving end if the transmitting end receives the first remaining length of the receiving end container. If the transmitting end receives the second remaining length of the receiving end container, the transmitting end generates an I frame and the first data carrying SPS and PPS information is the first detection data, and sends the first detection data to the receiving end.

[0071] For more detailed working principles and processes of this embodiment, please refer to, but not limited to, the relevant description of Embodiment 1.

[0072] The screen projection code stream adjustment system is used to execute the screen projection code stream adjustment method. After receiving the data sent by the sending end, the receiving end stores one data after receiving it. After obtaining the redundant data length and the remaining length of the receiving end container according to the current received data length and the total length of the receiving end container, it is determined whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and the corresponding instructions are sent to the sending end according to the situation that the redundant data length is greater than the preset redundant length, so that the sending end adjusts the data code stream according to the corresponding instructions, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, the client is notified of the dynamic change of the data code stream through the data storage margin, which can improve the current high latency problem caused by data packet redundancy and improve the stability of screen projection.

[0073] Embodiment 3

[0074] Correspondingly, the electronic device for the screen projection code stream adjustment method provided by the present invention includes: a processor and a memory, wherein a computer program is stored in the memory, and the computer program is configured to be executed by the processor, and the processor executes operations corresponding to the screen projection code stream adjustment method shown in Example 1 of the present application.

[0075] Embodiment 4

[0076] Accordingly, the present invention provides a computer-readable storage medium, on which a computer-executable program is stored. When the program is executed by a processor, the screen projection code stream adjustment method shown in the first embodiment of the present application is implemented.

[0077] The present invention provides a method for adjusting screen projection code stream. After receiving data sent by a sending end, the receiving end stores one data after receiving one data, obtains the redundant data length and the remaining length of the receiving end container according to the current received data length and the total length of the receiving end container, and then determines whether the current redundant data length is greater than the preset redundant length. If it is greater than the preset redundant length, the redundant data packets are cleared and the corresponding instructions are sent to the sending end according to the situation that the redundant data length is greater than the preset redundant length, so that the sending end adjusts the data code stream according to the corresponding instructions, and then sends the processed data to the receiving end. If it is less than the preset redundant length, the remaining length of the receiving end container and the current first data information are returned to the sending end. Through this method, the client is notified of the dynamic change of the data code stream through the data storage margin, which can improve the current high latency problem caused by data packet redundancy and improve the stability of screen projection.

[0078] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for adjusting screen projection code stream, characterized in that: include: Receive first data sent by a transmitting end, store the first data into a receiving end container and obtain a current first data length and current first data information, wherein the first data information includes a current first data frame width, a current first data code stream, and a current first data frame rate; Obtaining a current first data redundant length and a remaining length of a receiving-end container according to the current first data length and a total length of a receiving-end container; Determine whether the redundant length of the current first data is greater than a preset redundant length; if so, clear the redundant data of the current first data and send the remaining length of the receiving-end container to the sending end, so that the sending end performs code stream adjustment on the first data according to the remaining length of the receiving-end container to obtain preprocessed data, and then sends the preprocessed data to the receiving end; if not, return the remaining length of the receiving-end container and the current first data information to the sending end.

2. The screen projection code stream adjustment method according to claim 1, characterized in that: The receiving end sends the first data, stores the first data in the receiving end container and obtains the current first data length and the current first data information, specifically: The first data is sent by the sending end through the TCP connection communication receiving end; The first data is stored in a storage container using a queue and a current first data redundancy length value and current first data information are obtained.

3. The screen projection code stream adjustment method according to claim 1, characterized in that: The determining whether the redundant length of the current first data is greater than the preset redundant length, and if so, clearing the redundant data of the current first data and sending the remaining length of the receiving end container to the sending end is specifically: If the redundant length of the current first data is greater than the first preset redundant length, the redundant data of the current first data is cleared and the first remaining length of the receiving end container is sent to the sending end; If the redundant length of the current first data is greater than the second preset redundant length, the redundant data of the current first data is cleared and the second remaining length of the receiving end container is sent to the sending end.

4. The screen projection code stream adjustment method according to claim 1, characterized in that: After the transmitting end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, the pre-processed data is sent to the receiving end, specifically: If the sending end receives the first remaining length of the container at the receiving end, reducing the bitstream of the first data to obtain preprocessed first data and sending the preprocessed first data to the receiving end; If the sending end receives the second remaining length of the container at the receiving end, the sending end generates an I frame and carries the first data of the SPS and PPS information as the first detection data, and sends the first detection data to the receiving end.

5. A screen projection code stream adjustment system, characterized in that: include: A data receiving module, configured to receive first data sent by a transmitting end, store the first data into a receiving end container and obtain a current first data length and current first data information, wherein the first data information includes a current first data frame width, a current first data code stream, and a current first data frame rate; A data calculation module, used for obtaining a current first data redundant length and a remaining length of a receiving end container according to the current first data length and a total length of a receiving end container; The data judgment module is used to judge whether the redundant length of the current first data is greater than the preset redundant length. If so, the redundant data of the current first data is cleared and the remaining length of the receiving end container is sent to the sending end, so that the sending end performs code stream adjustment on the first data according to the remaining length of the receiving end container to obtain pre-processed data, and then sends the pre-processed data to the receiving end. If not, the remaining length of the receiving end container and the current first data information are returned to the sending end.

6. The screen projection code stream adjustment system according to claim 5, characterized in that: The data receiving module includes a receiving unit and a calculating unit; Wherein, the receiving unit is used to receive the first data sent by the sending end by using TCP to establish a connection communication; The calculation unit is used for storing the first data in a storage container using a queue and obtaining a current first data redundant length and current first data information.

7. The screen projection code stream adjustment system according to claim 5, characterized in that: The data judgment module includes a processing unit and a control unit; The processing unit is configured to, if the current first data redundant length is greater than a first preset redundant length, clear the redundant data of the current first data and send a first remaining length of the receiving end container to the sending end; and if the current first data redundant length is greater than a second preset redundant length, clear the redundant data of the current first data and send a second remaining length of the receiving end container to the sending end; The control unit is used for reducing the code stream of the first data to obtain preprocessed first data and sending the preprocessed first data to the receiving end if the transmitting end receives the first remaining length of the receiving end container; if the transmitting end receives the second remaining length of the receiving end container, the transmitting end generates an I frame and the first data carrying SPS and PPS information is the first detection data, and sends the first detection data to the receiving end.

8. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for adjusting the screen projection code stream as described in any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the screen projection code stream adjustment method as described in any one of claims 1-4.

Images