A network transmission method and apparatus based on screen sharing

Abstract

This application discloses a network transmission method and apparatus based on screen sharing. The method includes: sending a screen sharing start command from a presenter to all receivers through a first connection channel, thereby connecting the receivers and a server through a second connection channel; transmitting data packets encoded by the presenter to the receivers through the second connection channel; comparing the frame sequence number of the data packets with the first frame sequence number of the receivers to obtain network latency; and adjusting the data transmission mode of the second connection channel and the data encoding mode of the presenter based on the network latency. This application dynamically adjusts the data transmission mode according to the current network conditions during screen sharing, reducing data transmission latency and transmitting high-definition image data as much as possible, thus improving the smoothness and quality of screen sharing.

Description

Technical Field

[0001] This application belongs to the field of screen sharing technology, specifically relating to a network transmission method and apparatus based on screen sharing. Background Technology

[0002] In online meetings, screen sharing allows a presenter to transmit their screen to other receivers on the same local area network, enabling one-to-many information transmission. During this one-to-many transmission, it's crucial to ensure synchronization between the presenter's and receiver's displays to achieve low latency, while simultaneously providing the highest possible display clarity.

[0003] Existing methods for one-to-many screen sharing within the same local area network involve calling the system's screen capture API to encode the captured screen image into h.264 format and then sending the data byte stream to a streaming media server via a TCP socket connection. The streaming media server then forwards the video data one-to-one to all other receiving terminals via TCP socket connections. However, since the receiving terminals rely on Wi-Fi for network transmission, network congestion can occur when there are too many devices participating in screen sharing. This leads to data packet loss, increased latency, and consequently, varying degrees of screen asynchrony, stuttering, and screen distortion. Summary of the Invention

[0004] This application proposes a network transmission method and apparatus based on screen sharing. When sharing the screen, the data transmission mode is dynamically adjusted according to the current network conditions, which reduces data transmission latency and transmits high-definition image data as much as possible, thereby improving the smoothness and quality of screen sharing.

[0005] A first aspect of this application provides a network transmission method based on screen sharing, the method comprising:

[0006] The screen sharing enable command from the presenter is sent to all receivers through the first connection channel, so that the receivers and the server can connect through the second connection channel.

[0007] The data packets encoded at the presenter end are transmitted to the receiver end through the second link channel;

[0008] The network latency is obtained by comparing the frame sequence number of the data packet with the first frame sequence number of the receiving end; wherein, the first frame sequence number is the frame sequence number of the previous data packet received by the receiving end.

[0009] Adjust the data transmission method of the second link channel and the data encoding method of the main speaker based on the network latency.

[0010] The above solution establishes a first connection channel between the presenter and multiple receivers via a server. A screen sharing enable command is sent to each receiver through this first connection channel, enabling the receiver to connect to a second connection channel to receive data packets sent by the presenter. The command is transmitted through the first connection channel, and the data is transmitted through the second connection channel, ensuring network transmission consistency. The frame sequence number of the data packet is compared with the first frame sequence number of the receiver to determine if packet loss has occurred, thus obtaining network latency information. Based on the network latency, the data transmission method is dynamically adjusted to mitigate the severity of network latency and reduce data transmission delay. Then, the data encoding method of the presenter is dynamically adjusted, and by changing the image quality and frame rate, the probability of screen stuttering is reduced, improving the smoothness and quality of screen sharing.

[0011] In one possible implementation of the first aspect, the screen sharing enable command from the presenter is sent to all receivers via a first connection channel, thereby enabling the receivers and the server to connect via a second connection channel. Specifically:

[0012] The time is synchronized between the presenter and the server through the first link channel;

[0013] The screen sharing enable command of the main terminal is sent to all receiving terminals through the first connection channel, and it is determined whether the receiving terminal is the broadcast user based on the screen sharing enable command.

[0014] A second link channel is established between the receiving end, which is identified as the broadcasting user, and the server, and the data transmission mode of the second link channel is set to UDP multicast mode.

[0015] The above scheme first synchronizes the time of the presenter and the server to ensure that their time sources are synchronized, thereby improving the accuracy and reliability of network transmission. Then, it connects the receiving end designated as the broadcast user to the server through a second link channel, ensuring that all invited receiving ends participating in screen sharing can receive the screen data. Receiving ends not invited to participate in screen sharing will not connect to the server for data transmission, reducing the waste of network resources. The data transmission mode of the second link channel is set to UDP multicast mode to ensure low latency in network transmission.

[0016] In one possible implementation of the first aspect, time synchronization is performed between the presenter and the server via a first link channel, specifically as follows:

[0017] The time acquisition request from the main speaker is sent to the server at a preset synchronization frequency to obtain the server's system time;

[0018] Set the server's system time to the presenter's system time to achieve time synchronization between the presenter and the server.

[0019] The above method performs time synchronization between the presenter and the server at a preset synchronization frequency to ensure that the time sources of the presenter and the server are always consistent, thereby reducing the error of calculation delay.

[0020] In one possible implementation of the first aspect, the frame sequence number of the data packet is compared with the first frame sequence number of the receiving end to obtain the network latency information, specifically as follows:

[0021] The data packet is decoded to obtain the frame sequence number and frame output time of the data packet;

[0022] The frame sequence number is compared with the first frame sequence number of the receiving end to obtain the comparison result;

[0023] If the comparison result meets the preset packet loss conditions, the network latency is obtained, and decoding of the data packet is stopped;

[0024] If the comparison result meets the preset decoding conditions, the data packet is decoded and the image is rendered, and the first frame number is updated to the frame number.

[0025] The above scheme compares the frame sequence number with the first frame sequence number of the receiving end to determine whether packet loss has occurred. If packet loss has occurred, decoding of the data packet is stopped and network latency is obtained to analyze how to deal with packet loss and solve the stuttering problem of screen sharing. If no packet loss has occurred, the data packet is decoded and the screen is rendered to achieve screen sharing, and the first frame sequence number is updated to the frame sequence number for the next packet loss detection.

[0026] In one possible implementation of the first aspect, if the comparison result meets a preset packet loss condition, the network latency is obtained, and decoding of the data packet is stopped, specifically:

[0027] If the comparison result is that the first frame number is equal to the first threshold and the frame number is not equal to 0, or if the comparison result is that the first frame number is not equal to the first threshold and the result of subtracting the first frame number from the frame number is not equal to 1, then it is determined that the comparison result meets the preset packet loss condition, and the decoding of the data packet is stopped.

[0028] A packet loss signal is sent to the presenter through the first link channel to obtain the latency statistics of the presenter. The data packet containing the preset key frame is then transmitted to the receiver through the second link channel so that the receiver can restart decoding and image rendering.

[0029] Based on the latency statistics, the network latency situation is obtained.

[0030] The above scheme first determines whether packet loss has occurred based on preset packet loss conditions, and sends a packet loss signal to the main speaker through the first connection channel to obtain the latency statistics of the main speaker. The receiving end only restarts decoding and image rendering after receiving a data packet containing keyframes, thus avoiding image stuttering caused by decoding lost data packets. Moreover, the network latency is obtained based on the latency statistics, providing data support for subsequent judgment of the severity of network latency.

[0031] In one possible implementation of the first aspect, a packet loss signal is sent to the presenter via a first link channel to obtain the latency statistics of the presenter, specifically as follows:

[0032] After sending a packet loss signal to the main speaker through the first link channel, the total number of times the main speaker receives the packet loss signal within a preset time is calculated and recorded as the total number of packet losses.

[0033] The delay information is obtained by calculating the frame output time of the data packet and the system time of the main speaker.

[0034] The delay statistics are obtained based on the total number of packet losses and the delay information.

[0035] In one possible implementation of the first aspect, the data transmission method of the second link channel and the data encoding method of the presenter are adjusted according to the network latency, specifically as follows:

[0036] Based on the network latency, the total number of packet losses and the latency level are obtained; wherein, the total number of packet losses is the total number of times the main terminal receives the packet loss signal within a preset time.

[0037] If the total number of packet losses exceeds the second threshold, the data transmission mode of the second connection channel will be set to TCP forwarding mode.

[0038] If the latency level is the first level, then reduce the video resolution and frame rate used for encoding at the main speaker.

[0039] If the latency level is level two, then the video resolution and frame rate used for encoding at the main speaker are increased.

[0040] The above scheme analyzes network latency to obtain the total number of packet losses and the latency level. When the total number of packet losses exceeds the second threshold, it indicates excessive packet loss, and the data transmission mode of the second connection channel needs to be set to TCP forwarding mode to reduce the packet loss rate and improve network transmission stability. If the latency level is the first level, it indicates severe latency, and the video resolution and frame rate used for encoding at the main speaking end need to be reduced to decrease network transmission volume and avoid further network latency caused by network congestion. If the latency level is the second level, it indicates that there is no abnormality in latency, and the video resolution and frame rate used for encoding at the main speaking end can be increased to provide a higher definition image for the receiving end.

[0041] In one possible implementation of the first aspect, the latency level is obtained based on the network latency situation, specifically as follows:

[0042] Based on network latency, identify the receiving end that sent the packet loss signal;

[0043] Based on the delay duration of the receivers that sent packet loss signals, the delay percentage of all receivers is obtained.

[0044] Based on the stated delay percentage, the delay level is determined.

[0045] In one possible implementation of the first aspect, after setting the data transmission mode of the second connection channel to TCP forwarding mode, if the main speaker does not receive a packet loss signal within a preset judgment time, the data transmission mode of the second connection channel is set to UDP multicast mode; if the number of failures in setting to UDP multicast mode exceeds a third threshold, the data transmission mode of the second connection channel will remain set to TCP forwarding mode until the current screen sharing ends.

[0046] In the above scheme, if no packet loss signal is received for a long time, it means that the network transmission is stable and there is no packet loss. The data transmission mode of the second connection channel can be switched to UDP multicast mode to reduce network transmission latency. However, if the number of failures to switch to UDP multicast mode is too many, it means that the current network conditions do not support UDP multicast mode with a high packet loss rate. The data transmission mode of the second connection channel can only be set to TCP forwarding mode to ensure that the network transmission during the screen sharing process can remain stable.

[0047] A second aspect of this application provides a network transmission device based on screen sharing, the device comprising: a screen sharing link establishment module, a data packet transmission module, a network latency generation module, and a network transmission mode adjustment module;

[0048] The screen sharing link establishment module is used to send the screen sharing start command of the presenter to all receivers through the first link channel, so that the receivers and the server can connect through the second link channel.

[0049] The data packet transmission module is used to transmit the data packet encoded by the presenter to the receiver through the second link channel;

[0050] The network latency generation module is used to compare the frame sequence number of the data packet with the first frame sequence number of the receiving end to obtain the network latency information; wherein, the first frame sequence number is the frame sequence number of the previous data packet received by the receiving end.

[0051] The network transmission mode adjustment module is used to adjust the data transmission mode of the second link channel and the data encoding mode of the main speaker terminal according to the network latency. Attached Figure Description

[0052] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a schematic flowchart illustrating a network transmission method based on screen sharing according to a certain embodiment of this application;

[0054] Figure 2 This is a schematic diagram illustrating the specific process of a network transmission device based on screen sharing, provided in a certain embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0056] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0057] Example 1

[0058] During screen sharing in online meetings, maintaining low network latency is crucial to ensure synchronization between the presentation screen and the displays of multiple receivers. However, when too many receivers are involved in screen sharing, the large volume of data transmitted over the network increases the likelihood of packet loss and increased latency. Therefore, adjusting data transmission methods in real-time based on current network conditions to strike a balance between low latency and high-definition display is worth exploring.

[0059] like Figure 1 As shown, Figure 1 This application provides a schematic flowchart of a network transmission method based on screen sharing according to a certain embodiment. The network transmission method based on screen sharing in this embodiment includes steps S1 to S4, which are described in detail below:

[0060] Step S1: Send the screen sharing enable command from the main speaker to all receiving ends through the first link channel, so that the receiving ends and the server can connect through the second link channel.

[0061] In this step, screen sharing is based on an Android app, but in some embodiments, screen sharing can be based on other software, such as a PC app.

[0062] After the Android app starts, it will provide two connection channels to the server: the first connection channel and the second connection channel, and both of these connection channels are based on TCP Socket.

[0063] The first connection channel is used for sending and receiving standard protocols, including login, enabling / disabling screen sharing, and other control commands. The second connection channel is used for sending streaming media data, such as captured screen image data.

[0064] Preferably, the second link channel includes two data transmission methods. One is TCP forwarding, which is used to forward the screen display data transmitted to the host terminal of the server to the receiving terminal via TCP Socket; the other is UDP multicast, which is used to broadcast the screen display data transmitted to the host terminal of the server to the receiving terminal via UDP multicast.

[0065] Compared to TCP forwarding, TCP forwarding offers more stable network transmission with a lower packet loss rate, but it also results in higher latency. UDP multicast, on the other hand, benefits from the characteristics of the UDP protocol, offering lower latency than TCP forwarding, but it is less stable and has a higher packet loss rate.

[0066] It is worth mentioning that during the process of forwarding data packets, the server will only choose to use either TCP forwarding or UDP multicast, and will not use both methods simultaneously.

[0067] The presenter first establishes a first connection channel and a second connection channel with the server. Then, the presenter first sends a time retrieval request to the server through the first connection channel, and the server returns its own system time to the presenter. After obtaining the server's system time, the presenter sets the server's system time to the presenter's system time, thus achieving the first time synchronization between the presenter and the server. After completing the first time synchronization, the presenter sends a time retrieval request to the server every 10 to 30 minutes to resynchronize the time, ensuring that the presenter and the server's time sources are consistent and reducing calculation delay errors.

[0068] The frequency with which the presenter sends a time retrieval request to the server gradually decreases from the first time time synchronization begins. This is because network instability may occur during the initial connection establishment period after screen sharing, necessitating more frequent time synchronization. For example, a time retrieval request is sent after the first 10 minutes of screen sharing. The interval is then adjusted to 15 minutes, and synchronization resumes after another 15-minute interval. This interval is then further extended to 20 minutes before synchronizing again. Each increment is 5 minutes, with a maximum fixed interval of 30 minutes.

[0069] After time synchronization is completed, the server establishes a first connection channel with all receiving ends. Then, the screen sharing start command from the presenter is sent to the server through the first connection channel. The server then forwards the command to all receiving ends through the first connection channel. At the same time, the presenter also informs the server through the screen sharing start command that the data transmission mode of the second connection channel should be set to UDP multicast mode, which brings lower latency, server pressure and bandwidth usage to the initial network data transmission.

[0070] After receiving the screen sharing enable command, all receiving ends use the ID of the broadcast user included in the command to confirm whether they are the broadcast user. A second connection channel is established between the receiving end identified as the broadcast user and the server for subsequent data packet reception.

[0071] Step S2: Transmit the data packet encoded by the presenter to the receiver through the second link channel.

[0072] In this step, the presenter first captures the screen content and encodes it into an h263 format byte stream. During encoding, only I-frames and P-frames need to be encoded; B-frames do not need to be encoded. During decoding, only I-frames are needed to reconstruct the complete image; therefore, I-frames are also keyframes and are the preset keyframes in this application. Furthermore, if screen sharing is based on an Android app, the Android system's MediaProjection capability is used for screen content capture, typically using 720p30 frames per second for encoding by default.

[0073] A byte stream is generated for each frame, resulting in a corresponding data packet. The header of this data packet needs to include an additional byte stream of customizable length (typically 28 bytes). This header contains the host ID, the second link channel ID, the system time (accurate to milliseconds) at the time of frame capture, the length of the current frame's main byte stream, the current frame sequence number, and the current frame's output time. The main body of the data packet is the byte stream of the current frame, which can be used for decoding and rendering the image.

[0074] Before the transmitting end sends a data packet, it determines whether the data packet is an I-frame. If it is an I-frame, a list is used to record the frame sequence number corresponding to the data packet. When the frame sequence number in the list reaches a preset maximum threshold, the list is cleared and the recording of frame sequence numbers starts again.

[0075] Finally, the presenter transmits the data packet to the server through the second link channel, and the server then forwards the data packet to the receiver.

[0076] Step S3: Compare the frame sequence number of the data packet with the first frame sequence number of the receiving end to obtain the network latency information.

[0077] In this step, the receiving end decodes the header of the data packet to obtain the frame sequence number and frame output time of the data packet.

[0078] The frame sequence number is 2 bytes long, unsigned, and ranges from 0 to 65535. When it reaches 65535, the next frame sequence number is reset to 0. The frame output time is the system time encoded and output by the presenter.

[0079] In each receiving end, there is a variable named "First Frame Sequence Number" to record the frame sequence number of the previous data packet received by that receiving end. The process of comparing this frame sequence number with the first frame sequence number is as follows:

[0080] (1) If the sequence number of the first frame is 65535, then proceed to (2); otherwise proceed to (3).

[0081] (2) If the frame number is 0, the comparison result meets the preset decoding conditions; otherwise, the comparison result meets the preset packet loss conditions.

[0082] (3) If the result of subtracting the first frame number from the frame number is equal to 1, then the comparison result meets the preset decoding condition; otherwise, the comparison result meets the preset packet loss condition.

[0083] If the comparison result meets the preset packet loss conditions, the receiving end sends a packet loss signal to the speaking end through the first link channel to inform the speaking end that packet loss has occurred at the receiving end. The packet loss signal also includes the frame sequence number. At the same time, the receiving end stops decoding the packet body of the data packet.

[0084] After the packet loss signal arrives at the presenter, the presenter checks the list set to determine if an I-frame sequence number exists. If an I-frame sequence number exists, it means that the receiver has already adjusted to the next I-frame data packet's frame group. To prevent frame stuttering at the receiver, the presenter will not resend the latest output I-frame data packet, but will continue to send the next encoded data packet. If no I-frame sequence number exists, the presenter re-acquires and encodes the most recently output I-frame data packet and transmits it to the receiver through the second link channel.

[0085] Encoding and rendering only resume after the receiving end receives a data packet that is an I-frame. Although this will cause screen delay and pause at the receiving end, it can prevent the video from being distorted.

[0086] After sending a packet loss signal to the presenter via the first link channel, the total number of times the presenter receives the packet loss signal within a preset time is calculated and recorded as the total number of packet losses. Then, screen delay statistics are performed on the presenter by calculating the frame output time of the data packet and the system time of the presenter to obtain delay information, the unit of which is milliseconds.

[0087] Based on the total number of packet losses and latency information, the latency statistics are obtained, and then the network latency is determined based on the latency statistics.

[0088] If the comparison result meets the preset decoding conditions, the data packet is decoded and the image is rendered, and the first frame sequence number is updated to the frame sequence number for the next packet loss detection.

[0089] Step S4: Adjust the data transmission method of the second link channel and the data encoding method of the main speaker according to the network latency.

[0090] In this step, based on network latency, the receivers that sent packet loss signals within 3 seconds are identified. Then, based on a configured latency interval constant, the latency duration of the receivers that sent packet loss signals is analyzed to obtain the latency percentage of all receivers. The latency interval constant includes low latency intervals, medium latency intervals, and high latency intervals.

[0091] Then, based on the aforementioned delay ratio, the proportion of receivers in the low-delay, medium-delay, and high-delay ranges is calculated to obtain the delay level.

[0092] Then, based on the network latency, the total number of packet losses is obtained.

[0093] If the total number of packet losses exceeds the second threshold, the data transmission mode of the second connection channel is set to TCP forwarding. After setting the data transmission mode of the second connection channel to TCP forwarding, if the presenter does not receive a packet loss signal within a preset judgment time, the data transmission mode of the second connection channel is set to UDP multicast. If the number of failed attempts to set it to UDP multicast exceeds a third threshold, the data transmission mode of the second connection channel will remain set to TCP forwarding until the current screen sharing ends. In some embodiments, the total number of packet losses is generally the total number of times the presenter receives the packet loss signal within 5 seconds; the preset judgment time is generally 30 seconds, and the third threshold is generally set to 3 times.

[0094] If the latency level is Level 1, it indicates that the network latency is relatively severe. It is necessary to reduce the video resolution and frame rate used for encoding on the main end to reduce the data transmission volume of the second link channel and the possibility of network congestion, thereby reducing network latency.

[0095] If the latency level is Level 2, it indicates that the network latency is not severe. The video resolution and frame rate used for encoding on the main speaker can be gradually increased. Under the condition that the network latency is not abnormal, high-definition images should be provided to the receiving end as much as possible to improve the quality of screen sharing.

[0096] When screen sharing is ready to end, the presenter sends a screen sharing end command to the receiver through the first link channel. Upon receiving the screen sharing end command, all receivers stop encoding and rendering, and screen sharing ends.

[0097] Example 2

[0098] Furthermore, in order to execute the network transmission device based on screen sharing corresponding to the above method embodiments, and to achieve the corresponding functions and technical effects, Figure 2A specific structural diagram of a network transmission device based on screen sharing is provided. For ease of explanation, only the parts relevant to this embodiment are shown. The network transmission device based on screen sharing provided in this application embodiment includes:

[0099] The screen sharing link establishment module 201 is used to send the screen sharing start command of the presenter to all receivers through the first link channel, so that the receivers and the server can connect through the second link channel.

[0100] The data packet transmission module 202 is used to transmit the data packet encoded by the presenter to the receiver through the second link channel;

[0101] The network latency generation module 203 is used to compare the frame sequence number of the data packet with the first frame sequence number of the receiving end to obtain the network latency information; wherein, the first frame sequence number is the frame sequence number of the previous data packet received by the receiving end.

[0102] The network transmission mode adjustment module 204 is used to adjust the data transmission mode of the second link channel and the data encoding mode of the main speaker according to the network latency.

[0103] In some embodiments, the screen sharing link establishment module 201 further includes:

[0104] The data transmission preparation unit is used to synchronize the time between the presenter and the server through a first link channel; send the screen sharing enable command of the presenter to all receiving ends through the first link channel, and determine whether the receiving end is a broadcast user based on the screen sharing enable command; connect the receiving end determined to be a broadcast user to the server through a second link channel, and set the data transmission mode of the second link channel to UDP multicast mode.

[0105] The presenter time synchronization unit is used to send the presenter's time acquisition request to the server at a preset synchronization frequency to obtain the server's system time; and set the server's system time to the presenter's system time to achieve time synchronization between the presenter and the server.

[0106] In some embodiments, the network latency generation module 203 further includes:

[0107] The packet loss judgment unit is used to decode the data packet to obtain the frame sequence number and frame output time of the data packet; compare the frame sequence number with the first frame sequence number of the receiving end to obtain the comparison result. If the comparison result meets the preset packet loss conditions, the network latency situation is obtained, and the decoding of the data packet is stopped; if the comparison result meets the preset decoding conditions, the data packet is decoded and the image is rendered, and the first frame sequence number is updated to the frame sequence number.

[0108] A network latency generation unit is configured to determine that the comparison result meets a preset packet loss condition and stop decoding the data packet if the comparison result is that the first frame number is equal to a first threshold and the frame number is not equal to 0, or the comparison result is that the first frame number is not equal to the first threshold and the result of subtracting the first frame number from the frame number is not equal to 1; send a packet loss signal to the presenter through a first link channel to obtain the latency statistics of the presenter; and transmit the data packet containing the re-acquired preset key frame to the receiver through a second link channel so that the receiver can restart decoding and image rendering; and obtain the network latency status based on the latency statistics.

[0109] The delay statistics result generation unit is used to calculate the total number of times the main speaker receives the packet loss signal within a preset time after sending the packet loss signal to the main speaker through the first link channel, and record it as the total number of packet losses; calculate the frame output time of the data packet and the system time of the main speaker to obtain delay information; and obtain the delay statistics result based on the total number of packet losses and the delay information.

[0110] In some embodiments, the network transmission mode adjustment module 204 further includes:

[0111] The data transmission and encoding adjustment unit is used to obtain the total number of packet losses and the latency level based on network latency conditions. The total number of packet losses is the total number of times the presenter receives the packet loss signal within a preset time. If the total number of losses exceeds a second threshold, the data transmission mode of the second connection channel is set to TCP forwarding mode. If the latency level is at the first level, the video resolution and frame rate used for encoding at the presenter are reduced. If the latency level is at the second level, the video resolution and frame rate used for encoding at the presenter are increased.

[0112] The delay status level determination unit is used to determine the receiving end that has sent a packet loss signal based on the network delay status; to obtain the delay percentage of all receiving ends based on the delay duration of the receiving end that sent the packet loss signal; and to obtain the delay status level based on the delay percentage.

[0113] The data transmission mode switching unit is used to set the data transmission mode of the second connection channel to TCP forwarding mode. If the main speaker does not receive a packet loss signal within a preset judgment time, the data transmission mode of the second connection channel is set to UDP multicast mode. If the number of failures in setting to UDP multicast mode exceeds a third threshold, the data transmission mode of the second connection channel will remain set to TCP forwarding mode until the current screen sharing ends.

[0114] This application proposes a network transmission method and apparatus based on screen sharing: A screen sharing activation command from the presenter is sent to all receivers via a first connection channel, enabling the receivers and server to connect via a second connection channel; a data packet encoded by the presenter is transmitted to the receivers via the second connection channel; the frame sequence number of the data packet is compared with the first frame sequence number of the receiver to obtain network latency; wherein the first frame sequence number is the frame sequence number of the previous data packet received by the receiver; and the data transmission mode of the second connection channel and the data encoding mode of the presenter are adjusted according to the network latency. Its advantages are: during screen sharing, the data transmission mode is dynamically adjusted according to the current network conditions, reducing data transmission latency and transmitting high-definition image data as much as possible, thus improving the smoothness and quality of screen sharing.

[0115] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, or improvements made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A network transmission method based on screen sharing, characterized in that, The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a screen sharing process. The application relates to a method for adjusting a data transmission mode and a data encoding mode of a host terminal in a The delay statistical result of the host is obtained by sending a packet loss signal to the host through the first link channel, and specifically: After sending the packet loss signal to the host through the first link channel, the total number of times that the host receives the packet loss signal within a preset time is calculated, which is recorded as the total number of packet losses; The frame output time of the data packet and the system time of the host are calculated to obtain the delay information; According to the total number of packet losses and the delay information, the delay statistical result is obtained.

2. The screen sharing based network transmission method of claim 1, wherein, The screen sharing opening instruction of the host is sent to all receiving ends through the first link channel, so that the receiving ends and the server are connected through the second link channel, and specifically: Through the first link channel, the host and the server are time-synchronized; The screen sharing opening instruction of the host is sent to all receiving ends through the first link channel, and it is judged whether the receiving end is a broadcast user according to the screen sharing opening instruction; The receiving end judged as a broadcast user and the server are connected through the second link channel, and the data transmission mode of the second link channel is set to the UDP multicast mode.

3. The screen sharing based network transmission method of claim 2, wherein, Through the first link channel, the host and the server are time-synchronized, and specifically: The host's request for obtaining time is sent to the server at a preset synchronization frequency to obtain the system time of the server; The system time of the server is set as the system time of the host to realize time synchronization between the host and the server.

4. The screen sharing based network transmission method of claim 1, wherein, According to the network delay condition, the delay condition level is obtained, and specifically: According to the network delay condition, the receiving end that has sent a packet loss signal is determined; According to the delay duration of the receiving end that has sent a packet loss signal, the delay proportion of all receiving ends is obtained; According to the delay proportion, the delay condition level is obtained.

5. A network transmission device based on screen sharing, characterized in that, It includes: A screen sharing link establishment module, a data packet transmission module, a network delay condition generation module and a network transmission mode adjustment module; The screen sharing link establishment module is used to send the screen sharing opening instruction of the host to all receiving ends through the first link channel, so that the receiving ends and the server are connected through the second link channel; The data packet transmission module is used to transmit the data packet encoded by the host to the receiving end through the second link channel; The network delay condition generation module is used to compare the frame sequence number of the data packet with the first frame sequence number of the receiving end to obtain the network delay condition; wherein the first frame sequence number is the frame sequence number of the last data packet received by the receiving end; The network transmission mode adjusting module is configured to adjust the data transmission mode of the second link channel and the data encoding mode of the host according to the network delay condition, specifically: obtaining the total number of packet loss and the delay condition level according to the network delay condition; wherein the total number of packet loss is the total number of times that the host receives a packet loss signal within a preset time; if the total number of packet loss is greater than a second threshold, the data transmission mode of the second link channel is set to a TCP forwarding mode; if the delay condition level is a first level, the video resolution and frame rate used for encoding by the host are reduced; if the delay condition level is a second level, the video resolution and frame rate used for encoding by the host are increased; Wherein, after setting the data transmission mode of the second link channel to the TCP forwarding mode, if the host does not receive a packet loss signal within a preset judgment time, the data transmission mode of the second link channel is set to a UDP multicast mode; if the number of times of setting to the UDP multicast mode exceeds a third threshold, the data transmission mode of the second link channel will always be set to the TCP forwarding mode until the end of this screen sharing; The frame sequence number of the data packet and the first frame sequence number of the receiving end are compared to obtain the network delay condition, specifically: The data packet is decoded to obtain the frame sequence number of the data packet and the frame output time; The frame sequence number and the first frame sequence number of the receiving end are compared to obtain a comparison result; If the comparison result meets the preset packet loss condition, the network delay condition is obtained, and decoding of the data packet is stopped; If the comparison result meets the preset decoding condition, the data packet is decoded and picture rendering, and the first frame sequence number is updated to the frame sequence number; If the comparison result is that the first frame sequence number is equal to a first threshold and the frame sequence number is not equal to 0, or the comparison result is that the first frame sequence number is not equal to a first threshold and the result of the frame sequence number minus the first frame sequence number is not equal to 1, it is determined that the comparison result meets the preset packet loss condition, and decoding of the data packet is stopped; A packet loss signal is sent to the host through the first link channel to obtain the delay statistical result of the host, and a data packet containing a preset key frame is transmitted to the receiving end through the second link channel, so that the receiving end restarts decoding and picture rendering; The network delay condition is obtained according to the delay statistical result; The delay statistical result of the host is obtained by sending a packet loss signal to the host through the first link channel, specifically: After sending a packet loss signal to the host through the first link channel, the total number of times that the host receives the packet loss signal within a preset time is calculated, which is recorded as the total number of packet loss; The frame output time of the data packet and the system time of the host are calculated to obtain delay information; ​ According to the total number of packet loss and delay information, the delay statistical result is obtained.

Images