A weak network transmission method, system, device and medium of a streaming media server

Abstract

The present application relates to the field of data transmission, and specifically relates to a weak network transmission method, system, device and medium of a streaming media server, the method comprising: reading media data to be sent in the streaming media server and dividing the media data into a plurality of GOP sequences, and encapsulating each frame of the media data into an RTP data packet; sending the RTP data packet in the current GOP sequence to a playing end and counting the sending state of the RTP data packet; calculating the available network bandwidth according to the sending state and determining the number of frames to be sent in the next GOP sequence. The present application independently calculates the available network bandwidth from the streaming media server end, overcomes the dependence on the code stream information feedback of the playing end, and determines the number of frames to be sent in the next GOP sequence according to the calculated available network bandwidth, so as to realize the real-time adjustment of the transmission code rate of the media data, and ensure the adaptive ability of the streaming media server in the weak network environment without code stream retransmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data transmission, in particular to a weak network transmission method of a streaming media server. BACKGROUND

[0002] The audio and video streaming media server plays an important role in the whole video monitoring system as the core part of forwarding audio and video streaming media. The actual deployment network environment of the video monitoring system often has limited bandwidth, more access devices, more playback ends and the like, so the streaming media server is required to have the ability to forward in a weak network environment to a certain extent, that is, the weak network transmission capability. The weak network transmission strategy has different implementations in various open source streaming media frameworks, such as the strategy of playing back based on NACK (Negative Acknowledgment) feedback, calculating the available bandwidth and available frame rate, adjusting the code rate of the encoding end, and retransmitting. For example, the strategy of sending after buffering the RTP data packet, and re-requesting the server to retransmit the data packet when the playback end detects a packet loss. In general, the core transmission strategy of the existing streaming media server against the weak network environment can be summarized as follows: the strategy of playing back based on packet loss feedback, the server re-adjusting the code rate, and retransmitting the data packet.

[0003] However, the existing weak network transmission strategy of the streaming media server has two technical problems: first, it relies on the feedback of the playback end on the code stream information to calculate the network condition, but ignores the fact that in the case of poor network, there is a certain packet loss rate in the data transmission between the playback end and the streaming media server itself, that is, when the playback end feeds back the received code stream information (media data information) to the server under the weak network condition, there may be a problem of packet loss or excessive delay, so the server cannot accurately obtain the receiving information of the playback end in real time; second, the method of code stream retransmission is used to improve the adaptability of the streaming media server in the weak network environment, that is, the strategy of the streaming media server retransmitting the missing media data to the playback end, but the code stream retransmission requires the streaming media server to cache a large amount of media data before transmission, resulting in a large consumption of memory resources of the streaming media server, and the worse the network condition, the more the cache, the greater the consumption of server memory capacity. SUMMARY

[0004] The existing weak network transmission method of the streaming media server has the above technical problems, and therefore, the present application creatively proposes a weak network transmission method, system, device and medium of a streaming media server, aiming at solving the above technical problems.

[0005] The present application adopts the following technical scheme: a weak network transmission method of a streaming media server, comprising the following steps:

[0006] Step 101, reading media data to be sent in a streaming media server, dividing the media data to be sent into several groups of video frames, each group of video frames being recorded as a GOP sequence, and performing slicing and encapsulating each frame of the media data into an RTP data packet;

[0007] Step 102, sending the RTP data packet in the current GOP sequence to a playing end and counting the sending state of the RTP data packet;

[0008] Step 103, calculating available network bandwidth according to the sending state;

[0009] Step 104, determining the number of frames to be sent in the next GOP sequence according to the available network bandwidth, and sending the next GOP sequence according to the number of frames to be sent.

[0010] Wherein, GOP (Group of Pictures) is a concept in video coding, which defines the structure and coding method of video sequences. In video coding, video frames are divided into different groups, each group being called a GOP. RTP (Real-time Transport Protocol) is a network protocol for real-time transmission of audio and video data, which is widely used in real-time communication applications such as VoIP (Voice over IP) and real-time video streaming. RTP protocol provides a standardized way to segment, transmit and reassemble audio and video data, which is an end-to-end protocol for transmitting real-time data between sender and receiver. The main function of RTP is to segment data, that is, RTP divides the audio or video data stream into small data packets, which are called RTP data packets.

[0011] The technical concept of the present application is that, in the process that the streaming media server sends the required media data to the playing end and the playing end plays the media data in real time under a weak network environment, first, the media data to be sent is read from the source media data of the streaming media server, the media data to be sent is divided into several groups of video frames, each group of video frames is recorded as a GOP sequence, and each frame of the media data is sliced and packaged into an RTP data packet, that is, a GOP sequence can be split into multiple RTP data packets for transmission, the RTP data packet is used as a transmission unit for packaging the media data, and the GOP sequence is used as a statistical unit for the sending state of the media data; secondly, considering the delay of the packaging and transmission of the media data to be sent, a method of sending the RTP data packet at a fixed time is adopted, that is, the streaming media server is set to try to send the RTP data packet to the playing end several times per second, one RTP data packet is sent each time, and the sending state of the RTP data packet is counted; then, the available network bandwidth is calculated according to the sending state of the RTP data packet at the streaming media server end, the available network bandwidth is used as index data representing the current network condition, and the dependence of the calculation of the network condition on the feedback of the code stream information of the playing end is completely overcome; the number of frames that should be sent in the next GOP sequence is determined according to the calculated current available network bandwidth, that is, the code rate of the next GOP sequence should be sent, the real-time adjustment of the transmission code rate of the media data is realized, and the adaptive ability of the streaming media server under a weak network environment is improved without code stream retransmission; until the playing end stops playing the media data in the streaming media server, the weak network transmission operation of the streaming media server proposed in the present application is stopped.

[0012] The present application creatively proposes a weak network transmission method of a streaming media server, the available network bandwidth is independently calculated at the streaming media server end, the available network bandwidth is used as index data representing the current network condition, the dependence of the calculation of the network condition on the feedback of the code stream information of the playing end is completely overcome, the technical problem that the packet loss or the delay is too large when the playing end feeds back the code stream information under a weak network condition is solved, the number of frames that should be sent in the next GOP sequence is determined according to the calculated available network bandwidth, the real-time adjustment of the transmission code rate of the media data is realized, the adaptive ability of the streaming media server under a weak network environment is ensured without code stream retransmission, and the consumption of the memory resources in the streaming media server is reduced.

[0013] As a preferred, the step 102 specifically comprises:

[0014] The RTP data packet in the current GOP sequence is sent to the playing end, and the effective sending in the process of sending the RTP data packet is screened out.

[0015] The sending state of the RTP data packet in the effective sending is counted, and the sending state comprises the number of times of successful sending of the RTP data packet and the number of times of failed sending of the RTP data packet.

[0016] When sending RTP packets to the playback client, most RTP packets contain media data, which is called a valid transmission. However, sometimes RTP packets may not contain media data, which is called an invalid transmission. Invalid transmission may occur because a batch of RTP packets has been sent, but the last few or the very last RTP packet contains no actual media data; instead, it's an empty packet. An empty packet doesn't contain actual media data but still carries the RTP header and other control information to maintain the structural integrity of the RTP packet. Furthermore, in weak network environments, due to network instability, not every transmission will result in the playback client successfully receiving the RTP packets transmitted from the streaming media server; this is called a transmission failure.

[0017] By considering the empty packet phenomenon during the transmission of streaming media servers and the transmission failure phenomenon in weak network environments, the accuracy of the subsequent independent calculation of available network bandwidth by the streaming media server is increased to a certain extent.

[0018] Preferably, in step 103, the formula for calculating the available network bandwidth is:

[0019]

[0020] Where B0 represents the available network bandwidth after a GOP sequence has been successfully sent, T0 represents the time taken to successfully send all RTP packets in a GOP sequence, M0 represents the total number of successful RTP packet transmissions within the T0 time period, and M1 represents the total number of failed RTP packet transmissions within the T0 time period.

[0021] Preferably, step 104 specifically includes:

[0022] Determine the current network status based on available network bandwidth;

[0023] Determine the frame drop ratio for the next GOP sequence based on the current network conditions;

[0024] Calculate the number of frames to be sent in the next GOP sequence based on the frame loss ratio, and send the next GOP sequence according to the calculated number of frames.

[0025] Preferred methods for determining the current network status based on available network bandwidth include:

[0026] The available network bandwidth is divided into several value ranges based on the bandwidth value, and these value ranges are used to characterize different network conditions.

[0027] The calculated available network bandwidth is compared with the threshold range. Based on the threshold range into which the calculated available network bandwidth falls, the network condition that matches the calculated available network bandwidth is determined and recorded as the current network condition.

[0028] In this context, network condition refers to the quality and stability of data transmission within the network, while available network bandwidth refers to the amount of bandwidth available for data transmission within the network. The quality of network condition directly impacts the available network bandwidth. Poor network conditions, such as network congestion, signal interference, and packet loss, lead to a decline in data transmission quality, thereby affecting the available network bandwidth. Therefore, this invention uses available network bandwidth as an indicator of the current network condition.

[0029] Preferably, the network conditions include normal conditions, weak network conditions, and extreme conditions. Therefore, the method for determining the frame drop ratio of the next GOP sequence based on the current network conditions includes:

[0030] A mapping table is established, reflecting the mapping relationship between different network conditions and the frame drop ratio of the next GOP sequence. Specifically, if the network condition is normal, the frame drop ratio of the next GOP sequence is determined to be zero; if the network condition is weak, the frame drop ratio of the next GOP sequence matching the weak network condition is determined according to the mapping table; and if the network condition is extreme, the frame drop ratio of the next GOP sequence is determined to be... Where N represents the number of frames in the current GOP sequence;

[0031] The current network conditions are matched with the mapping table to obtain the frame drop ratio of the next GOP sequence.

[0032] Preferably, the method for calculating the number of frames to be sent in the next GOP sequence based on the frame loss ratio includes:

[0033] Subtract the frame loss ratio from 1, and then multiply it by the number of frames in the current GOP sequence. The resulting value is the number of frames to be sent in the next GOP sequence.

[0034] By establishing several objective and clear threshold ranges representing different network conditions and the corresponding frame drop ratio of the next GOP sequence, and then calculating the number of frames to be sent in the next GOP sequence based on the frame drop ratio, i.e. the bitrate that the next GOP sequence should send, the real-time adjustment of the transmission bitrate of media data is realized without the need for bitstream retransmission, thereby reducing the consumption of streaming media server memory resources.

[0035] A streaming media server's weak network transmission system includes:

[0036] The data acquisition module is used to read the media data to be sent from the streaming media server;

[0037] The preprocessing module is used to divide the media data to be sent into several groups of video frames, each group of video frames is recorded as a GOP sequence, and to slice each frame of the media data and encapsulate it into RTP data packets.

[0038] The network sending module is used to send RTP data packets within the current GOP sequence to the playback terminal;

[0039] The transmission status statistics module is used to track the transmission status of RTP data packets.

[0040] A network status calculation module is used to calculate the available network bandwidth based on the transmission status.

[0041] The frame loss processing module is used to determine the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent.

[0042] A computer device, comprising:

[0043] processor;

[0044] Memory for storing the executable instructions of the processor;

[0045] The processor is configured to execute a weak network transmission method for a streaming media server as described above by executing the executable instructions.

[0046] A computer-readable storage medium,

[0047] The computer-readable storage medium stores a computer program that, when executed by a processor, implements a weak network transmission method for a streaming media server as described above.

[0048] The beneficial technical effects of this invention include: It creatively proposes a method, system, device, and medium for weak network transmission of a streaming media server. The available network bandwidth is independently calculated by the streaming media server, and this available bandwidth is used as an indicator of the current network condition. This completely overcomes the dependence of network condition calculation on the feedback of bitstream information from the playback end, thus solving the technical problem of packet loss or excessive latency that may occur when the playback end provides bitstream information under weak network conditions. By establishing several objective and clear threshold ranges representing different network conditions and the corresponding frame loss ratio of the next GOP sequence, and then calculating the number of frames to be sent in the next GOP sequence based on the frame loss ratio (i.e., the bitrate to be sent in the next GOP sequence), real-time adjustment of the media data transmission bitrate is achieved. This ensures the adaptability of the streaming media server in weak network environments without requiring bitstream retransmission and also reduces the consumption of streaming media server memory resources.

[0049] Other features and advantages of the present invention will be disclosed in detail in the following detailed description and accompanying drawings. Attached Figure Description

[0050] The invention will be further described below with reference to the accompanying drawings:

[0051] Figure 1 This is a flowchart of a weak network transmission method for a streaming media server according to an embodiment of the present invention.

[0052] Figure 2 A flowchart illustrating the method for determining the frame loss ratio and sending the next GOP sequence in an embodiment of the present invention.

[0053] Figure 3 This is a schematic diagram of the weak network transmission system structure of the streaming media server according to an embodiment of the present invention.

[0054] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.

[0055] The module consists of: 1. Data acquisition module; 2. Preprocessing module; 3. Network transmission module; 4. Transmission status statistics module; 5. Network status calculation module; 6. Frame loss processing module; 7. Processor; and 8. Memory. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of the present invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of the present invention.

[0057] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to indicate orientation or positional relationship for the convenience of describing the embodiments and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0058] This application provides a method for weak network transmission of a streaming media server, applied to the process of a streaming media server sending the required media data to a playback terminal in a weak network environment, and the playback terminal playing the data in real time. Please refer to the appendix. Figure 1 This includes the following steps:

[0059] Step 101: Read the media data to be sent from the streaming media server, divide the media data to be sent into several groups of video frames, each group of video frames is recorded as a GOP sequence, and slice each frame of the media data and encapsulate it into RTP data packets.

[0060] In video coding, GOP (Group of Pictures) is a concept that defines the structure and encoding method of a video sequence. In video coding, video frames are divided into different groups, each group called a GOP. RTP (Real-time Transport Protocol) is a network protocol used for real-time transmission of audio and video data. It is widely used in real-time communication applications such as VoIP (Voice over IP) and real-time video streaming. RTP provides a standardized way to segment, transmit, and reassemble audio and video data. It is an end-to-end protocol used to transmit real-time data between sender and receiver. RTP's main function is data segmentation; that is, RTP divides audio or video data streams into small data packets, called RTP packets. In this embodiment, each frame of media data is sliced ​​into equal-length segments and encapsulated into RTP packets.

[0061] On the other hand, in this embodiment, step 101 can be implemented in ways including but not limited to the following two. One implementation is as follows: A portion of the media data to be sent is directly read from the source media data of the streaming media server. The media data to be sent is divided into several groups of video frames, each group of video frames is denoted as a GOP sequence, and each GOP sequence has the same number of frames. Each frame of the media data is sliced ​​and encapsulated into an RTP data packet, and the RTP data packet is placed in the waiting queue of the streaming media server to await transmission. Another implementation is as follows: The streaming media server sets up a fixed-size buffer for each playback end. The buffer stores a portion of the media data to be sent from the source media data. The media data to be sent is read from the buffer, divided into several groups of video frames, each group of video frames is denoted as a GOP sequence, and each GOP sequence has the same number of frames. Each frame of the media data to be sent is sliced ​​and encapsulated into an RTP data packet, and the RTP data packet is placed in the waiting queue of the buffer to await transmission. The latter implementation sends the media data in the current buffer, stores the next segment of media data to be sent, and then continues sending. This eliminates the need for the streaming media server to cache a large amount of RTP data packets before sending, providing a buffer for the streaming media server to send media data and reducing the memory resource usage of the streaming media server to a certain extent.

[0062] Step 102: Send the RTP data packets within the current GOP sequence to the playback terminal and check the sending status of the RTP data packets.

[0063] In this embodiment, considering the delays in the encapsulation and transmission of the media data to be sent, a timed RTP data packet sending method is adopted. That is, the streaming media server is set to attempt to send a number of RTP data packets to the playback terminal every second, sending one RTP data packet from the current GOP sequence each time. For example, the timer is set to send k times per second, sending one RTP data packet at the head of the queue to be sent each time.

[0064] Step 103: Calculate the available network bandwidth based on the sending status.

[0065] Step 104: Determine the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent.

[0066] In this context, the playback client, also known as the client-side device, is a device or application that communicates with the streaming media server to receive and play streaming media content, and controls the receiving and playback process according to the server's transmission strategy and feedback mechanism. In this embodiment, the playback client can be a computer, smartphone, tablet, or other similar terminal device.

[0067] The technical concept of this invention is as follows: In the process of a streaming media server sending the required media data to a playback terminal in a weak network environment and the playback terminal playing it in real time, firstly, the media data to be sent is read from the source media data of the streaming media server, and the media data to be sent is divided into several groups of video frames. Each group of video frames is recorded as a GOP sequence, and the number of frames in each GOP sequence is the same. Each frame of the media data is sliced ​​and encapsulated into RTP data packets. That is, a GOP sequence may be split into multiple RTP data packets for transmission. RTP data packets are used as the transmission unit for encapsulating media data, and GOP sequences are used as the statistical unit for the media data transmission status. Secondly, considering the delay in the encapsulation and transmission of the media data to be sent, a timed RTP data packet sending method is adopted, that is, the streaming media server is set to send RTP data packets every second. The system attempts to send several RTP data packets to the playback end, sending one RTP data packet at a time and recording the sending status of the RTP data packets. Then, it calculates the available network bandwidth based on the sending status of the RTP data packets on the streaming media server side, using the available network bandwidth as an indicator of the current network status, completely overcoming the dependence of network status calculation on the playback end's bitstream information feedback. Next, based on the calculated current available network bandwidth, it determines the number of frames to be sent in the next GOP sequence, i.e., the bitrate to be sent in the next GOP sequence, realizing real-time adjustment of the media data transmission bitrate, thereby improving the streaming media server's adaptability to transmission in weak network environments without requiring bitstream retransmission. The weak network transmission operation of the streaming media server proposed in this invention stops when the playback end stops playing the media data in the streaming media server.

[0068] In contrast to existing technologies where streaming media servers require the playback client to continuously collect and transmit received media data back to the server when calculating available network bandwidth, this method is inherently flawed due to packet loss in poor network conditions. To address this, this invention creatively proposes a weak network transmission method for streaming media servers. The streaming media server independently calculates the available network bandwidth, using it as an indicator of current network conditions. This completely overcomes the dependence of network condition calculation on playback client bitstream information feedback, thus solving the technical problems of packet loss or excessive latency that may occur when the playback client transmits bitstream information under weak network conditions. Furthermore, based on the calculated available network bandwidth, the number of frames to be sent in the next GOP sequence is determined, enabling real-time adjustment of the media data transmission bitrate. This ensures the adaptability of the streaming media server in weak network environments without requiring bitstream retransmission and also reduces the consumption of streaming media server memory resources.

[0069] On the other hand, in this embodiment, step 102 specifically includes:

[0070] Send RTP data packets within the current GOP sequence to the playback terminal, and filter out valid transmissions during the RTP data packet sending process;

[0071] The system tracks the transmission status of RTP packets during valid transmissions, including the number of successful RTP packet transmissions and the number of failed RTP packet transmissions.

[0072] When sending RTP packets to the playback end, most RTP packets contain media data, which is called a valid transmission. However, sometimes RTP packets do not contain media data, which is called an invalid transmission. Invalid transmission may occur because a batch of RTP packets has been sent, but the last few or the very last RTP packet contains no actual media data; instead, it is an empty packet. Specifically, an "empty packet" may occur in the following situations:

[0073] 1) Encoder operation complete: In the encoder, media data is sliced ​​and encapsulated into RTP packets. If the last few RTP packets do not have enough data to fill them after all media data has been sliced ​​and encapsulated, empty packets will appear.

[0074] 2) End of data stream: If the source media data stream has ended, that is, there is no more media data available to be encapsulated into RTP packets, an empty packet may occur.

[0075] 3) Insufficient media data duration: Some media data may be too short to fill an RTP packet. In this case, the last or last few RTP packets may be empty.

[0076] Although empty packets do not contain actual media data, they still carry RTP headers and other control information to maintain the structural integrity of the RTP data packet. When the playback client receives these empty packets, it may process them according to the media type and other information, such as continuing to play the previous media data or muting it; this embodiment will not elaborate on this. Moreover, in a weak network environment, due to network instability, not every transmission will result in the playback client successfully receiving the RTP data packet transmitted by the streaming media server; this is called a transmission failure. Therefore, this embodiment records whether each RTP data packet transmission is successful or failed. In a weak network environment, due to increased network latency, RTP data packets may experience significant delays during transmission. The streaming media server can determine and record the transmission status of each RTP data packet by monitoring the transmission time. If the transmission time exceeds a set threshold, the RTP data packet transmission can be determined to have failed.

[0077] This embodiment, by considering the empty packet phenomenon during the transmission process of the streaming media server and the transmission failure phenomenon in a weak network environment, increases the accuracy of the subsequent independent calculation of available network bandwidth by the streaming media server in a weak network environment.

[0078] On the other hand, in this embodiment, the formula for calculating the available network bandwidth in step 103 is:

[0079]

[0080] Where B0 represents the available network bandwidth after a GOP sequence has been successfully sent, T0 represents the time taken to successfully send all RTP packets in a GOP sequence, M0 represents the total number of successful RTP packet transmissions within the T0 time period, M1 represents the total number of failed RTP packet transmissions within the T0 time period, and L represents the length of each frame of the media data after slicing.

[0081] The simplified formula for calculating available network bandwidth is:

[0082]

[0083] This embodiment uses GOP sequences as the statistical unit for media data transmission status. Based on the number of successful transmissions, the number of failed transmissions, and the transmission time within a GOP sequence, the available network bandwidth after transmitting a GOP sequence is calculated in real time. This allows for better control over the size and frequency of media data transmission, resulting in better bandwidth utilization and more stable data transmission. Furthermore, real-time performance is crucial in streaming media applications because data needs to be transmitted and played at specific time intervals. Therefore, using GOP sequences as the unit for statistical analysis can better control the transmission time of media data, ensuring that media data arrives at the playback end on time and remains continuous and smooth during playback, thereby improving the quality and reliability of streaming media transmission.

[0084] On the other hand, in this embodiment, please refer to the appendix. Figure 2 In step 104, the method of determining the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and sending the next GOP sequence according to the number of frames to be sent, specifically includes:

[0085] Step 201: Determine the current network status based on available network bandwidth;

[0086] Step 202: Determine the frame drop ratio of the next GOP sequence based on the current network conditions;

[0087] Step 203: Calculate the number of frames to be sent in the next GOP sequence based on the frame loss ratio, and send the next GOP sequence according to the calculated number of frames.

[0088] On the other hand, in this embodiment, step 201, the method for determining the current network status based on available network bandwidth includes:

[0089] The available network bandwidth is divided into several threshold ranges based on the bandwidth value. These threshold ranges are used to characterize different network conditions.

[0090] The calculated available network bandwidth is compared with the threshold range. Based on the threshold range into which the calculated available network bandwidth falls, the network condition that matches the calculated available network bandwidth is determined and recorded as the current network condition.

[0091] In this context, network conditions refer to the quality and stability of data transmission within the network, while available network bandwidth refers to the amount of bandwidth available for data transmission. The quality of network conditions directly impacts the available network bandwidth. Poor network conditions, such as network congestion, signal interference, and packet loss, lead to a decline in data transmission quality, thus affecting the available network bandwidth. In practice, the worse the network conditions, the smaller the available network bandwidth calculated using the method proposed in this embodiment, and vice versa. Under normal network conditions, the calculated available network bandwidth remains relatively stable at a certain value. Therefore, this embodiment uses the calculated available network bandwidth as an indicator of the current network conditions, ensuring the accuracy of the streaming media server's independent determination of the current network status.

[0092] On the other hand, in this embodiment, the network conditions include normal conditions, weak network conditions, and extreme conditions. Therefore, in step 202, the method for determining the frame drop ratio of the next GOP sequence based on the current network conditions includes:

[0093] A mapping table is established to reflect the mapping relationship between different network conditions and the frame drop ratio of the next GOP sequence. Specifically, if the network condition is normal, the frame drop ratio of the next GOP sequence is set to zero. If the network condition is weak, the frame drop ratio of the next GOP sequence matching the weak network condition is determined according to the mapping table. If the network condition is extreme, the frame drop ratio of the next GOP sequence is set to... Where N represents the number of frames in the current GOP sequence;

[0094] By matching the current network conditions with the mapping table, the frame drop ratio of the next GOP sequence can be obtained.

[0095] On the other hand, in this embodiment, step 203, the method for calculating the number of frames to be sent in the next GOP sequence based on the frame loss ratio, includes:

[0096] Subtract the frame loss ratio from 1, and then multiply it by the number of frames in the current GOP sequence. The resulting value is the number of frames to be sent in the next GOP sequence.

[0097] For example, in this embodiment, the implementation of determining the number of frames to be sent in the next GOP sequence based on the calculated available network bandwidth, and sending the next GOP sequence according to the number of frames to be sent, is as follows:

[0098] First, set four - level thresholds for the available network bandwidth according to the bandwidth value. Among them, the first threshold is B41, the second threshold is B42, the third threshold is B43, and the fourth threshold is B44, where B41 < B42 < B43 < B44 < 1. Five threshold intervals representing network conditions are obtained according to the four - level thresholds. (0, B41] represents normal conditions, (B41, B42] represents relatively mild weak - network conditions, (B42, B43] represents relatively severe weak - network conditions, (B43, B44] represents severe weak - network conditions, and (B44, 1] represents extreme conditions;

[0099] Secondly, establish a mapping table. The mapping table reflects the mapping relationship between five network conditions and the frame - loss ratio of the next GOP sequence. Among them, the frame - loss ratios of the next GOP sequence corresponding to the five network conditions are 0, P41, P42, P43, and P44 respectively, where 0 < P41 < P42 < P43 < P44 < 1;

[0100] Next, since the number of frames in each GOP sequence is the same, obtain the number of frames N in a GOP sequence, and compare the calculated available network bandwidth B0 with the threshold intervals.

[0101] If B0 < B41, it is determined that the current network condition is normal, the frame - loss ratio of the next GOP sequence is 0, and the number of frames to be sent in the next GOP sequence is N(1 - 0)=N, that is, the frames in the next GOP sequence can be sent normally without frame - loss transmission;

[0102] If B41 < B0 ≤ B42, it is determined that the current network condition is a relatively mild weak - network condition, there are frame losses in the network, the frame - loss ratio is determined to be P41, and the number of frames to be sent in the next GOP sequence is N(1 - P41);

[0103] If B42 < B0 ≤ B43, it is determined that the current network condition is a relatively severe weak - network condition, the network has serious frame losses, the frame - loss ratio is determined to be P42, and the number of frames to be sent in the next GOP sequence is N(1 - P42);

[0104] If B43 < B0 ≤ B44, it is determined that the current network condition is a severe weak - network condition, the network has serious frame losses, the frame - loss ratio is determined to be P43, and the number of frames to be sent in the next GOP sequence is N(1 - P43);

[0105] If B0 > B44, it is determined that the current network condition is an extreme condition. In this case, the network condition is very poor, so it is set to an extreme frame - loss state, and the frame - loss ratio is determined to be Then the number of frames to be sent in the next GOP sequence is The total data volume N of a GOP sequence is the sum of the data volume of one I-frame and all P-frames. Therefore, in extreme cases, all P-frames in the entire GOP sequence to be sent are discarded and only I-frames are sent.

[0106] In video sequences, I-frames (Intra-coded frames), also known as keyframes, are independent frames that do not rely on other frames for decoding. I-frames are usually the starting point of a video sequence, containing complete image information and serving as a reference for other frames. P-frames (Predictive-coded frames) are frames that are differentially encoded based on the previous keyframe or P-frame. They only contain the difference information from the previous frame, and the image can be reconstructed by decoding the previous frame and restoring the difference information.

[0107] When the playback device decodes the received GOP sequence, the first P-frame within the GOP sequence needs to reference the preceding I-frame, and subsequent P-frames reference the previous P-frame sequentially, such as P1 depending on I, P2 depending on P1, P3 depending on P2, P4 depending on P3, and so on. Therefore, after determining the frame drop ratio, the dropping starts from the last P-frame in the next GOP sequence. For example, in a GOP sequence with 50 frames, if the calculated frame drop ratio is determined to be 0.16, meaning 8 frames should be dropped, the frames sent in the next GOP sequence will be the first 42 frames in the next GOP sequence.

[0108] In practice, the threshold value B44, which represents the extreme condition, has a good adaptive bit rate control effect within 30%. Therefore, in this embodiment, the threshold value B44, which represents the extreme condition, is preferably set to 30%. The corresponding other threshold values, such as B43, B42, and B41, should be less than B44 and decrease in sequence. Specific parameters can be set in the project.

[0109] This embodiment independently calculates the available network bandwidth on the streaming media server side and designs a frame dropping strategy to adapt to changes in the weak network transmission environment. Several objective and clear threshold ranges representing different network conditions are established, along with the corresponding frame dropping ratio for the number of frames that can be sent in the next GOP sequence. By discarding a certain proportion of media data (P frames) in the GOP sequence to be sent, the transmission frame rate of the next GOP sequence of the streaming media server is reduced, thereby reducing the network transmission bitrate of the media data. This method is convenient and reliable to implement on the streaming media server side and can adapt to fluctuations in network conditions without requiring retransmission of the bitstream, thus reducing the consumption of memory resources of the streaming media server.

[0110] On the other hand, this application also provides a streaming media server weak network transmission system that is consistent with the above-described streaming media server weak network transmission method concept. Please refer to the appendix. Figure 3 ,include:

[0111] Data acquisition module 1 is used to read media data to be sent from the streaming media server;

[0112] The preprocessing module 2 is used to divide the media data to be sent into several groups of video frames, each group of video frames is recorded as a GOP sequence, and each frame of the media data is sliced ​​and encapsulated into RTP data packets.

[0113] Network sending module 3 is used to send RTP data packets within the current GOP sequence to the playback terminal;

[0114] The sending status statistics module 4 is used to count the sending status of RTP data packets;

[0115] Network status calculation module 5 is used to calculate available network bandwidth based on the transmission status;

[0116] The frame loss processing module 6 is used to determine the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent.

[0117] On the other hand, embodiments of this application also provide a computer device, please refer to the appendix. Figure 4 ,include:

[0118] Processor 7;

[0119] Memory 8 is used to store the executable instructions of processor 7;

[0120] The processor 7 is configured to execute a weak network transmission method for a streaming media server, as described above, by executing executable instructions.

[0121] It should be noted that the computer device described above, based on the method embodiments, may also include other implementation methods. Specific implementation methods can be found in the descriptions of the relevant method embodiments, and will not be elaborated upon here.

[0122] The control system or computer device for a damper provided in this specification can also be applied to various data analysis and processing systems. The computer device can be a standalone server, or it can include a server cluster, system (including distributed system), software (application), actual operating device, logic gate circuit device, quantum computer, etc., combined with necessary implementation hardware, using the methods or systems of the embodiments in this specification.

[0123] The processor 7 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0124] The memory 8 stores program code, which can be executed by the processor 7 to perform any of the deep learning-based operation strategy triggering methods described above. In some embodiments, the memory 8 can be an internal storage unit of a computer device, such as a hard drive or memory. In other embodiments, the memory 8 can be an external storage device of the computer device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 8 can include both internal and external storage units of the computer device.

[0125] On the other hand, embodiments of this application also provide a computer-readable storage medium.

[0126] The computer-readable storage medium stores a computer program that, when executed by the processor 7, implements a weak network transmission method for a streaming media server as described above.

[0127] It should be noted that the computer-readable storage medium described above in this disclosure can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0129] The aforementioned computer-readable storage medium may be included in the aforementioned computer device; or it may exist independently and not assembled into the computer device.

[0130] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the modules themselves.

[0131] The above is merely a preferred embodiment disclosed in this application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by specific combinations of the above-mentioned technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-mentioned technical features or their equivalent features without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

[0132] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

Claims

1. A method for weak network transmission of a streaming media server, characterized in that, The streaming media server performs the following steps: Step 101: Read the media data to be sent from the streaming media server, divide the media data to be sent into several groups of video frames, each group of video frames is recorded as a GOP sequence, and slice each frame of the media data and encapsulate it into an RTP data packet. Step 102: Send the RTP data packets within the current GOP sequence to the playback terminal and count the sending status of the RTP data packets; Step 103: Calculate the available network bandwidth based on the transmission status; Step 104: Determine the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent. Step 102 specifically includes: Send RTP data packets within the current GOP sequence to the playback terminal, and filter out valid transmissions during the RTP data packet sending process; The sending status of RTP data packets in valid transmission is statistically analyzed, including the number of times RTP data packets were successfully sent and the number of times RTP data packets failed to be sent. In step 103, the formula for calculating the available network bandwidth is: ， in, This represents the available network bandwidth after a valid GOP sequence has been sent. This indicates the time taken to effectively send all RTP packets in a GOP sequence. express The total number of successful RTP packet transmissions within the time period. express The total number of RTP packet transmission failures within the time period; Step 104 specifically includes: Determine the current network status based on available network bandwidth; Determine the frame drop ratio for the next GOP sequence based on the current network conditions; Calculate the number of frames to be sent in the next GOP sequence based on the frame loss ratio, and send the next GOP sequence according to the calculated number of frames.

2. The method for weak network transmission of a streaming media server as described in claim 1, characterized in that, Methods for determining the current network status based on available network bandwidth include: The available network bandwidth is divided into several value ranges based on the bandwidth value, and these value ranges are used to characterize different network conditions. The calculated available network bandwidth is compared with the threshold range. Based on the threshold range into which the calculated available network bandwidth falls, the network condition that matches the calculated available network bandwidth is determined and recorded as the current network condition.

3. The method for weak network transmission of a streaming media server as described in claim 1, characterized in that, The network conditions include normal conditions, weak network conditions, and extreme conditions. Therefore, the method for determining the frame drop ratio of the next GOP sequence based on the current network conditions includes: A mapping table is established, reflecting the mapping relationship between different network conditions and the frame drop ratio of the next GOP sequence. Specifically, if the network condition is normal, the frame drop ratio of the next GOP sequence is determined to be zero; if the network condition is weak, the frame drop ratio of the next GOP sequence matching the weak network condition is determined according to the mapping table; and if the network condition is extreme, the frame drop ratio of the next GOP sequence is determined to be... ,in Indicates the number of frames in the current GOP sequence; The current network conditions are matched with the mapping table to obtain the frame drop ratio of the next GOP sequence.

4. The method for weak network transmission of a streaming media server as described in claim 1, characterized in that, The method for calculating the number of frames to be sent in the next GOP sequence based on the frame loss ratio includes: Subtract the frame loss ratio from 1, and then multiply it by the number of frames in the current GOP sequence. The resulting value is the number of frames to be sent in the next GOP sequence.

5. A weak network transmission system for a streaming media server, characterized in that, include: The data acquisition module is used to read the media data to be sent from the streaming media server; The preprocessing module is used to divide the media data to be sent into several groups of video frames, each group of video frames is recorded as a GOP sequence, and to slice each frame of the media data and encapsulate it into RTP data packets. The network sending module is used to send RTP data packets within the current GOP sequence to the playback terminal; The transmission status statistics module is used to track the transmission status of RTP data packets. A network status calculation module is used to calculate the available network bandwidth based on the transmission status. The frame loss processing module is used to determine the number of frames to be sent in the next GOP sequence based on the available network bandwidth, and send the next GOP sequence according to the number of frames to be sent. The network sending module is also used to send RTP data packets within the current GOP sequence to the playback terminal and to filter out valid transmissions during the RTP data packet sending process. The transmission status statistics module is also used to count the transmission status of RTP data packets in valid transmission, including the number of times RTP data packets were successfully transmitted and the number of times RTP data packets were failed to be transmitted. In the network status calculation module, the formula for calculating the available network bandwidth is: ， in, This represents the available network bandwidth after a valid GOP sequence has been sent. This indicates the time taken to effectively send all RTP packets in a GOP sequence. express The total number of successful RTP packet transmissions within the time period. express The total number of RTP packet transmission failures within the time period; The frame loss processing module is also used to perform the following steps: Determine the current network status based on available network bandwidth; Determine the frame drop ratio for the next GOP sequence based on the current network conditions; Calculate the number of frames to be sent in the next GOP sequence based on the frame loss ratio, and send the next GOP sequence according to the calculated number of frames.

6. A computer device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute a weak network transmission method for a streaming media server as described in any one of claims 1 to 4 by executing the executable instructions.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements a weak network transmission method for a streaming media server as described in any one of claims 1 to 4.

Images