RTP (Real-time Transport Protocol) stream multi-path aggregation and packet loss retransmission method under network monitoring scene

Abstract

The invention provides a method for multi-channel aggregation and packet loss retransmission of RTP (Real-time Transport Protocol) streams in a network monitoring scene, which comprises the following steps: multi-channel aggregation and separation of the RTP streams: a server aggregates a plurality of single-channel video streams to form an aggregated data stream, and sends the aggregated data stream to a player client; when the server receives a plurality of single-channel video streams, the server distributes an ID for each single-channel video stream; automatically requesting data packet accumulation processing and packet loss detection of the retransmission object; automatically requesting packet loss and retransmission processing of the retransmission object; processing the automatic request retransmission object under the condition of no packet loss; and the server processes the packet loss request: when the server receives the packet loss request from the client, the server extracts the sequence number of the lost data packet contained in the request, and searches the corresponding data packet in the cache. According to the invention, by using a packet loss retransmission technology, the integrity of a network video monitoring picture and the playing fluency can be improved when the network fluctuates.

Description

Technical Field

[0001] The present invention relates to the field of multimedia communication technology, and in particular to a method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario. Background Art

[0002] RTP, or Real-time Transport Protocol, is a network transmission protocol used to transmit audio and video over the Internet. It provides end-to-end transmission services with real-time characteristics, including information such as timestamps, sequence numbers, and payload formats to support the transmission of real-time multimedia streams. When using the RTP protocol based on the UDP protocol for video data stream transmission, there is no packet loss retransmission mechanism.

[0003] In response to the problem of network packet loss, CN202011384070.3 solves it by referring to the method of packet loss retransmission. The patent determines whether packet loss has occurred by detecting whether the sequence numbers of the first RTP packet and the RTP packet corresponding to the currently playing audio and video data are continuous. If packet loss occurs, a packet loss request is sent to the server based on the sequence number of the detected lost RTP packet to be retransmitted. The method provided by the patent is to trigger the search for the packet loss sequence number in the cache queue in time when reading the RTP packet from the cache queue, so as to send a packet loss request to the server in time, increase the hit rate of the packet loss request hitting the server window period, so as to obtain the packet loss data with a relatively high probability and ensure the quality of audio and video playback.

[0004] CN202111176877.2 realizes reliable transmission of video frames through the RTP extension header method. It can be extended based on the RTP protocol itself. By expanding the RTP extension header, the compatibility of video devices is guaranteed, and the packet loss retransmission method based on the extension of the RTP header is adopted. Under the condition of low network packet loss rate, the reliable transmission of video frames can be realized with very little bandwidth, effectively ensuring the video quality.

[0005] Usually one RTP session corresponds to only one media stream. Aggregating multiple RTP data streams into one RTP data stream can conveniently transmit multiple video images. The current related patent introduces a packet loss retransmission mechanism for RTP, which treats one RTP stream as one audio and video stream, and does not take into account the situation where a single RTP stream transmits multiple media. If this RTP data stream contains multiple audio and video streams, when the round-trip time RTT of the data packet is too large, one or more audio and video streams will lose packets and cannot be processed in time. In this case, if the real-time performance of the audio and video stream is pursued, it will easily cause the audio and video playback to be stuck and the playback to be not smooth. If the smoothness of the audio and video stream playback is pursued, it will cause a large delay. Summary of the invention

[0006] In view of this, the purpose of the present invention is to propose a method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario, so as to solve the technical problem that when an RTP data stream contains multiple audio and video streams, packet loss cannot be processed in time, resulting in jamming.

[0007] The technical means adopted by the present invention are as follows:

[0008] A method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario comprises the following steps:

[0009] Multi-channel aggregation and separation of RTP streams: The server aggregates multiple single-channel video streams to form an aggregated data stream and sends it to the player client; when the server receives multiple single-channel video streams, the server assigns an ID to each single-channel video stream; the server inserts the ID of each single-channel video stream into a data packet and sends it to the player client; the player client separates the aggregated data stream according to the ID of each data packet; after receiving the data packet, the player client obtains its ID to determine the video stream it belongs to. If it is a new data stream, the client will create an automatic request retransmission object and pass the data packet to the object. If it is an existing data stream, the data packet will be passed to the corresponding object;

[0010] Data packet accumulation processing and packet loss detection of the object of automatic retransmission request: After receiving a new data packet, the object of automatic retransmission request determines whether there is packet loss or data packet accumulation in the current cache queue. If there is data packet accumulation, the extra data packets are added to the code stream recovery queue in sequence and deleted from the cache queue. While removing the extra data packets from the cache queue, check whether there is packet loss between data packets. If packet loss occurs, the sequence number of the lost packet is recorded and sent to the server. When packet loss is found or there is no data packet accumulation, the packet loss and retransmission processing steps of the object of automatic retransmission request are executed;

[0011] Automatically request the packet loss and retransmission processing of the retransmission object: Check whether the received data packet is the server's retransmission data packet. If it is a retransmission data packet, insert the retransmission data packet into the corresponding position of the cache queue and remove it from the packet loss sequence number queue; if it is not a retransmission data packet and the waiting time has not timed out, continue to wait for the next data packet; if it times out, no longer wait for retransmission, directly send the first data packet in the cache queue to the code stream recovery queue, and clear the corresponding packet loss sequence number;

[0012] Processing when there is no packet loss in the automatic retransmission request object: If there is no packet loss in the current data stream, the data packets are added to the cache queue of the automatic retransmission request object and sorted to prevent misjudgment due to disordered data packets; when the data packets in the cache queue are greater than or equal to the predetermined minimum value, they will be processed in the data packet accumulation processing and packet loss detection steps of the automatic retransmission request object in the next cycle;

[0013] Server processing of packet loss request: When the server receives a packet loss request from the client, the server extracts the lost data packet sequence number contained in the request and searches for the corresponding data packet in the cache; if the corresponding data packet is found, the server marks the corresponding data packet as a retransmission packet and sends it to the client.

[0014] Furthermore, the specific structure of the RTP data packet is:

[0015] The version number V indicates the protocol version; the padding bit P indicates whether there is padding at the end of the data packet; the extension bit X indicates whether an extension header is included; the contribution source count CC indicates the number of CSRC fields; the mark bit M is used to mark special data packets; the payload type PT indicates the type of payload carried in the data packet; the sequence number identifies the order of the data packets; the timestamp is used to identify the sampling time of the media data; the synchronization source identifier SSRC uniquely identifies the source of the data stream in the same session; the contribution source identifier CSRC is used to identify the contribution source of the media stream generated by mixing or synthesis; the custom field data stream identifier ID is extended to distinguish different data streams; the payload represents the audio and video data; the padding is used to align the data packet to a specific block size.

[0016] Furthermore, the multi-channel aggregation of the RTP stream specifically includes the following steps:

[0017] S11, waiting to receive RTP data packets, and executing S12 after receiving the data packets;

[0018] S12, obtain the SSRC of the RTP data packet, determine whether it is a new RTP session, if so, execute S13, otherwise execute S14;

[0019] S13, find the ID corresponding to the RTP session SSRC, and execute S15;

[0020] S14, assign an ID to the new data stream, and ensure that the ID does not conflict with other IDs, and execute S15;

[0021] S15, divide the 32-bit unsigned integer ID into four 8-bit unsigned numbers from high to low, write them into the first 4 bytes of the send buffer array, and execute S16;

[0022] S16, copy the data in the received data packet to the fourth byte of the send buffer array, and execute S17 starting from the fifth byte;

[0023] S17, setting the timestamp and flag of the data packet to be sent to be the same as those of the received data packet, sending the data packet to the player client, and executing S11.

[0024] Furthermore, the automatic retransmission request object is responsible for packet loss detection and retransmission request, and the automatic retransmission request object includes a data packet cache queue, a packet loss sequence number queue, and a stream recovery queue; the data packet cache queue is used to store data packets; the packet loss sequence number queue is used to record the sequence number of the lost data packet; the stream recovery queue is used to restore the fragmented H.265 data to the original H.265 code stream.

[0025] Furthermore, demultiplexing of the RTP stream includes the following steps:

[0026] S110, initializing RTP session parameters and establishing an RTP session;

[0027] S120, waiting to receive a data packet sent by the server;

[0028] S130, when a data packet is received, read the ID of the data stream to which the data packet belongs, and determine whether it is a new data stream. If it is a new data stream, execute S140, otherwise execute S150;

[0029] S140, according to the ID of the data packet, create a corresponding automatic request return object for this data stream, which is used to detect whether this data stream has packet loss, and can request retransmission from the server when packet loss occurs;

[0030] S150, passing the data packet to the automatic request return object, allowing the object to detect whether packet loss occurs and implement a packet loss retransmission function.

[0031] Furthermore, the data packet accumulation processing and packet loss detection of the automatic retransmission request object, the packet loss and retransmission processing of the automatic retransmission request object, and the processing of the automatic retransmission request object without packet loss specifically include the following steps:

[0032] S160, determining whether the packet loss sequence number queue of the object automatically requested for return transmission is empty and whether the cache queue is greater than or equal to the minimum value; if the data flow corresponding to the object has not experienced packet loss, and the data packet in the cache queue is greater than the minimum value, executing S170; otherwise, if there is insufficient data in the cache queue or packet loss has occurred in the current data flow, executing S180;

[0033] S170, determining whether the sequence numbers of the first two data packets in the cache queue are continuous, if so, it means that there is no packet loss between the two data packets, and executing S171, otherwise, packet loss occurs, and executing S172;

[0034] S171, adding the first data packet to the video code stream recovery queue and clearing it from the cache queue to recover the original video data for decoding;

[0035] S172, record the missing data packet sequence number in the two data packets, and add the data packet sequence number to the packet loss sequence number queue;

[0036] S173, sending the missing data packet sequence number to the server, executing S120, and continuing to receive data packets;

[0037] S180, determining whether the packet loss sequence number queue of the current data flow is empty, if it is empty, indicating that no packet loss is found in the current data flow, and executing S190, otherwise executing S200;

[0038] S190: If no packet loss is found in the current data flow, the data packet is directly added to the cache queue of the automatic request return object and sorted;

[0039] S200, determining whether the received data packet is a retransmitted data packet, if yes, executing S201, otherwise executing S210;

[0040] S201, adding the retransmitted data packet to the corresponding position in the cache queue, and clearing the sequence number of the data packet in the packet loss sequence number queue;

[0041] S210, if the received data packet is not a retransmitted data packet, then the data packet is added to a cache queue;

[0042] S220, determining whether the waiting time for retransmitting the data packet has timed out, the timeout period being set according to the parameters of the video stream, if not timed out, executing S120, continuing to wait for receiving the data packet, otherwise executing S230;

[0043] S230: Add the first data packet in the buffer queue to the code stream recovery queue, and clear the packet loss sequence number queue.

[0044] Furthermore, the server processes the packet loss request in the following steps:

[0045] Establish a TCP connection with the player client program;

[0046] Waiting to receive TCP message containing packet loss sequence number message;

[0047] Search for this packet in the cache queue. If it is found, execute the next step, otherwise execute the previous step.

[0048] All found data packets are sent to the client again, and the packet type PT is set to 103 to represent resent data packets.

[0049] The present invention also provides a storage medium, which includes a stored program, wherein when the program is running, the method of RTP stream multi-channel aggregation and packet loss retransmission in any of the above-mentioned network monitoring scenarios is executed.

[0050] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for RTP stream multi-channel aggregation and packet loss retransmission in any of the above-mentioned network monitoring scenarios through the computer program.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] The present invention uses packet loss retransmission technology to improve the integrity of network video monitoring images and the smoothness of playback when the network fluctuates.

[0053] The present invention aggregates multiple RTP video streams into one channel, thereby facilitating the forwarding processing of multiple videos by a server.

[0054] The present invention decomposes the aggregated video stream and performs packet loss retransmission, which facilitates the management of the packet loss retransmission waiting time of each channel, is compatible with video streams of different frame rates, and avoids mutual influence between video streams when the network fluctuates. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0056] Figure 1 This is a flow chart of RTP aggregate stream decomposition, packet loss detection and retransmission application according to the present invention.

[0057] Figure 2 This is the RTP stream aggregation flow chart of the present invention.

[0058] Figure 3 This is a flow chart of the server responding to a packet loss request of the present invention.

[0059] Figure 4 It is the RTP data packet header extension structure of the present invention. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0062] The present invention is used for a network video monitoring system, which consists of three parts: a video transmitter, a server, and a player client. There can be many video transmitters, which send video stream data to the server, and the server program forwards the data to the player client after receiving the data. This patent is used for network communication between the server and the player client.

[0063] like Figure 1-3 As shown, the present invention provides a method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario, comprising the following steps:

[0064] (1) Multi-channel aggregation and separation of RTP streams

[0065] The aggregation task is performed by the server program. When the server receives multiple single-channel video streams, the server assigns an ID to each video stream. The server inserts the corresponding ID into the data packet and sends it to the player client. The client separates the aggregated data stream according to the ID of each data packet. After receiving the data packet, the client first obtains its ID to determine the video stream to which it belongs. If it is a new data stream, the client will create an automatic request retransmission object and pass the data packet to the object. If it is an existing data stream, the data packet will be passed to the corresponding object.

[0066] (2) Automatically requesting retransmission of data packets and detecting packet loss

[0067] After receiving a new data packet, the automatic retransmission object first determines whether there is packet loss or data packet accumulation in the current cache queue. If there is data packet accumulation, the extra data packets will be added to the stream recovery queue in sequence and deleted from the cache queue. While removing the extra data packets from the cache queue, it will also check whether there is packet loss between data packets. If there is packet loss, the sequence number of the lost packet will be recorded and sent to the server. When packet loss is found or there is no data packet accumulation, the next step is to process the received data packet.

[0068] (3) Packet loss and retransmission processing of automatic retransmission request objects

[0069] The program first determines whether there is packet loss in the current data stream. If there is packet loss, the program will check whether the received data packet is a retransmitted data packet from the server. If it is a retransmitted data packet, the program will insert it into the corresponding position of the cache queue and remove it from the packet loss sequence number queue; if not, and the waiting time has not timed out, it will continue to wait for the next data packet; if it times out, the program will no longer wait for retransmission, and directly send the first data packet in the cache queue to the code stream recovery queue, and clear the corresponding packet loss sequence number.

[0070] (4) Automatically requesting retransmission of an object without packet loss

[0071] If there is no packet loss in the current data stream, the program will add the data packet to the cache queue of the automatic request retransmission object and sort it to prevent misjudgment due to packet disorder. When the number of data packets in the cache queue is greater than or equal to the predetermined minimum value, it will be processed in step (2) in the next cycle.

[0072] (5) Server handles packet loss request

[0073] When the server receives a packet loss request from the client, it extracts the lost packet sequence number contained in the request and searches for the corresponding packet in the cache. If the corresponding packet is found, the server marks it as a retransmission packet and sends it to the client.

[0074] Figure 4The RTP data packet structure provided by an embodiment of the present invention is shown, including: a version number (V, 2 bits), indicating a protocol version; a padding bit (P, 1 bit), indicating whether there is padding at the end of the data packet; an extension bit (X, 1 bit), indicating whether an extension header is included; a contribution source count (CC, 4 bits), indicating the number of CSRC fields; a mark bit (M, 1 bit), used to mark a special data packet; a payload type (PT, 7 bits), indicating the type of payload carried in the data packet; a sequence number (16 bits), identifying the order of the data packet; a timestamp (32 bits), used to identify the sampling time of the media data; a synchronization source identifier (SSRC, 32 bits), uniquely identifying the source of the data stream in the same session; a contribution source identifier (CSRC, 32 bits / piece, the number is specified by CC), used to identify the contribution source of the media stream generated by mixing or synthesis; a custom field data stream identifier (ID, 32 bits), extended to distinguish different data streams; a payload, indicating audio and video data; and padding, used to align the data packet to a specific block size.

[0075] Based on the original RTP data packet structure, the present invention adds a 32-bit ID before the original payload to distinguish which data stream the data packet belongs to. In the present invention, different PT values ​​are used to distinguish normal data packets from retransmitted data packets, 98 is used to represent H.265 encoded video stream data packets, and 103 is used to represent retransmitted data packets. The present invention uses detection of whether the sequence numbers of adjacent data packets are continuous to determine whether packet loss has occurred.

[0076] Figure 2 The figure is a flow chart of RTP stream aggregation provided by an embodiment of the present invention. The key steps in the flow chart are described in detail below:

[0077] S1: Waiting to receive RTP data packets, and executing S2 after receiving the data packets;

[0078] S2: Get the SSRC of the RTP data packet and determine whether it is a new RTP session. If so, execute S3; otherwise, execute S4.

[0079] S3: Find the ID corresponding to the SSRC of this RTP session and execute S5;

[0080] S4: Assign an ID to the new data stream and ensure that the ID does not conflict with other IDs, and then execute S5;

[0081] S5: Divide the 32-bit unsigned integer ID into four 8-bit unsigned numbers from high to low, write them into the first 4 bytes of the send buffer array, and execute S6;

[0082] S6: copy the data in the received data packet to the fourth byte after the send buffer array, that is, starting from the fifth byte, and execute S7;

[0083] S7: Set the timestamp and flag of the data packet to be sent to be the same as those of the received data packet, send the data packet to the player client, and execute S1;

[0084] In the player client program, an automatic request return object is defined to detect whether packet loss occurs in each decomposed data stream. If packet loss occurs, a retransmission request with the lost data packet sequence number is automatically sent to the server via the TCP protocol.

[0085] The automatic request retransmission object is responsible for packet loss detection and request retransmission. It has a data packet cache queue, a packet loss sequence number queue, and a stream recovery queue. The packet cache queue is used to store data packets to facilitate packet loss detection and prevent data packets from being out of order; the packet loss sequence number queue is used to record the sequence number of lost data packets; and the stream recovery queue is used to restore the fragmented H.265 data to the original H.265 stream.

[0086] Figure 1 The flowchart of the present invention for implementing RTP aggregate flow decomposition, packet loss detection and retransmission application is shown in the figure. The key steps in the flowchart are described in detail below:

[0087] S110: Initialize RTP session parameters and establish an RTP session;

[0088] S120: Waiting to receive a data packet sent by the server;

[0089] S130: When a data packet is received, the ID of the data stream to which the data packet belongs is read to determine whether it is a new data stream. If it is a new data stream, S140 is executed; otherwise, S150 is executed.

[0090] S140: creating a corresponding automatic request return object for the data stream according to the ID of the data packet, so as to detect whether the data stream has packet loss, and requesting retransmission from the server when packet loss occurs;

[0091] S150: passing the data packet to the automatic request return object, allowing the object to detect whether packet loss occurs and implement a packet loss retransmission function;

[0092] S160: Determine whether the packet loss sequence number queue of the object for automatic request backhaul is empty and whether the cache queue is greater than or equal to the minimum value. If the data flow corresponding to this object has not experienced packet loss, and the data packets in the cache queue are greater than the minimum value, it means that data has accumulated in the current data flow, and the excess data should be processed, and S170 is executed. Otherwise, either there is not enough data in the cache queue, or packet loss has occurred in the current data flow, and S180 is executed;

[0093] S170: Determine whether the sequence numbers of the first two data packets in the cache queue are continuous. If so, it means that there is no packet loss between the two data packets, and S171 is executed. Otherwise, packet loss occurs, and S172 is executed.

[0094] S171: adding the first data packet to the video code stream recovery queue and clearing it from the cache queue to recover the original video data for decoding;

[0095] S172: Record the missing data packet sequence number in the two data packets, and add the data packet sequence number to the packet loss sequence number queue;

[0096] S173: Send the missing data packet sequence number to the server, execute S120, and continue to receive data packets;

[0097] S180: Determine whether the packet loss sequence number queue of the current data flow is empty. If it is empty, it means that no packet loss is found in the current data flow, and execute S190; otherwise, execute S200;

[0098] S190: If no packet loss is found in the current data flow, the data packet is directly added to the cache queue of the automatic request return object and sorted;

[0099] S200: Determine whether the received data packet is a retransmitted data packet, if yes, execute S201, otherwise execute S210;

[0100] S201: adding the retransmitted data packet to the corresponding position in the buffer queue, and clearing the sequence number of the data packet in the packet loss sequence number queue;

[0101] S210: If the received data packet is not a retransmitted data packet, then the data packet is added to a cache queue;

[0102] S220: Determine whether the waiting time for retransmitting the data packet has timed out. The timeout period is set according to the parameters of the video stream. The present invention sets the timeout period to ensure the smoothness of video playback. That is, the inverse of the video frame rate. If it has not timed out, execute S120 and continue to wait for receiving data packets, otherwise execute S230;

[0103] S230: The retransmitted data packet may be lost again or the retransmitted data packet has not arrived yet, resulting in the waiting time having timed out and being unable to continue waiting for the retransmitted data packet. The first data packet in the cache queue is added to the code stream recovery queue, and the lost packet sequence number queue is cleared.

[0104] Figure 3 This is a flow chart of a server responding to a packet loss request provided by an embodiment of the present invention. The key steps in the flow chart are described in detail below:

[0105] Establish a TCP connection with the player client program

[0106] Waiting to receive TCP message containing packet loss sequence number message;

[0107] Search for this packet in the cache queue. If it is found, execute the next step, otherwise execute the previous step.

[0108] All found data packets are sent to the client again, and the packet type PT is set to 103 to represent resent data packets.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario, characterized in that: The steps include: Multi-channel aggregation and separation of RTP streams: The server aggregates multiple single-channel video streams to form an aggregated data stream and sends it to the player client; when the server receives multiple single-channel video streams, the server assigns an ID to each single-channel video stream; the server inserts the ID of each single-channel video stream into the data packet and sends it to the player client; The player client separates the aggregated data stream according to the ID of each data packet; After receiving the data packet, the player client obtains its ID to determine the video stream to which it belongs. If it is a new data stream, the client will create an automatic retransmission request object and pass the data packet to the object. If it is an existing data stream, the client will pass the data packet to the corresponding object. Data packet accumulation processing and packet loss detection of the object of automatic retransmission request: After receiving a new data packet, the object of automatic retransmission request determines whether there is packet loss or data packet accumulation in the current cache queue. If there is data packet accumulation, the extra data packets are added to the code stream recovery queue in sequence and deleted from the cache queue. While removing the extra data packets from the cache queue, check whether there is packet loss between data packets. If packet loss occurs, the sequence number of the lost packet is recorded and sent to the server. When packet loss is found or there is no data packet accumulation, the packet loss and retransmission processing steps of the object of automatic retransmission request are executed; Automatically request the packet loss and retransmission processing of the retransmission object: Check whether the received data packet is the server's retransmission data packet. If it is a retransmission data packet, insert the retransmission data packet into the corresponding position of the cache queue and remove it from the packet loss sequence number queue; if it is not a retransmission data packet and the waiting time has not timed out, continue to wait for the next data packet; if it times out, no longer wait for retransmission, directly send the first data packet in the cache queue to the code stream recovery queue, and clear the corresponding packet loss sequence number; Processing when there is no packet loss in the automatic retransmission request object: If there is no packet loss in the current data stream, the data packets are added to the cache queue of the automatic retransmission request object and sorted to prevent misjudgment due to disordered data packets; when the data packets in the cache queue are greater than or equal to the predetermined minimum value, they will be processed in the data packet accumulation processing and packet loss detection steps of the automatic retransmission request object in the next cycle; Server processing of packet loss request: When the server receives a packet loss request from the client, the server extracts the lost data packet sequence number contained in the request and searches for the corresponding data packet in the cache; if the corresponding data packet is found, the server marks the corresponding data packet as a retransmission packet and sends it to the client.

2. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: The specific structure of the RTP data packet includes: The version number V indicates the protocol version; the padding bit P indicates whether there is padding at the end of the data packet; the extension bit X indicates whether an extension header is included; the contribution source count CC indicates the number of CSRC fields; the mark bit M is used to mark special data packets; the payload type PT indicates the type of payload carried in the data packet; the sequence number identifies the order of the data packets; the timestamp is used to identify the sampling time of the media data; the synchronization source identifier SSRC uniquely identifies the source of the data stream in the same session; the contribution source identifier CSRC is used to identify the contribution source of the media stream generated by mixing or synthesis; the custom field data stream identifier ID is extended to distinguish different data streams; the payload represents the audio and video data; the padding is used to align the data packet to a specific block size.

3. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: The multiplexing of RTP streams specifically includes the following steps: S11, waiting to receive RTP data packets, and executing S12 after receiving the data packets; S12, obtain the SSRC of the RTP data packet, determine whether it is a new RTP session, if so, execute S13, otherwise execute S14; S13, find the ID corresponding to the RTP session SSRC, and execute S15; S14, assign an ID to the new data stream, and ensure that the ID does not conflict with other IDs, and execute S15; S15, divide the 32-bit unsigned integer ID into four 8-bit unsigned numbers from high to low, write them into the first 4 bytes of the send buffer array, and execute S16; S16, copy the data in the received data packet to the fourth byte of the send buffer array, and execute S17 starting from the fifth byte; S17, setting the timestamp and flag of the data packet to be sent to be the same as those of the received data packet, sending the data packet to the player client, and executing S11.

4. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: The automatic retransmission request object is responsible for packet loss detection and retransmission request, and the automatic retransmission request object includes a data packet cache queue, a packet loss sequence number queue, and a stream recovery queue; the data packet cache queue is used to store data packets; the packet loss sequence number queue is used to record the sequence number of the lost data packet; the stream recovery queue is used to restore the fragmented H.265 data to the original H.265 code stream.

5. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: RTP stream demultiplexing includes the following steps: S110, initializing RTP session parameters and establishing an RTP session; S120, waiting to receive a data packet sent by the server; S130, when a data packet is received, read the ID of the data stream to which the data packet belongs, and determine whether it is a new data stream. If it is a new data stream, execute S140, otherwise execute S150; S140, according to the ID of the data packet, create a corresponding automatic request return object for this data stream, which is used to detect whether this data stream has packet loss, and can request retransmission from the server when packet loss occurs; S150, passing the data packet to the automatic request return object, allowing the object to detect whether packet loss occurs and implement a packet loss retransmission function.

6. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: The data packet accumulation processing and packet loss detection of the automatic retransmission request object, the packet loss and retransmission processing of the automatic retransmission request object, and the processing of the automatic retransmission request object without packet loss specifically include the following steps: S160, determining whether the packet loss sequence number queue of the object automatically requested for return transmission is empty and whether the cache queue is greater than or equal to the minimum value; if the data flow corresponding to the object has not experienced packet loss, and the data packet in the cache queue is greater than the minimum value, executing S170; otherwise, if there is insufficient data in the cache queue or packet loss has occurred in the current data flow, executing S180; S170, determining whether the sequence numbers of the first two data packets in the cache queue are continuous, if so, it means that there is no packet loss between the two data packets, and executing S171, otherwise, packet loss occurs, and executing S172; S171, adding the first data packet to the video code stream recovery queue and clearing it from the cache queue to recover the original video data for decoding; S172, record the missing data packet sequence number in the two data packets, and add the data packet sequence number to the packet loss sequence number queue; S173, sending the missing data packet sequence number to the server, executing S120, and continuing to receive data packets; S180, determining whether the packet loss sequence number queue of the current data flow is empty, if it is empty, indicating that no packet loss is found in the current data flow, and executing S190, otherwise executing S200; S190: If no packet loss is found in the current data flow, the data packet is directly added to the cache queue of the automatic request return object and sorted; S200, determining whether the received data packet is a retransmitted data packet, if yes, executing S201, otherwise executing S210; S201, adding the retransmitted data packet to the corresponding position in the cache queue, and clearing the sequence number of the data packet in the packet loss sequence number queue; S210, if the received data packet is not a retransmitted data packet, then the data packet is added to a cache queue; S220, determining whether the waiting time for retransmitting the data packet has timed out, the timeout period being set according to the parameters of the video stream, if not timed out, executing S120, continuing to wait for receiving the data packet, otherwise executing S230; S230: Add the first data packet in the buffer queue to the code stream recovery queue, and clear the packet loss sequence number queue.

7. The method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario according to claim 1, characterized in that: The server processes packet loss requests in the following steps: Establish a TCP connection with the player client program; Waiting to receive TCP message containing packet loss sequence number message; Search for this packet in the cache queue. If it is found, execute the next step, otherwise execute the previous step. All found data packets are sent to the client again, and the packet type PT is set to 103 to represent resent data packets.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario described in any one of claims 1 to 7 is executed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the method for RTP stream multi-channel aggregation and packet loss retransmission in a network monitoring scenario as described in any one of claims 1 to 7 through the computer program.

Images