A method, device, equipment and medium for handling packet loss in video data transmission

Abstract

The present invention relates to the field of video data transmission, and discloses a method, device, equipment and medium for handling packet loss in video data transmission. In the video session established between the video receiving system and the video sending system, the present invention can determine the sequence number corresponding to the lost data packet, i.e., the sequence number to be retransmitted, through the new data packet determination condition and the packet loss determination mechanism, and instruct the video sending system to retransmit the lost video data packet to the video receiving system through the video session, so as to realize the retransmission of the lost data packet and ensure the video quality of the video receiving system.

Description

Technical Field

[0001] The present invention relates to the field of video data transmission, and in particular, to a method, device, equipment and medium for handling packet loss in video data transmission. Background Art

[0002] With the development of science and technology, video data transmission technology has been continuously improved.

[0003] When a video sender sends a video, the video content is decomposed into multiple video frames, and each video frame is further divided into multiple data packets with consecutive sequence numbers for transmission. For example, the first video frame is divided into 50 data packets with sequence numbers from 1 to 50 for transmission, and the second video frame is divided into 10 data packets with sequence numbers from 51 to 60 for transmission. The video sender sends the data packets in ascending order of the sequence numbers, such as sending the data packets with sequence numbers from 1 to 50 in sequence. When the video receiver receives the data packets, the sequence numbers of the data packets may be out of order. For example, the data packet with sequence number 10 is received first, and then the data packet with sequence number 9 is received.

[0004] Due to problems such as network congestion, hardware failure or configuration error, the video receiver may not receive some data packets, that is, packet loss occurs. When the packet loss rate is relatively large, it will affect the video quality, such as problems like mosaic and flicker.

[0005] However, the related technologies cannot effectively solve the problem of packet loss in the process of video data transmission. Summary of the Invention

[0006] The present invention provides a method, device, equipment and medium for handling packet loss in video data transmission, so as to solve the defect that the related technologies cannot effectively solve the problem of packet loss in the process of video data transmission, realize packet loss determination and packet loss retransmission, and improve the video quality.

[0007] In a first aspect, the present invention provides a method for handling packet loss in video data transmission, including:

[0008] When it is monitored that a video receiving system receives any first video data packet in a video session, at least determine the target sequence number in the first video data packet; wherein, the video session is established between the video receiving system and a video sending system;

[0009] If the target sequence number is greater than the minimum sequence number in each sequence number stored in a first queue, or the sum value of the target sequence number and a set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, determine that the first video data packet is a newly arrived data packet, and store the target sequence number in the first queue;

[0010] When it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the second smallest sequence number in the first queue from the smallest sequence number to obtain a corresponding sequence number difference;

[0011] If the sequence number difference is greater than 1, determine that a packet loss has occurred, and determine each integer greater than the smallest sequence number and less than the second smallest sequence number as a sequence number to be retransmitted;

[0012] Delete the smallest sequence number in the first queue, and instruct the video sending system to retransmit the video data packet corresponding to each sequence number to be retransmitted to the video receiving system through the video session.

[0013] Optionally, when it is monitored that the video receiving system receives any first video data packet in the video session, at least determine the target sequence number in the first video data packet, including:

[0014] When it is monitored that the video receiving system and the video sending system have completed the establishment of the video session, obtain the session information of the video session; wherein, the session information of the video session includes the address information and port information of the video receiving system, and the address information and port information of the video sending system;

[0015] Monitor the data received by the video receiving system in the video session according to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information;

[0016] When it is monitored that the video receiving system receives any first video data packet in the video session, at least determine the target sequence number in the first video data packet.

[0017] Optionally, before it is monitored that the video receiving system receives any first video data packet in the video session, the method further includes:

[0018] Create an empty queue and use the empty queue as the first queue;

[0019] After at least determining the target sequence number in the first video data packet, if no sequence number is stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, determine that the first video data packet is a newly arrived data packet, and store the target sequence number in the first video data packet into the first queue.

[0020] Optionally, at least determining the target sequence number in the first video data packet includes:

[0021] Determine the target sequence number in the first video data packet and the receiving time of the first video data packet by the video receiving system;

[0022] After determining that the first video data packet is a newly sent data packet, the method further includes:

[0023] Correspondingly save the target sequence number in the first video data packet and the receiving time;

[0024] When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the second smallest sequence number in the first queue from the smallest sequence number to obtain a corresponding sequence number difference, including:

[0025] When it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, or the receiving duration corresponding to the smallest sequence number exceeds a preset duration threshold, subtract the second smallest sequence number from the smallest sequence number to obtain the sequence number difference;

[0026] Wherein, the receiving duration is obtained by subtracting the receiving time corresponding to the smallest sequence number from the current time.

[0027] Optionally, the video sending system includes a media service and a monitoring service; the media service is used to send a plurality of second video data packets to the video receiving system through the video session; the monitoring service is used to monitor that the media service sends any one of the second video data packets in the video session, and store the monitored second video data packet into a created second queue;

[0028] The instruction to make the video sending system retransmit the video data packet corresponding to each retransmission sequence number to the video receiving system through the video session includes:

[0029] Generate a data packet retransmission instruction according to each retransmission sequence number, and each retransmission sequence number is included in the data packet retransmission instruction;

[0030] Send the data packet retransmission instruction to the monitoring service, so that the monitoring service: respectively find the corresponding second video data packet in the second queue according to each retransmission sequence number in the data packet retransmission instruction, and send each found second video data packet to the video receiving system through the video session.

[0031] Optionally, after monitoring the data received by the video receiving system in the video session, the method further includes:

[0032] Continuously detect the session state of the video session;

[0033] When it is detected that the session state of the video session is session termination, stop monitoring the data received by the video receiving system in the video session.

[0034] Optionally, after at least determining the target sequence number in the first video data packet, the method further includes:

[0035] If the target sequence number is not greater than the minimum sequence number, and the sum value of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, determine that the first video data packet is a retransmitted data packet, and prohibit storing the target sequence number into the first queue.

[0036] In a second aspect, the present invention provides a packet loss handling device in video data transmission, including

[0037] A monitoring unit, configured to monitor any first video data packet received by a video receiving system in a video session, and at least determine a target sequence number in the first video data packet; wherein, the video session is established between the video receiving system and a video sending system;

[0038] A first determination unit, configured to determine that the first video data packet is a newly sent data packet if the target sequence number is greater than the minimum sequence number in each sequence number stored in a first queue, or the sum value of the target sequence number and a set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue;

[0039] A storage unit, configured to store the target sequence number into the first queue;

[0040] A monitoring unit, configured to, when it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the second smallest sequence number in the first queue from the minimum sequence number to obtain a corresponding sequence number difference;

[0041] A second determination unit, configured to determine that packet loss occurs if the sequence number difference is greater than 1;

[0042] A third determination unit, configured to determine each integer greater than the minimum sequence number and less than the second smallest sequence number as a sequence number to be retransmitted;

[0043] A deletion unit, configured to delete the minimum sequence number in the first queue;

[0044] An instruction unit, configured to instruct the video sending system to retransmit video data packets corresponding to each of the sequence numbers to be retransmitted to the video receiving system through the video session.

[0045] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the packet loss handling method in video data transmission according to the first aspect or any corresponding embodiment thereof.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to perform the packet loss handling method in video data transmission according to the first aspect or any corresponding embodiment thereof.

[0047] For the packet loss handling method, device, equipment and medium in video data transmission provided by the present invention, when it is monitored that any first video data packet is received by the video receiving system in a video session, at least the target sequence number in the first video data packet is determined. If the target sequence number is greater than the minimum sequence number in the first queue, or the sum of the target sequence number and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly arrived packet, and the target sequence number is stored in the first queue. When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, the second smallest sequence number in the first queue is subtracted from the minimum sequence number in the first queue to obtain a corresponding sequence number difference. If the sequence number difference is greater than 1, it is determined that a packet loss has occurred, and each integer greater than the minimum sequence number in the first queue and less than the second smallest sequence number in the first queue is determined as a sequence number to be retransmitted. The minimum sequence number in the first queue is deleted, and the video sending system is instructed to retransmit the video data packets corresponding to each sequence number to be retransmitted to the video receiving system through the video session. The present invention can, in the video session established between the video receiving system and the video sending system, determine the sequence numbers corresponding to the lost data packets, i.e., the sequence numbers to be retransmitted, through the newly arrived packet determination condition and the packet loss determination mechanism, and instruct the video sending system to retransmit the lost video data packets to the video receiving system through the video session, so as to retransmit the lost data packets and ensure the video quality of the video receiving system. Description of the Drawings

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a flowchart of a packet loss handling method in video data transmission provided by an embodiment of the present invention;

[0050] Figure 2It is a signaling interaction diagram of a superior-subordinate video monitoring system provided by an embodiment of the present invention;

[0051] Figure 3 It is another signaling interaction diagram of a superior-subordinate video monitoring system provided by an embodiment of the present invention;

[0052] Figure 4 It is a working flowchart of a superior SIP signaling monitoring service provided by an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the session information content of a video session provided by an embodiment of the present invention;

[0054] Figure 6 It is a working flowchart of a superior video reception monitoring service provided by an embodiment of the present invention;

[0055] Figure 7 It is a working flowchart of a subordinate video transmission monitoring service provided by an embodiment of the present invention;

[0056] Figure 8 It is a schematic structural diagram of a packet loss processing device in video data transmission provided by an embodiment of the present invention;

[0057] Figure 9 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0058] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0059] The following combines Figures 1-7 to describe the packet loss processing method in video data transmission of the present invention.

[0060] As Figure 1 shown, the first packet loss processing method in video data transmission is proposed in this embodiment, and the method may include the following steps:

[0061] S101. When it is monitored that a video receiving system receives any first video data packet in a video session, at least determine the target sequence number in the first video data packet; wherein, the video session is established between the video receiving system and the video sending system.

[0062] Among them, the first video data packet is the video data packet received by the video receiving system during the video session and sent by the video sending system.

[0063] Specifically, the target sequence number refers to the sequence number carried in the first video data packet.

[0064] It should be noted that the video receiving system and the video sending system can establish a video session and transmit video streams through the video session. Specifically, the video receiving system and the video sending system can establish a video session by sending the Session Initialization Protocol (SIP). In the established video session, video data is transmitted. The video sending system can use the Real-Time Transport Protocol (RTP) in the video session to send multiple video data packets to the video receiving system in ascending order of the sequence number. For example, in ascending order of the sequence number, the video sending system sends video data packets with sequence numbers from 1 to 50 to the video receiving system. The video receiving system can receive the video data packets sent by the video sending system in the video session. Each video data packet carries a sequence number. The sequence numbers of the video data packets received by the video receiving system may be out of order. For example, the video receiving system first receives a video data packet with a sequence number of 8, then receives a video data packet with a sequence number of 6, and then receives a video data packet with a sequence number of 10.

[0065] Specifically, this embodiment can monitor the first video data packet received by the video receiving system in the video session. Whenever it is monitored that the video receiving system receives the first video data packet in the video session, the first video data packet is parsed to obtain the target sequence number carried in the first video data packet.

[0066] S102: If the target sequence number is greater than the minimum sequence number among each sequence number stored in the first queue, or the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly arrived data packet.

[0067] Among them, the first queue can be the queue created by this embodiment for storing the sequence numbers of one or more newly arrived data packets in the video session.

[0068] It should be noted that the first video data packet received by the video receiving system in the video session can be divided into two types: retransmitted data packets and newly arrived data packets. The retransmitted data packet is the first video data packet that has been sent before in the video session and is sent again. The newly arrived data packet is the first video data packet that has not been sent before and is sent for the first time in the video session.

[0069] In practical applications, the sequence numbers of the various video data packets received by the video receiving system in a video session may not be strictly sorted in ascending order, but in disorder. The degree of disordered offset of the sequence numbers generally does not exceed a specific range.

[0070] Specifically, the maximum disordered positive offset is the maximum offset of the sequence number when the video data packet is in disorder. In this embodiment, it can be set by those skilled in the art according to the actual situation. For example, it can be set to 6, or it can be set to 5.

[0071] In practical applications, when the total number of sequence numbers stored in the first queue is relatively small, the maximum sequence number and the minimum sequence number in the first queue are relatively close. At this time, if the sum obtained by adding the target sequence number in the first video data packet to the maximum disordered positive offset is greater than the maximum sequence number in the first queue, it can be determined that the first video data packet is a newly arrived data packet. When the total number of sequence numbers stored in the first queue is relatively large, the difference between the maximum sequence number and the minimum sequence number in the first queue is relatively large. At this time, if the target sequence number in the first video data packet is greater than the minimum sequence number in the first queue, it can be determined that the first video data packet is a newly arrived data packet.

[0072] Specifically, in this embodiment, a judgment condition for determining that the first video data packet is a newly arrived data packet can be set. This judgment condition includes a first condition and a second condition. Among them, the first condition is that the target sequence number in the first video data packet is greater than the minimum sequence number in the first queue. The second condition is that the sum of the target sequence number and the maximum disordered positive offset is greater than the maximum sequence number in the first queue.

[0073] It can be understood that the first condition is applicable to the judgment of newly arrived data packets when the total number of sequence numbers in the first queue is relatively large, and the second condition is applicable to the judgment of newly arrived data packets when the total number of sequence numbers in the first queue is relatively small. As long as the target sequence number in the first video data packet satisfies any one of the first condition and the second condition, this embodiment can determine that the first video data packet is a newly arrived data packet.

[0074] Optionally, in the packet loss handling method for other video data transmissions proposed in this embodiment, after the above step S101, it may further include:

[0075] If the target sequence number in the first video data packet is not greater than the minimum sequence number in the first queue, and the sum of the target sequence number in the first video data packet and the maximum disordered positive offset is not greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a retransmitted data packet, and the target sequence number is prohibited from being stored in the first queue.

[0076] It can be understood that if the target sequence numbers in the first video data packet do not meet the first condition and the second condition, that is, the target sequence number is not greater than the minimum sequence number in the first queue, and the sum value of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number in the first queue, it can be determined that the first video data packet is a retransmitted data packet, and it is prohibited to store the target sequence number in the first video data packet into the first queue.

[0077] S103. Store the target sequence number into the first queue.

[0078] Specifically, in this embodiment, after determining that the first video data packet is a newly arrived data packet, the target sequence number in the first video data packet can be stored into the first queue.

[0079] S104. When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the second smallest sequence number in the first queue from the smallest sequence number to obtain the corresponding sequence number difference.

[0080] It can be understood that in this embodiment, whenever it is determined that the first video data packet received by the video receiving system is a newly arrived data packet, the target sequence number in the first video data packet is stored into the first queue. Therefore, the sequence numbers stored in the first queue are in dynamic change, and the size sorting of each sequence number in the first queue is also in dynamic change. The smallest sequence number in the first queue is the smallest sequence number among the currently stored sequence numbers in the first queue. The second smallest sequence number in the first queue is the second smallest sequence number among the currently stored sequence numbers in the first queue.

[0081] It should be noted that monitoring whether the total number of sequence numbers in the first queue reaches the preset threshold is the starting condition for packet loss determination. Specifically, in this embodiment, it can continuously monitor whether the total number of sequence numbers in the first queue reaches the preset threshold. If the total number of sequence numbers in the first queue does not reach the preset threshold, it can continue to monitor. If it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, the smallest sequence number and the second smallest sequence number in the first queue can be determined, and the second smallest sequence number is subtracted from the smallest sequence number to obtain the corresponding sequence number difference.

[0082] Among them, the preset threshold can be set by technicians according to the actual situation, and this embodiment does not make a limitation. Optionally, the preset threshold can be equal to the sequence number capacity in the first queue. At this time, in this embodiment, when it is monitored that the first queue is full of stored sequence numbers, the second smallest sequence number in the first queue can be subtracted from the smallest sequence number.

[0083] To better introduce step S104, this embodiment presents Example 1 for illustration.

[0084] Example 1: The above preset threshold is 11. The first queue has stored 10 serial numbers, which are 20, 23, 24, 25, 26, 27, 28, 29, 30, and 31 respectively. When the monitoring video receives the first video data packet with the serial number 32, the serial number 32 is stored in the first queue. At this time, the smallest serial number in the first queue is 20, and the second smallest serial number is 23. In this embodiment, the serial number difference can be obtained by subtracting 20 from 23, which is 3.

[0085] S105: If the serial number difference is greater than 1, it is determined that a packet loss has occurred.

[0086] Specifically, in this embodiment, the obtained serial number difference can be compared with 1 in size, and whether a packet loss has occurred can be determined according to the comparison result. Among them, if the serial number difference is greater than 1, it can be determined that a packet loss has occurred. If the serial number difference is not greater than 1, it can be determined that no packet loss has occurred. Continuing with the above Example 1, since the serial number difference is greater than 3, it can be determined that a packet loss has occurred.

[0087] S106: Each integer greater than the smallest serial number in the first queue and less than the second smallest serial number in the first queue is determined as a retransmission serial number.

[0088] Specifically, in this embodiment, when it is determined that a packet loss has occurred, each integer greater than the smallest serial number in the first queue and less than the second smallest serial number in the first queue is determined as a retransmission serial number. Continuing with the above Example 1, in this embodiment, each integer greater than 20 and less than 23, that is, 21 and 22, can be determined as retransmission serial numbers.

[0089] It can be understood that the video data packet corresponding to each retransmission serial number is a video data packet that is lost in the video session and not received by the video receiving system.

[0090] S107: Delete the smallest serial number in the first queue.

[0091] Specifically, in this embodiment, after determining the retransmission serial numbers, the smallest serial number in the first queue can be deleted from the first queue. Continuing with the above Example 1, in this embodiment, the serial number 20 can be deleted from the first queue.

[0092] S108: Instruct the video sending system to retransmit the video data packets corresponding to each retransmission serial number to the video receiving system through the video session.

[0093] Specifically, in this embodiment, a retransmission instruction can be sent to the video sending system so that the video sending system retransmits the video data packets corresponding to each retransmission serial number to the video receiving system through the video session.

[0094] In the packet loss handling method for video data transmission proposed in this embodiment, when it is monitored that any first video data packet is received by the video receiving system during a video session, at least the target sequence number in the first video data packet is determined. If the target sequence number is greater than the minimum sequence number in the first queue, or the sum of the target sequence number and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly arrived data packet, and the target sequence number is stored in the first queue. When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, the second smallest sequence number in the first queue is subtracted from the minimum sequence number in the first queue to obtain the corresponding sequence number difference. If the sequence number difference is greater than 1, it is determined that packet loss has occurred, and each integer greater than the minimum sequence number in the first queue and less than the second smallest sequence number in the first queue is determined as a sequence number to be retransmitted. The minimum sequence number in the first queue is deleted, and the video sending system is instructed to retransmit the video data packets corresponding to each sequence number to be retransmitted to the video receiving system through the video session. In this embodiment, in the video session established between the video receiving system and the video sending system, the sequence numbers corresponding to the lost data packets, that is, the sequence numbers to be retransmitted, can be determined through the set newly arrived data packet determination condition and packet loss determination mechanism, and the video sending system is instructed to retransmit the lost video data packets to the video receiving system through the video session, so as to effectively reduce the packet loss rate and ensure the video quality of the video receiving system.

[0095] Based on Figure 1 , the second packet loss handling method for video data transmission is proposed in this embodiment. In this method, step S101 may include:

[0096] When it is monitored that the video receiving system and the video sending system have completed the establishment of a video session, the session information of the video session is obtained; wherein, the session information of the video session includes the address information and port information of the video receiving system, and the address information and port information of the video sending system;

[0097] According to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information, the data received by the video receiving system during the video session is monitored;

[0098] When it is monitored that the video receiving system receives any first video data packet during the video session, at least the target sequence number in the first video data packet is determined.

[0099] Specifically, this embodiment can monitor the session behavior between the video receiving system and the video sending system. When it is determined that the video session is established between the two, the session information of the video session is obtained. According to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information, the data receiving behavior of the video receiving system in the video session and the data it receives are monitored.

[0100] Among them, after obtaining the session information of the video session, this embodiment can input the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information into the created monitoring task to start the monitoring task and enable the monitoring task to monitor the data received by the video receiving system in the video session according to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information. When this embodiment monitors that the video receiving system receives any first video data packet in the video session through the monitoring task, the target sequence number in the first video data packet is determined.

[0101] Optionally, in other packet loss handling methods in video data transmission proposed in this embodiment, before monitoring that the video receiving system receives any first video data packet in the video session, the method further includes:

[0102] Create an empty queue and use the empty queue as the first queue.

[0103] After at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, it is determined that the first video data packet is a newly sent packet, and the target sequence number in the first video data packet is stored in the first queue.

[0104] Specifically, this embodiment can create an empty queue as the first queue. Whenever it is determined that a first video data packet is a newly sent packet, the target sequence number in the first video data packet is stored in the first queue.

[0105] It can be understood that the first first video data packet received by the video receiving system during a video session is a newly sent data packet. Before the video receiving system receives the first first video data packet, no sequence numbers are stored in the first queue. In this embodiment, when it is monitored that the video receiving system receives the first first video data packet during a video session, it can be directly determined that the first video data packet is a newly sent data packet, and the target sequence number in the first video data packet is stored in the first queue. In this embodiment, when it is monitored that the video receiving system receives a video data packet after the first video data packet during a video session, it can be determined whether the video data packet is a newly sent data packet or a retransmitted data packet based on the above first condition and second condition.

[0106] Optionally, after monitoring the data received by the video receiving system during a video session, the method may further include:

[0107] Continuously detect the session state of the video session;

[0108] When it is detected that the session state of the video session is session termination, stop monitoring the data received by the video receiving system during the video session.

[0109] Specifically, in this embodiment, when it is monitored that the video session terminates, the data reception behavior of the video receiving system through the video session can be stopped from being monitored, reducing the unnecessary consumption of monitoring resources.

[0110] The packet loss handling method in video data transmission proposed in this embodiment can monitor the session behavior between the video receiving system and the video sending system. When it is monitored that the two establish a video session, obtain the session information of the video session, and monitor the first video data packets received by the video receiving system during the video session according to the session information, effectively realizing the monitoring of the first video data packets, and can identify each newly sent data packet received by the video receiving system, and store the sequence numbers in each newly sent data packet using the created first queue to effectively ensure the implementation of the packet loss determination mechanism.

[0111] Based on Figure 1 , a third packet loss handling method in video data transmission is proposed in this embodiment. In this method, step S101 may include:

[0112] When it is monitored that the video receiving system receives any first video data packet during a video session, determine the target sequence number in the first video data packet and the reception time of the first video data packet by the video receiving system.

[0113] At this time, after step S102, the method may further include:

[0114] Correspondingly save the target sequence number and reception time in the first video data packet.

[0115] At this time, the above step S104 may include:

[0116] When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, or the reception duration corresponding to the smallest sequence number in the first queue exceeds a preset duration threshold, subtract the second smallest sequence number in the first queue from the smallest sequence number in the first queue to obtain a sequence number difference;

[0117] Wherein, the reception duration is obtained by subtracting the reception time corresponding to the smallest sequence number in the first queue from the current moment.

[0118] Specifically, in this embodiment, the target sequence number in each first video data packet can be determined, as well as the reception time thereof by the video reception system, and the target sequence number and the reception time in the first video data packet are correspondingly saved.

[0119] Among them, the reception time corresponding to the smallest sequence number in the first queue is the reception time of the first video data packet corresponding to the smallest sequence number by the video reception system. In this embodiment, the current moment can be subtracted from this reception time to obtain the reception duration corresponding to the smallest sequence number.

[0120] At this time, this embodiment can optimize the start condition of the packet loss determination, monitor whether the total number of sequence numbers in the first queue reaches a preset threshold, and monitor whether the reception duration corresponding to the smallest sequence number in the first queue exceeds a preset duration threshold. When it is determined that the total number of sequence numbers in the first queue reaches a preset threshold, or the reception duration corresponding to the smallest sequence number in the first queue exceeds a preset duration threshold, this embodiment can start the packet loss determination process, subtract the second smallest sequence number in the first queue from the smallest sequence number to obtain the corresponding sequence number difference.

[0121] It should be noted that the preset duration threshold can be set by those skilled in the art according to actual situations, and this embodiment does not make any limitations.

[0122] Among them, in this embodiment, when it is determined that the total number of sequence numbers in the first queue does not reach a preset threshold, and the reception duration corresponding to the smallest sequence number in the first queue does not exceed a preset duration threshold, the packet loss determination process can be not started, and it is prohibited to subtract the second smallest sequence number in the first queue from the smallest sequence number.

[0123] The packet loss processing method in video data transmission proposed by this embodiment can optimize the start condition content of the packet loss determination mechanism, enrich the packet loss determination scenarios, and improve the packet loss recognition accuracy.

[0124] It should be noted that the video receiving system and the video sending system in this embodiment can be the upper-level video monitoring system and the lower-level video monitoring system respectively. The media service in the lower-level video monitoring system can directly send video data packets to the upper-level video monitoring system. In some scenarios, this embodiment cannot directly send instructions or requests to the media service in the lower-level video monitoring system. At this time, when this embodiment determines that packet loss occurs, it cannot directly request or instruct the media service in the lower-level video monitoring system to retransmit the lost video data packets to the video receiving system. For example, in GB / T28181, the Real-time Transport Control Protocol (RTCP) defined in Request for Comments (RFC) 3550 of the Internet Engineering Task Force (IETF) is adopted, but RFC3550 does not stipulate packet loss retransmission and does not mandatorily require the implementation of packet loss retransmission. The IETF recommends using Negative Acknowledgement (NACK) defined in RFC4585 for packet loss retransmission. In the actual environment, the upper-level and lower-level video monitoring systems may be developed by different manufacturers, and some manufacturers do not implement the NACK mechanism and cannot achieve packet loss retransmission.

[0125] Based on Figure 1 , this embodiment proposes a fourth method for handling packet loss in video data transmission. In this method, the video sending system includes a media service and a monitoring service; the media service is used to send a plurality of second video data packets to the video receiving system through a video session; the monitoring service is used to monitor that any second video data packet is sent by the media service in the video session, and store the monitored second video data packet into a created second queue.

[0126] At this time, step S108 may include:

[0127] Generate a data packet retransmission instruction according to each sequence number to be retransmitted. The data packet retransmission instruction includes each sequence number to be retransmitted;

[0128] Send the data packet retransmission instruction to the monitoring service, so that the monitoring service: respectively find the corresponding second video data packet in the second queue according to each sequence number to be retransmitted in the data packet retransmission instruction, and send each found second video data packet to the video receiving system through the video session.

[0129] Among them, the second video data packet is a video data packet sent by the media service to the video receiving system based on the receiving address and port information in the video session.

[0130] Specifically, the monitoring service can monitor the session behavior between the media service and the video receiving system. When it monitors that a video session is established between the two, it starts to monitor the data sending behavior of the media service in the video session and the data it sends. Whenever the monitoring service monitors that the media service sends a second video data packet in the video session, it stores the second video data packet in the second queue.

[0131] Specifically, in this embodiment, after determining each sequence number to be retransmitted, a data packet retransmission instruction carrying all the sequence numbers to be retransmitted can be generated, and the data packet retransmission quality is sent to the monitoring service in the video sending system. The monitoring service can respond to the data packet retransmission instruction, parse the data packet retransmission instruction, obtain all the sequence numbers to be retransmitted in the data packet retransmission instruction, and find the second video data packet corresponding to each sequence number to be retransmitted in the second queue, that is, the data packet to be retransmitted. Then, the monitoring service can send the found second video data packet corresponding to each sequence number to be retransmitted to the video receiving system according to the receiving address and port information in the video session.

[0132] Optionally, the above monitoring service can also be used to: for any second video data packet sent by the media service monitored in the video session, determine the sequence number in the second video data packet and the sending time of the media service for the second video data packet, and correspondingly save the sequence number and the sending time.

[0133] Specifically, the monitoring service can also monitor the sending duration corresponding to each second video data packet in the second queue. The sending duration corresponding to the second video data packet is obtained by subtracting the sending time corresponding to the second video data packet from the current time. When it is determined that the sending duration corresponding to any second video data packet exceeds the set duration threshold, it is determined that the second video data packet no longer belongs to the range of lost packet retransmission, and the second video data packet is deleted in the second queue, releasing the storage space of the second queue and reducing the unnecessary consumption of storage resources.

[0134] It can be understood that the set duration threshold is not less than the above preset duration threshold, ensuring that all lost packets of the video receiving system can be found in the second queue, effectively guaranteeing the search for lost data packets and the subsequent smooth implementation of lost packet retransmission.

[0135] The packet loss handling method in video data transmission proposed in this embodiment sets up a media service and a monitoring service in the video sending system, creates a second queue, and uses the second queue to store each second video data packet sent by the media service. When it is determined that packet loss occurs in the video receiving system, the monitoring service can be instructed to find the second video data packet corresponding to the retransmission sequence number in the second queue, and the monitoring service is instructed to retransmit each found second video data packet to the video receiving system through the video session, effectively realizing the retransmission of lost data packets and ensuring the video quality at the video receiving end.

[0136] In practical applications, the video receiving system and the video sending system in this embodiment can be different video monitoring systems. Specifically, the video receiving system can be a superior video monitoring system, and the video sending system can be an inferior video monitoring system.

[0137] In the related art, with the large-scale application and popularization of video monitoring systems, the demand for cascading and interconnection between video monitoring systems is increasing. The introduction of relevant standards has unified the docking interfaces of video monitoring systems. During construction, the technical levels of different manufacturers vary. Some platforms are not fully considered for the application scale during construction. With the reasons such as the expansion of the scale of cameras, the increase of camera bitrates, and the expansion of the application scope, the packet loss rate in network transmission increases, affecting the viewing quality. Especially for some video monitoring systems, it is even difficult to upgrade the software to improve due to some historical reasons. This embodiment can solve this problem without changing the software and hardware of the original video monitoring system, realizing the retransmission of lost packets and thus improving the viewing effect.

[0138] Such as Figure 2As shown in the figure, the upper-level video monitoring system in the related art may include an upper-level SIP service and an upper-level media service, and the lower-level video monitoring system may include a lower-level SIP service and a lower-level media service. When the upper and lower video monitoring systems are docked, they need to be connected to each other on the network and may pass through multiple network devices. When the two systems are docked, the SIP protocol is used for signaling interaction, and the RTP protocol is used for transmitting media streams. After the lower-level video monitoring system registers with the upper-level video monitoring system, the lower-level video monitoring system reports relevant video resources and directory resources to the upper-level video monitoring system, and the upper-level video monitoring system can then call the lower-level video monitoring video for viewing. When the upper-level video monitoring system calls a video, it sends an INVITE transaction request to the lower-level video monitoring system, and the upper and lower levels establish a video session. The RTP protocol information is exchanged through the Session Description Protocol (SDP), including using the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP), the IP and port information of the sender and receiver, etc. The lower-level media service then sends video and audio streams to the corresponding upper-level media service. Figure 2 The INVITE request, response OK, and confirmation ACK in Figure 2 are the interaction signaling between the upper and lower video monitoring systems during the establishment of a video session, and the BYE request and OK are the interaction signaling between the upper and lower video monitoring systems when closing the video session.

[0139] In the related art, the RTP transmission of media streams can use three methods: UDP, TCP active, and TCP passive. When using the TCP method to transmit video streams, although the reliability can be guaranteed, it will also cause a large delay and exacerbate congestion, which may in turn cause video stuttering in the upper-level video monitoring system when the network is poor. When using the UDP method, many manufacturers have not developed the QoS mechanism for RTP. In this way, in some video monitoring system environments, in the initial stage after construction, due to the small network environment pressure and low network packet loss rate, both UDP and TCP transmissions are normal. However, as the number of access devices, docking systems, and video users increases, the network pressure increases, and there will be a phenomenon that the UDP transmission has a frozen screen due to packet loss, while the TCP transmission has stuttering and frame skipping due to too many concurrent connections. Moreover, since the upper and lower monitoring systems belong to different manufacturers, the upper-level monitoring system may not be able to directly request the lower-level media service in the lower-level video monitoring system to retransmit the lost packets.

[0140] Specifically, this embodiment can reinforce the media stream transmission of the video monitoring system based on the raw socket method. Without upgrading the software of the original video monitoring system, when the video monitoring system uses the UDP method for transmission, it retransmits the lost packets after the video monitoring system is docked, thereby reducing the frozen screen.

[0141] Among them, in this embodiment, the raw socket technology can be used to receive data packets on the network card, so as to monitor and analyze the signaling data packets and video data packets in the video session of the upper-level video monitoring system and the lower-level video monitoring system. As Figure 3 shown, this embodiment can add 4 modules: the upper-level SIP signaling monitoring service, the upper-level video reception monitoring service, the lower-level SIP signaling monitoring service, and the lower-level video transmission monitoring service. The red part is the modules and execution processes added in this embodiment compared with the related technology.

[0142] Specifically, the upper-level SIP signaling monitoring service can monitor the SIP signaling of the upper-level SIP service and control the start and stop of the tasks of the upper-level video reception monitoring service. The upper-level video reception monitoring service can monitor the RTP packets received by the upper-level media service, timely detect the situation of not receiving RTP packets, and send a retransmission notice. The lower-level SIP signaling monitoring service can monitor the SIP signaling of the lower-level media service and control the start and stop of the tasks of the lower-level video transmission monitoring service. The lower-level video transmission monitoring service can monitor the RTP packets sent by the lower-level media service and save them for a certain period of time. After receiving the retransmission notice, it will retransmit the corresponding RTP packets to the upper-level media service.

[0143] This embodiment can work together with the existing video monitoring system in the related technology. The existing video monitoring system does not need to upgrade the software. Only the software components of this embodiment, that is, the above 4 modules, need to be deployed on the corresponding servers of the existing video monitoring system.

[0144] Combined with Figure 3 shown, in the scenario where the upper and lower level video monitoring systems transmit video data packets through a video session, the video session in this embodiment can specifically be a video session established between the upper-level media service and the lower-level media service. The first video data packet is the video data packet received by the upper-level media service from the lower-level media service in the video session, and the second video data packet is the video data packet sent by the lower-level media service to the upper-level media service in the video session. This embodiment can regard the upper-level video reception monitoring service and the upper-level SIP signaling monitoring service as a whole. This embodiment can be applied to this whole, and the whole will execute as Figure 1The various processes shown implement the packet loss determination and packet loss retransmission processes. Specifically, in this embodiment, the packet loss mechanism can be set to determine and identify the packet loss and the sequence number to be retransmitted during the reception of the first video data packet by the upper-level media service. The sequence number to be retransmitted is sent to the lower-level video transmission monitoring service, so that the lower-level video transmission monitoring service can find the corresponding second video data packet according to the sequence number to be retransmitted and retransmit it to the upper-level media service, realizing packet loss retransmission, reducing the packet loss rate, improving the UDP-based video transmission quality of the upper and lower video monitoring systems, and thus improving the user experience. Among them, the lower-level video transmission monitoring service can be regarded as the monitoring service in the lower-level video monitoring system.

[0145] Specifically, the upper-level SIP signaling monitoring service can be deployed on the server where the upper-level SIP service is located. Through the raw socket method, the SIP data sent by the upper-level SIP service can be read from the network card, and the data sent to the lower-level SIP service and port can be filtered. When an INVITE request and its response are parsed, the key fields in the SDP message carried by it are parsed (for example, the receiving IP is parsed from the c field, the video receiving port is parsed from the m field, the SSRC is parsed from the y field, and RTP / AVP in the m field indicates the use of UDP), and a session monitoring is prepared to be started. After the ACK of the INVITE request is parsed, the information related to the video stream to be monitored is sent to the upper-level video reception monitoring service, so that the upper-level video reception monitoring service can perform session monitoring according to this information.

[0146] As Figure 4 As shown, in this embodiment, the upper-level SIP signaling monitoring service can monitor the SIP signaling of the upper-level SIP service, receive data packets, and filter data packets according to the configuration information. When it is detected that the upper-level SIP service receives an INVITE request, SIP and SDP are parsed. When it is detected that the upper-level SIP service receives an INVITE response, SIP and SDP are parsed. When it is detected that the upper-level SIP service receives an ACK, SIP is parsed. A start video monitoring command is sent to the upper-level video reception monitoring service. When it is detected that the upper-level SIP service receives a BYE request, SIP is parsed, and a stop video monitoring command is sent to the upper-level video reception monitoring service. Prepare for the next round of monitoring tasks.

[0147] Among them, the content in the information related to the video stream is the session information of the video session, such as Figure 5As shown, the Call-ID is used to uniquely identify a video session. The VideoReceiverIP is the device IP address of the upper-level media service, the VideoReceiverPort is the port of the upper-level media service, the VideoSenderrIP is the device IP address of the lower-level media service, and the VideoSenderPort is the port of the lower-level media service. The VideoSSRC is the synchronization source identifier, which is used to uniquely identify a media stream in the RTP protocol. When the upper-level SIP signaling monitoring service parses a BYE request, it notifies the upper-level video reception monitoring service to stop monitoring this session.

[0148] Specifically, the upper-level video reception monitoring service can be deployed on the server where the upper-level media service is located. It parses and filters RTP data packets according to the UDP port and SSRC given by the upper-level SIP signaling monitoring service. According to the sequence number and packet loss judgment mechanism of the RTP packets, when a packet loss is detected, it notifies the lower-level video transmission monitoring service.

[0149] As Figure 6 shown, in this embodiment, the upper-level video reception monitoring service can be started. When it receives a notification from the upper-level SIP signaling monitoring service, it starts a video data monitoring session. It receives UDP packets, filters them according to the received IP and port, and parses the RTP. Then, it performs a packet loss judgment. If a packet loss is determined, it sends an RTP retransmission notification. This round of monitoring stops until a stop notification is received.

[0150] Specifically, during the process of judging the packet loss mechanism by the upper-level video reception monitoring service, each RTP packet carries a sequence number, which can be a cyclic sequence from 0 to 65535. In the actual process, since the received RTP may be out of order, it cannot be simply judged according to the order. In this embodiment, it can rely on time and queue length to judge whether there is a packet loss. Create a first queue. When it monitors that the upper-level media service receives each video data packet in the video session, it extracts the RTP sequence number and records the reception time. If this sequence number is larger than the smallest sequence number in the first queue, or the sum of this sequence number and the maximum out-of-order positive offset is greater than the largest sequence number in the first queue, then it determines that this video data packet is a newly arrived data packet, puts the sequence number in this video data packet into the first queue for storage, and can ensure that the sequence numbers in the first queue are sorted from small to large. If the total number of sequence numbers in the first queue reaches the preset threshold N, or the duration from the reception time corresponding to the smallest sequence number in the first queue to the current time exceeds the preset duration threshold, then subtract the second smallest sequence number from the smallest sequence number in the first queue to get the corresponding difference. If the difference is greater than 1, indicating a packet loss, then it is considered that there is a packet loss, and each integer greater than the smallest sequence number in the first queue and less than the second smallest sequence number is determined as the sequence number corresponding to the lost data packet, that is, the sequence number to be retransmitted.

[0151] Specifically, the lower-level SIP signaling monitoring service can be deployed on the server where the lower-level SIP service is located. Its execution process can refer to the execution process of the upper-level SIP signaling monitoring service. The lower-level SIP signaling monitoring service can send the information related to the video stream to be monitored, that is, the session information of the video session, to the lower-level video sending monitoring service, so that the lower-level video sending monitoring service can perform session monitoring based on this information.

[0152] Specifically, the lower-level video sending monitoring service can be deployed on the server where the lower-level media service is located. It resolves and filters RTP data packets according to the UDP port and synchronization source identifier given by the lower-level SIP signaling monitoring service, records the time of the RTP packet data, stores it in a queue, and sets a timeout value. If the timeout occurs, it discards the packet. The timeout time should not be less than the packet loss detection timeout time of the upper-level video receiving monitoring service. When a request for retransmitting an RTP packet is received, the corresponding RTP packet is found from the queue according to the lost packet sequence number and sent to the upper-level media service.

[0153] Such as Figure 7 As shown, in this embodiment, the lower-level video sending monitoring service can be started. After receiving the notice sent by the lower-level SIP signaling monitoring service, a video data monitoring session is started, UDP packets are received, RTP is filtered and parsed, and stored in a queue. It is determined whether a packet loss notice is received. If so, the RTP is taken out and sent according to the sequence number, and the data timeout is detected, and the timeout data in the queue is discarded. Until a stop monitoring notice is received, this round of monitoring stops.

[0154] The packet loss handling method in video data transmission proposed in this embodiment can be applied to detecting the video data packets lost in the video data transmission between the upper-level video monitoring system and the lower-level video monitoring system, determining the sequence number to be retransmitted, and can instruct the lower-level video sending monitoring service in the lower-level video monitoring system to retransmit the lost packets to the upper-level media service in the upper-level video monitoring system, realizing packet loss detection and retransmission, reducing the packet loss rate of the upper-level media service, and ensuring the video quality of the upper-level media service.

[0155] Such as Figure 8 As shown, this embodiment proposes a packet loss handling device in video data transmission. The device includes:

[0156] A monitoring unit 801, configured to monitor that the video receiving system receives any first video data packet in a video session, and at least determine a target sequence number in the first video data packet; wherein, the video session is established between the video receiving system and the video sending system;

[0157] The first determination unit 802 is configured to determine that the first video data packet is a newly arrived data packet if the target sequence number is greater than the minimum sequence number among each sequence number stored in the first queue, or if the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue;

[0158] The storage unit 803 is configured to store the target sequence number into the first queue;

[0159] The monitoring unit 804 is configured to, when monitoring that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the second smallest sequence number in the first queue from the smallest sequence number to obtain a corresponding sequence number difference;

[0160] The second determination unit 805 is configured to determine that a packet loss has occurred if the sequence number difference is greater than 1;

[0161] The third determination unit 806 is configured to determine each integer that is greater than the smallest sequence number and less than the second smallest sequence number as a sequence number to be retransmitted;

[0162] The deletion unit 807 is configured to delete the smallest sequence number in the first queue;

[0163] The instruction unit 808 is configured to instruct the video sending system to retransmit the video data packet corresponding to each sequence number to be retransmitted to the video receiving system through the video session.

[0164] It should be noted that the processing procedures and the beneficial effects brought by the monitoring unit 801, the first determination unit 802, the storage unit 803, the monitoring unit 804, the second determination unit 805, the third determination unit 806, the deletion unit 807, and the instruction unit 808 can respectively refer to Figure 1 Steps S101 to S108 therein, which will not be elaborated herein.

[0165] Optionally, the monitoring unit 801 is further configured to:

[0166] When monitoring that the video receiving system and the video sending system have established a video session, obtain the session information of the video session; wherein, the session information of the video session includes the address information and port information of the video receiving system, as well as the address information and port information of the video sending system;

[0167] Monitor the data received by the video receiving system in the video session according to the address information and port information of the video receiving system in the session information, as well as the address information and port information of the video sending system;

[0168] When monitoring that the video receiving system receives any first video data packet in the video session, at least determine the target sequence number in the first video data packet.

[0169] Optionally, the above device further includes:

[0170] A queue creation unit, configured to create an empty queue before monitoring that the video receiving system receives any first video data packet in a video session, and use the empty queue as the first queue;

[0171] The storage unit 803 is further configured to: after at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, determine that the first video data packet is a newly issued data packet, and store the target sequence number in the first video data packet into the first queue.

[0172] Optionally, the monitoring unit 801 is further configured to: when monitoring that the video receiving system receives any first video data packet in a video session, determine the target sequence number in the first video data packet and the receiving time of the video receiving system for the first video data packet;

[0173] The above device further includes:

[0174] A saving unit, configured to, after determining that the first video data packet is a newly issued data packet, correspondingly save the target sequence number and the receiving time in the first video data packet;

[0175] The monitoring unit 804 is further configured to: when monitoring that the total number of sequence numbers in the first queue reaches a preset threshold, or the receiving duration corresponding to the minimum sequence number exceeds a preset duration threshold, subtract the second minimum sequence number in the first queue from the minimum sequence number to obtain a sequence number difference;

[0176] Wherein, the receiving duration is obtained by subtracting the receiving time corresponding to the minimum sequence number from the current time.

[0177] Optionally, the video sending system includes a media service and a monitoring service; the media service is configured to send a plurality of second video data packets to the video receiving system through a video session; the monitoring service is configured to, when monitoring that the media service sends any second video data packet in the video session, store the monitored second video data packet into the created second queue;

[0178] The instruction unit 808 is further configured to:

[0179] Generate a data packet retransmission instruction according to each sequence number to be retransmitted, where the data packet retransmission instruction includes each sequence number to be retransmitted;

[0180] Send the data packet retransmission instruction to the monitoring service, so that the monitoring service: respectively find the corresponding second video data packet in the second queue according to each sequence number to be retransmitted in the data packet retransmission instruction, and send each found second video data packet to the video receiving system through the video session.

[0181] Optionally, the above device further includes:

[0182] A stop unit, configured to continuously detect the session state of the video session after monitoring the data received by the video receiving system in the video session; when it is detected that the session state of the video session is session termination, stop monitoring the data received by the video receiving system in the video session.

[0183] Optionally, the above device further includes:

[0184] A fourth determination unit, configured to, after at least determining the target sequence number in the first video data packet, if the target sequence number is not greater than the minimum sequence number, and the sum value of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, determine that the first video data packet is a retransmitted packet, and prohibit storing the target sequence number into the first queue.

[0185] The packet loss handling device in video data transmission provided in this embodiment can, in the video session established between the video receiving system and the video sending system, determine the sequence number corresponding to the lost packet, that is, the sequence number to be retransmitted, through the new packet determination condition and the packet loss determination mechanism, and instruct the video sending system to retransmit the lost video data packet to the video receiving system through the video session, so as to implement the retransmission of the lost packet and ensure the video quality of the video receiving system.

[0186] The packet loss handling device in video data transmission in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0187] The embodiment of the present invention further provides a computer device having the above Figure 8 shown packet loss handling device in video data transmission.

[0188] Please refer to Figure 9, A schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. The computer device includes: one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 9 Taking one processor 10 as an example in Figure 9 .

[0189] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field-programmable gate array, a generic array logic, or any combination thereof.

[0190] Among them, the memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiments.

[0191] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function. The data storage area can store data created according to the use of the computer device. In addition, the memory 20 can include a high-speed random access memory and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The memory 20 can include a volatile memory, such as a random access memory. The memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive. The memory 20 can also include a combination of the above types of memories.

[0193] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0194] Embodiments of the present invention also provide a computer-readable storage medium. The method according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing packet loss in video data transmission, characterized in that: include: When monitoring that the video receiving system receives any first video data packet in a video session, at least determining a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; If the target sequence number is greater than the minimum sequence number of each sequence number stored in the first queue, or the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly sent data packet, and the target sequence number is stored in the first queue; When it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number is subtracted from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; If the sequence number difference is greater than 1, it is determined that packet loss has occurred, and each integer greater than the minimum sequence number and less than the second minimum sequence number is determined as a sequence number to be retransmitted; The minimum sequence number is deleted from the first queue, and the video sending system is instructed to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

2. The method according to claim 1, characterized in that The monitoring that the video receiving system receives any first video data packet in the video session, then at least determining the target sequence number in the first video data packet, comprises: When monitoring that the video receiving system and the video sending system have completed the establishment of the video session, acquiring session information of the video session; wherein the session information of the video session includes address information and port information of the video receiving system, and address information and port information of the video sending system; Monitoring data received by the video receiving system in the video session according to the address information and port information of the video receiving system in the session information, and the address information and port information of the video sending system; If it is monitored that the video receiving system receives any of the first video data packets in the video session, at least a target sequence number in the first video data packet is determined.

3. The method according to claim 2, characterized in that Before monitoring that the video receiving system receives any of the first video data packets in the video session, the method further includes: Create an empty queue, and use the empty queue as the first queue; After at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, then the first video data packet is determined to be a newly sent data packet, and the target sequence number in the first video data packet is stored in the first queue.

4. The method according to claim 1, characterized in that The at least determining the target sequence number in the first video data packet comprises: Determining a target sequence number in the first video data packet and a time at which the video receiving system receives the first video data packet; After determining that the first video data packet is a newly sent data packet, the method further includes: Correspondingly saving the target sequence number and the receiving time in the first video data packet; The monitoring that the total number of sequence numbers in the first queue reaches a preset threshold, subtracting the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference, includes: If it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, or the receiving duration corresponding to the minimum sequence number exceeds the preset duration threshold, the minimum sequence number is subtracted from the second minimum sequence number to obtain the sequence number difference; The receiving duration is obtained by subtracting the receiving time corresponding to the minimum sequence number from the current time.

5. The method according to claim 1, characterized in that The video sending system includes a media service and a monitoring service; the media service is used to send a plurality of second video data packets to the video receiving system through the video session; the monitoring service is used to monitor that the media service sends any second video data packet in the video session, and then store the monitored second video data packet in the created second queue; The instructing the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session includes: Generate a data packet retransmission instruction according to each of the sequence numbers to be retransmitted, wherein the data packet retransmission instruction includes each of the sequence numbers to be retransmitted; The data packet retransmission instruction is sent to the monitoring service so that the monitoring service: searches for the corresponding second video data packet in the second queue according to each of the to-be-retransmitted sequence numbers in the data packet retransmission instruction, and sends each found second video data packet to the video receiving system through the video session.

6. The method according to claim 2, characterized in that After monitoring the data received by the video receiving system in the video session, the method further includes: Continuously detecting the session state of the video session; When it is detected that the session state of the video session is session termination, the monitoring of the data received by the video receiving system in the video session is stopped.

7. The method according to any one of claims 1 to 6, characterized in that After at least determining the target sequence number in the first video data packet, the method further includes: If the target sequence number is not greater than the minimum sequence number, and the sum of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, the first video data packet is determined to be a retransmitted data packet, and the target sequence number is prohibited from being stored in the first queue.

8. A packet loss processing device in video data transmission, characterized in that: include A monitoring unit, configured to monitor that the video receiving system receives any first video data packet in a video session, and then at least determine a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; A first determining unit, configured to determine that the first video data packet is a newly transmitted data packet if the target sequence number is greater than a minimum sequence number among each sequence number stored in the first queue, or if a sum of the target sequence number and a set maximum out-of-order positive offset is greater than a maximum sequence number in the first queue; A storage unit, configured to store the target sequence number in the first queue; a monitoring unit, configured to, when it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; A second determining unit, configured to determine that packet loss occurs if the sequence number difference is greater than 1; A third determining unit, configured to determine each integer greater than the minimum sequence number and less than the second minimum sequence number as a sequence number to be retransmitted; a deleting unit, configured to delete the minimum sequence number in the first queue; The instruction unit is used to instruct the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the packet loss processing method in video data transmission according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the packet loss processing method in video data transmission according to any one of claims 1 to 7.

Images