Packet loss processing method and device in video data transmission, equipment and medium

Abstract

The invention relates to the field of video data transmission, and discloses a packet loss processing method and device in video data transmission, equipment and a medium. According to the invention, in the video session established between the video receiving system and the video sending system, the serial number corresponding to the lost data packet, namely the serial number to be retransmitted, is determined through the newly transmitted data packet judgment condition and the packet loss judgment mechanism, and the video sending system is instructed to retransmit the lost video data packet to the video receiving system through the video session. Lost data packets are retransmitted, and the video quality of a video receiving system is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of video data transmission, and in particular to a packet loss processing method, device, equipment and medium in video data transmission. Background Art

[0002] With the development of science and technology, video data transmission technology is constantly improving.

[0003] When sending a video, the video sender will decompose the video content into multiple video frames, and each video frame will be divided into multiple data packets with consecutive serial numbers for transmission. For example, the first video frame is divided into 50 data packets with serial numbers from 1 to 50 for transmission, and the second video frame is divided into 10 data packets with serial numbers from 51 to 60 for transmission. The video sender will send data packets in order from small to large serial numbers, such as sending data packets with serial numbers from 1 to 50 in sequence. When the video receiver receives the data packets, the serial numbers of the data packets may be out of order, for example, the data packet with serial number 10 is received first, and then the data packet with serial number 9 is received.

[0004] Due to network congestion, hardware failure or configuration errors, the video receiver may not receive some data packets, which is called packet loss. When the packet loss rate is high, the video quality will be degraded, such as mosaic and flickering.

[0005] However, the related technology cannot effectively solve the packet loss problem during video data transmission. Summary of the invention

[0006] The present invention provides a packet loss processing method, device, equipment and medium in video data transmission, which are used to solve the defect that the related technology cannot effectively solve the packet loss problem in the process of video data transmission, realize packet loss determination and packet loss retransmission, and improve video quality.

[0007] In a first aspect, the present invention provides a method for processing packet loss in video data transmission, comprising: When monitoring that the video receiving system receives any first video data packet in a video session, at least determining a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; If the target sequence number is greater than the minimum sequence number of each sequence number stored in the first queue, or the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly sent data packet, and the target sequence number is stored in the first queue; When it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number is subtracted from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; If the sequence number difference is greater than 1, it is determined that packet loss has occurred, and each integer greater than the minimum sequence number and less than the second minimum sequence number is determined as a sequence number to be retransmitted; The minimum sequence number is deleted from the first queue, and the video sending system is instructed to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

[0008] Optionally, the monitoring that the video receiving system receives any first video data packet in the video session, then at least determining the target sequence number in the first video data packet includes: When monitoring that the video receiving system and the video sending system have completed the establishment of the video session, acquiring session information of the video session; wherein the session information of the video session includes address information and port information of the video receiving system, and address information and port information of the video sending system; Monitoring data received by the video receiving system in the video session according to the address information and port information of the video receiving system in the session information, and the address information and port information of the video sending system; If it is monitored that the video receiving system receives any of the first video data packets in the video session, at least a target sequence number in the first video data packet is determined.

[0009] Optionally, before monitoring that the video receiving system receives any first video data packet in the video session, the method further includes: Create an empty queue, and use the empty queue as the first queue; After at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, then the first video data packet is determined to be a newly sent data packet, and the target sequence number in the first video data packet is stored in the first queue.

[0010] Optionally, the at least determining a target sequence number in the first video data packet includes: Determining a target sequence number in the first video data packet and a time at which the video receiving system receives the first video data packet; After determining that the first video data packet is a newly sent data packet, the method further includes: Correspondingly saving the target sequence number and the receiving time in the first video data packet; The monitoring that the total number of sequence numbers in the first queue reaches a preset threshold, subtracting the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference, includes: If it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, or the receiving duration corresponding to the minimum sequence number exceeds the preset duration threshold, the minimum sequence number is subtracted from the second minimum sequence number to obtain the sequence number difference; The receiving duration is obtained by subtracting the receiving time corresponding to the minimum sequence number from the current time.

[0011] Optionally, the video sending system includes a media service and a monitoring service; the media service is used to send multiple second video data packets to the video receiving system through the video session; the monitoring service is used to monitor that the media service sends any second video data packet in the video session, and then store the monitored second video data packet in the created second queue; The instructing the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session includes: Generate a data packet retransmission instruction according to each of the sequence numbers to be retransmitted, wherein the data packet retransmission instruction includes each of the sequence numbers to be retransmitted; The data packet retransmission instruction is sent to the monitoring service so that the monitoring service: searches for the corresponding second video data packet in the second queue according to each of the to-be-retransmitted sequence numbers in the data packet retransmission instruction, and sends each found second video data packet to the video receiving system through the video session.

[0012] Optionally, after monitoring the data received by the video receiving system in the video session, the method further includes: Continuously detecting the session state of the video session; When it is detected that the session state of the video session is session termination, the monitoring of the data received by the video receiving system in the video session is stopped.

[0013] Optionally, after at least determining the target sequence number in the first video data packet, the method further includes: If the target sequence number is not greater than the minimum sequence number, and the sum of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, the first video data packet is determined to be a retransmitted data packet, and the target sequence number is prohibited from being stored in the first queue.

[0014] In a second aspect, the present invention provides a packet loss processing device in video data transmission, comprising: A monitoring unit, configured to monitor that the video receiving system receives any first video data packet in a video session, and then at least determine a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; A first determining unit, configured to determine that the first video data packet is a newly transmitted data packet if the target sequence number is greater than a minimum sequence number among each sequence number stored in the first queue, or if a sum of the target sequence number and a set maximum out-of-order positive offset is greater than a maximum sequence number in the first queue; A storage unit, configured to store the target sequence number in the first queue; a monitoring unit, configured to, when it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; A second determining unit, configured to determine that packet loss occurs if the sequence number difference is greater than 1; A third determining unit, configured to determine each integer greater than the minimum sequence number and less than the second minimum sequence number as a sequence number to be retransmitted; a deleting unit, configured to delete the minimum sequence number in the first queue; The instruction unit is used to instruct the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

[0015] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for handling packet loss in video data transmission of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the packet loss processing method in video data transmission according to the first aspect or any corresponding embodiment thereof.

[0017] The packet loss processing method, device, equipment and medium in video data transmission provided by the present invention monitors that a video receiving system receives any first video data packet in a video session, then at least determines the target sequence number in the first video data packet, if the target sequence number is greater than the minimum sequence number in the first queue, or the sum of the target sequence number and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, then determines that the first video data packet is a newly sent data packet, and stores the target sequence number in the first queue. When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number in the first queue is subtracted from the second minimum sequence number in the first queue to obtain the corresponding sequence number difference. If the sequence number difference is greater than 1, it is determined that a packet loss occurs, and each integer greater than the minimum sequence number in the first queue and less than the second minimum sequence number in the first queue is determined as a sequence number to be retransmitted. Delete the minimum sequence number in the first queue, and instruct the video sending system to retransmit each video data packet corresponding to the sequence number to be retransmitted to the video receiving system through the video session. The present invention can determine the sequence number corresponding to the lost data packet, i.e., the sequence number to be retransmitted, through the newly transmitted data packet determination condition and the packet loss determination mechanism in the video session established between the video receiving system and the video sending system, and instruct the video sending system to retransmit the lost video data packet to the video receiving system through the video session, thereby realizing the retransmission of the lost data packet and ensuring the video quality of the video receiving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a method for processing packet loss in video data transmission provided by an embodiment of the present invention; Figure 2 A signaling interaction diagram of an upper and lower level video surveillance system provided by an embodiment of the present invention; Figure 3 Another signaling interaction diagram of an upper and lower level video surveillance system provided by an embodiment of the present invention; Figure 4 A workflow diagram of an upper-level SIP signaling monitoring service provided by an embodiment of the present invention; Figure 5 A schematic diagram of session information content of a video session provided by an embodiment of the present invention; Figure 6 A workflow diagram of a superior video reception monitoring service provided by an embodiment of the present invention; Figure 7 A workflow diagram of a subordinate video sending monitoring service provided by an embodiment of the present invention; Figure 8 A schematic diagram of the structure of a packet loss processing device in video data transmission provided by an embodiment of the present invention; Fig. 9 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figure 1-Figure 7 The present invention describes a method for processing packet loss in video data transmission.

[0022] like Figure 1 As shown, this embodiment proposes a first method for processing packet loss in video data transmission, which may include the following steps: S101. When monitoring that a video receiving system receives any first video data packet in a video session, at least a target sequence number in the first video data packet is determined; wherein the video session is established between the video receiving system and the video sending system.

[0023] The first video data packet is a video data packet received by the video receiving system in a video session and sent by the video sending system.

[0024] Specifically, the target sequence number refers to the sequence number carried in the first video data packet.

[0025] It should be noted that the video receiving system and the video sending system can establish a video session and transmit video streams through the video session. Specifically, the video receiving system and the video sending system can establish a video session by sending a Session Initialization Protocol (SIP) and transmit video data in the established video session. The video sending system can use the Real-Time Transport Protocol (RTP) in the video session to send multiple video data packets to the video receiving system in order of sequence numbers from small to large. For example, the video data packets with sequence numbers from 1 to 50 are sent to the video receiving system in order of sequence numbers from small to large. The video receiving system can receive the video data packets sent by the video sending system in the video session. Each video data packet carries a sequence number. The sequence numbers of the video data packets received by the video receiving system may be out of order. For example, the video receiving system first receives the video data packet with sequence number 8, then receives the video data packet with sequence number 6, and then receives the video data packet with sequence number 10.

[0026] Specifically, this embodiment can monitor the first video data packet received by the video receiving system in a video session. Whenever the video receiving system is monitored to receive the first video data packet in the video session, the first video data packet is parsed to obtain the target sequence number carried in the first video data packet.

[0027] S102: If the target sequence number is greater than the minimum sequence number of each sequence number stored in the first queue, or the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, then the first video data packet is determined to be a newly transmitted data packet.

[0028] The first queue may be a queue created in this embodiment, and is used to store the sequence numbers of one or more newly sent data packets in the video session.

[0029] It should be noted that the first video data packet received by the video receiving system in the video session can be divided into two types: a retransmitted data packet and a newly transmitted data packet. A retransmitted data packet is a first video data packet that has been previously transmitted in the video session and is transmitted again. A newly transmitted data packet is a first video data packet that has not been previously transmitted in the video session and is transmitted for the first time.

[0030] In actual applications, the sequence numbers of the video data packets received by the video receiving system in the video session may not be strictly ordered from small to large, but may be in random order. The degree of random deviation of the sequence numbers generally does not exceed a specific range.

[0031] Specifically, the maximum out-of-order positive offset is the maximum offset of the sequence number of the video data packet when out-of-order occurs. In this embodiment, the technical personnel can set it according to the actual situation, for example, it can be set to 6, or for another example, it can be set to 5.

[0032] In practical applications, when the total number of sequence numbers stored in the first queue is relatively small, the maximum sequence number and the minimum sequence number in the first queue are relatively close. At this time, if the sum of the target sequence number in the first video data packet and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it can be determined that the first video data packet is a newly sent data packet. When the total number of sequence numbers stored in the first queue is relatively large, the difference between the maximum sequence number and the minimum sequence number in the first queue is relatively large. At this time, if the target sequence number in the first video data packet is greater than the minimum sequence number in the first queue, it can be determined that the first video data packet is a newly sent data packet.

[0033] Specifically, this embodiment can set a judgment condition for determining that the first video data packet is a newly sent data packet. The judgment condition includes a first condition and a second condition. Among them, the first condition is that the target sequence number in the first video data packet is greater than the minimum sequence number in the first queue. The second condition is that the sum of the target sequence number and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue.

[0034] It can be understood that the first condition is applicable to the judgment of a newly sent data packet when the total number of sequence numbers in the first queue is relatively large, and the second condition is applicable to the judgment of a newly sent data packet when the total number of sequence numbers in the first queue is relatively small. As long as the target sequence number in the first video data packet meets any one of the first condition and the second condition, this embodiment can determine that the first video data packet is a newly sent data packet.

[0035] Optionally, in other packet loss processing methods in video data transmission proposed in this embodiment, after the above step S101, the following steps may be further included: If the target sequence number in the first video data packet is not greater than the minimum sequence number in the first queue, and the sum of the target sequence number in the first video data packet and the maximum out-of-order positive offset is not greater than the maximum sequence number in the first queue, then the first video data packet is determined to be a retransmitted data packet, and it is prohibited to store the target sequence number in the first queue.

[0036] It can be understood that if the target sequence number in the first video data packet does not meet both the first condition and the second condition, that is, the target sequence number is not greater than the minimum sequence number in the first queue, and the sum of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number in the first queue, then it can be determined that the first video data packet is a retransmitted data packet, and it is prohibited to store the target sequence number in the first video data packet in the first queue.

[0037] S103: Store the target sequence number in the first queue.

[0038] Specifically, in this embodiment, after determining that the first video data packet is a newly sent data packet, the target sequence number in the first video data packet may be stored in the first queue.

[0039] S104: When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number is subtracted from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference.

[0040] It can be understood that, in this embodiment, whenever it is determined that the first video data packet received by the video receiving system is a newly sent data packet, the target sequence number in the first video data packet is stored in the first queue. Therefore, the sequence numbers stored in the first queue are in dynamic change, and the size sorting of the sequence numbers in the first queue is also in dynamic change. The minimum sequence number in the first queue is the smallest sequence number among the sequence numbers currently stored in the first queue. The second smallest sequence number in the first queue is the second smallest sequence number among the sequence numbers currently stored in the first queue.

[0041] It should be noted that monitoring whether the total number of sequence numbers in the first queue reaches a preset threshold is a starting condition for packet loss determination. Specifically, the present embodiment can continuously monitor whether the total number of sequence numbers in the first queue reaches a preset threshold. If the total number of sequence numbers in the first queue does not reach the preset threshold, monitoring can continue. If it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, the minimum sequence number and the second minimum sequence number in the first queue can be determined, and the minimum sequence number is subtracted from the second minimum sequence number to obtain the corresponding sequence number difference.

[0042] The preset threshold value can be set by a technician according to actual conditions, and is not limited in this embodiment. Optionally, the preset threshold value can be equal to the sequence number capacity in the first queue. In this case, when the first queue is detected to be full of sequence numbers, the embodiment can subtract the minimum sequence number from the second minimum sequence number in the first queue.

[0043] In order to better introduce step S104, this embodiment proposes Example 1 for illustration.

[0044] Example 1: The above preset threshold is 11, and the first queue has stored 10 serial numbers, namely 20, 23, 24, 25, 26, 27, 28, 29, 30 and 31. When the monitoring video receives the first video data packet with serial number 32, the serial number 32 is stored in the first queue. At this time, the minimum serial number in the first queue is 20, and the second minimum serial number is 23. In this embodiment, the serial number difference can be obtained by subtracting 23 from 20 to obtain 3.

[0045] S105: If the sequence number difference is greater than 1, it is determined that packet loss has occurred.

[0046] Specifically, in this embodiment, the obtained sequence number difference can be compared with 1, and whether packet loss occurs can be determined based on the comparison result. If the sequence number difference is greater than 1, it can be determined that packet loss occurs. If the sequence number difference is not greater than 1, it can be determined that packet loss does not occur. Continuing with the above example 1, if the sequence number difference is greater than 3, it can be determined that packet loss occurs.

[0047] S106. Determine each integer that is greater than the smallest sequence number in the first queue and less than the second smallest sequence number in the first queue as a sequence number to be retransmitted.

[0048] Specifically, when it is determined that packet loss has occurred, this embodiment can determine each integer greater than the minimum sequence number in the first queue and less than the second minimum sequence number in the first queue as the sequence number to be retransmitted. Continuing with the above example 1, this embodiment can determine each integer greater than 20 and less than 23, that is, 21 and 22, as the sequence number to be retransmitted.

[0049] It can be understood that each video data packet corresponding to the sequence number to be retransmitted is a video data packet lost in the video session and not received by the video receiving system.

[0050] S107. Delete the smallest sequence number in the first queue.

[0051] Specifically, in this embodiment, after determining the sequence number to be retransmitted, the smallest sequence number in the first queue can be deleted from the first queue. Continuing with the above example 1, in this embodiment, the sequence number 20 can be deleted from the first queue.

[0052] S108 , instructing the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

[0053] Specifically, this embodiment may send a retransmission instruction to the video sending system, so that the video sending system retransmits each video data packet corresponding to the sequence number to be retransmitted to the video receiving system through a video session.

[0054] The packet loss processing method in video data transmission proposed in this embodiment monitors that the video receiving system receives any first video data packet in the video session, then at least determines the target sequence number in the first video data packet, if the target sequence number is greater than the minimum sequence number in the first queue, or the sum of the target sequence number and the maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, then determines that the first video data packet is a newly sent data packet, and stores the target sequence number in the first queue. When it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number in the first queue is subtracted from the second minimum sequence number in the first queue to obtain the corresponding sequence number difference. If the sequence number difference is greater than 1, it is determined that a packet loss has occurred, and each integer greater than the minimum sequence number in the first queue and less than the second minimum sequence number in the first queue is determined as a sequence number to be retransmitted. Delete the minimum sequence number in the first queue, and instruct the video sending system to retransmit each video data packet corresponding to the sequence number to be retransmitted to the video receiving system through the video session. This embodiment can determine the sequence number corresponding to the lost data packet, i.e., the sequence number to be retransmitted, by setting the newly transmitted data packet determination condition and the packet loss determination mechanism in the video session established between the video receiving system and the video sending system, and instruct the video sending system to retransmit the lost video data packet to the video receiving system through the video session, thereby realizing the retransmission of the lost data packet, thereby effectively reducing the packet loss rate and ensuring the video quality of the video receiving system.

[0055] based on Figure 1 This embodiment proposes a second method for processing packet loss in video data transmission. In this method, step S101 may include: When monitoring that the video receiving system and the video sending system have established a video session, the session information of the video session is obtained; wherein the session information of the video session includes the address information and port information of the video receiving system, and the address information and port information of the video sending system; Monitoring data received by the video receiving system in the video session according to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information; When it is monitored that the video receiving system receives any first video data packet in the video session, at least a target sequence number in the first video data packet is determined.

[0056] Specifically, this embodiment can monitor the conversation behavior between the video receiving system and the video sending system. When it is determined that the two have established a complete video conversation, the session information of the video conversation is obtained. Based on the address information and port information of the video receiving system in the session information, as well as the address information and port information of the video sending system, the data receiving behavior of the video receiving system in the video conversation and the data it receives are monitored.

[0057] Among them, after acquiring the session information of the video session, this embodiment can input the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information into the created monitoring task, so as to start the monitoring task and enable the monitoring task to monitor the data received by the video receiving system in the video session according to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information. When the monitoring task monitors that the video receiving system receives any first video data packet in the video session, this embodiment determines the target sequence number in the first video data packet.

[0058] Optionally, in other packet loss processing methods in video data transmission proposed in this embodiment, the method further includes, before the video receiving system is monitored to receive any first video data packet in the video session: Create an empty queue and use it as the first queue.

[0059] After at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, the first video data packet is determined to be a newly sent data packet, and the target sequence number in the first video data packet is stored in the first queue.

[0060] Specifically, in this embodiment, an empty queue may be created as the first queue, and whenever a first video data packet is determined to be a newly sent data packet, the target sequence number in the first video data packet is stored in the first queue.

[0061] It is understandable that the first first video data packet received by the video receiving system in the video session is a newly sent data packet. Before the video receiving system receives the first first video data packet, no sequence number is stored in the first queue. When monitoring that the video receiving system receives the first first video data packet in the video session, this embodiment can directly determine that the first video data packet is a newly sent data packet, and store the target sequence number in the first video data packet in the first queue. When monitoring that the video receiving system receives the video data packet after the first video data packet in the video session, this embodiment can determine whether the video data packet is a newly sent data packet or a retransmitted data packet based on the above-mentioned first condition and second condition.

[0062] Optionally, after monitoring the data received by the video receiving system in the video session, the method may further include: Continuously monitor the session status of the video session; When it is detected that the session state of the video session is session termination, the monitoring of the data received by the video receiving system in the video session is stopped.

[0063] Specifically, when the termination of the video session is detected, the embodiment can stop monitoring the data receiving behavior of the video receiving system through the video session, thereby reducing unnecessary consumption of monitoring resources.

[0064] The packet loss processing method in video data transmission proposed in this embodiment can monitor the session behavior between the video receiving system and the video sending system. When it is monitored that the two have established a complete video session, the session information of the video session is obtained, and the first video data packet received by the video receiving system in the video session is monitored according to the session information, thereby effectively realizing the monitoring of the first video data packet, and each newly sent data packet received by the video receiving system can be identified, and the serial number in each newly sent data packet is stored by using the created first queue, so as to effectively ensure the implementation of the packet loss determination mechanism.

[0065] based on Figure 1 This embodiment proposes a third method for processing packet loss in video data transmission. In this method, step S101 may include: When it is monitored that the video receiving system receives any first video data packet in the video session, a target sequence number in the first video data packet and a time when the video receiving system receives the first video data packet are determined.

[0066] At this time, after step S102, the method may further include: The target sequence number and the receiving time in the first video data packet are correspondingly saved.

[0067] At this time, the above step S104 may include: If it is monitored that the total number of sequence numbers in the first queue reaches a preset threshold, or the receiving duration corresponding to the minimum sequence number in the first queue exceeds the preset duration threshold, the minimum sequence number in the first queue is subtracted from the second minimum sequence number in the first queue to obtain a sequence number difference; The receiving duration is obtained by subtracting the receiving time corresponding to the smallest sequence number in the first queue from the current time.

[0068] Specifically, this embodiment can determine the target sequence number in each first video data packet and the receiving time of the first video data packet by the video receiving system, and store the target sequence number and the receiving time in the first video data packet in correspondence.

[0069] The receiving time corresponding to the minimum sequence number in the first queue is the receiving time of the first video data packet corresponding to the minimum sequence number by the video receiving system. In this embodiment, the receiving time can be subtracted from the current time to obtain the receiving duration corresponding to the minimum sequence number.

[0070] At this time, the present embodiment can optimize the packet loss determination start condition, monitor whether the total number of sequence numbers in the first queue reaches a preset threshold, and monitor whether the reception duration corresponding to the minimum sequence number in the first queue exceeds a preset duration threshold. When it is determined that the total number of sequence numbers in the first queue reaches a preset threshold, or the reception duration corresponding to the minimum sequence number in the first queue exceeds the preset duration threshold, the present embodiment can start the packet loss determination process, subtract the minimum sequence number from the second minimum sequence number in the first queue, and obtain the corresponding sequence number difference.

[0071] It should be noted that the preset time threshold can be set by technical personnel according to actual conditions and is not limited in this embodiment.

[0072] Among them, in this embodiment, when it is determined that the total number of sequence numbers in the first queue has not reached the preset threshold, and the receiving duration corresponding to the minimum sequence number in the first queue has not exceeded the preset duration threshold, there is no need to start the packet loss determination process, and it is prohibited to subtract the second minimum sequence number in the first queue from the minimum sequence number.

[0073] The packet loss processing method in video data transmission proposed in this embodiment can optimize the start-up conditions of the packet loss determination mechanism, enrich the packet loss determination scenarios, and improve the accuracy of packet loss identification.

[0074] It should be noted that the video receiving system and the video sending system in this embodiment can be the upper-level video surveillance system and the lower-level video surveillance system, respectively. The media service in the lower-level video surveillance system can directly send a video data packet to the upper-level video surveillance system. In some scenarios, this embodiment cannot directly send instructions or requests to the media service in the lower-level video surveillance system. At this time, when it is determined that packet loss has occurred, this embodiment cannot directly request or instruct the media service in the lower-level video surveillance system to resend the lost video data packet to the video receiving system. For example, GB / T28181 stipulates that the RTP / Real-time Transport Control Protocol (RTCP) specified in the Request for Comments (RFC) 3550 of the Internet Engineering Task Force (IETF) is adopted, but RFC3550 does not specify packet loss retransmission, and does not force the implementation of packet loss retransmission. IETF recommends the negative acknowledgment (NACK) defined in RFC4585 for packet loss retransmission. In actual environments, the upper and lower level video surveillance systems may be developed by different manufacturers. Some manufacturers do not implement the NACK mechanism and cannot achieve packet loss retransmission.

[0075] based on Figure 1This embodiment proposes a fourth method for processing packet loss in video data transmission. In this method, the video sending system includes a media service and a monitoring service; the media service is used to send multiple second video data packets to the video receiving system through a video session; the monitoring service is used to monitor that the media service sends any second video data packet in the video session, and then store the monitored second video data packet in the created second queue.

[0076] At this time, step S108 may include: Generate a data packet retransmission instruction according to each sequence number to be retransmitted, wherein the data packet retransmission instruction includes each sequence number to be retransmitted; The data packet retransmission instruction is sent to the monitoring service, so that the monitoring service: searches for the corresponding second video data packet in the second queue according to each to-be-retransmitted sequence number in the data packet retransmission instruction, and sends each found second video data packet to the video receiving system through the video session.

[0077] The second video data packet is a video data packet sent by the media service to the video receiving system based on the receiving address and port information in the video session.

[0078] Specifically, the monitoring service can monitor the session behavior between the media service and the video receiving system, and when monitoring that the video session is established between the two, it starts to monitor the data sending behavior of the media service in the video session and the data sent. Whenever the monitoring service monitors that the media service sends a second video data packet in the video session, the monitoring service can store the second video data packet in the second queue.

[0079] Specifically, after determining each sequence number to be retransmitted, this embodiment can generate a data packet retransmission instruction carrying all sequence numbers to be retransmitted, and send the data packet retransmission quality to the monitoring service in the video transmission system. The monitoring service can respond to the data packet retransmission instruction, parse the data packet retransmission instruction, obtain all sequence numbers to be retransmitted in the data packet retransmission instruction, and find out the second video data packet corresponding to each sequence number to be retransmitted in the second queue, that is, the data packet to be retransmitted. Afterwards, the monitoring service can send the second video data packet corresponding to each sequence number to be retransmitted found to the video receiving system according to the receiving address and port information in the video session.

[0080] Optionally, the above-mentioned monitoring service can also be used to: determine the serial number in any second video data packet sent by the monitored media service in the video session, and determine the time when the media service sends the second video data packet, and save the serial number and sending time accordingly.

[0081] Specifically, the monitoring service can also monitor the sending duration corresponding to each second video data packet in the second queue. The sending duration corresponding to the second video data packet is obtained by subtracting the sending time corresponding to the second video data packet from the current time. When it is determined that the sending duration corresponding to any second video data packet exceeds the set duration threshold, it is determined that the second video data packet no longer falls within the range of packet loss and retransmission, and the second video data packet is deleted from the second queue, freeing up the storage space of the second queue and reducing unnecessary consumption of storage resources.

[0082] It can be understood that the set duration threshold is not less than the above-mentioned preset duration threshold, ensuring that lost packets of the video receiving system can be found in the second queue, effectively ensuring the smooth search for lost data packets and subsequent retransmission of lost packets.

[0083] The packet loss processing method in video data transmission proposed in this embodiment, by setting up a media service and a monitoring service in a video sending system, creating a second queue and using the second queue to store each second video data packet sent by the media service, can instruct the monitoring service to find the second video data packet corresponding to the sequence number to be retransmitted in the second queue when it is determined that packet loss has occurred in the video receiving system, and instruct the monitoring service to retransmit each found second video data packet to the video receiving system through a video session, thereby effectively retransmitting the lost data packets and ensuring the video quality of the video receiving end.

[0084] In practical applications, the video receiving system and the video sending system in this embodiment may be different video monitoring systems. Specifically, the video receiving system may be a superior video monitoring system, and the video sending system may be a subordinate video monitoring system.

[0085] In the related technologies, with the large-scale application and popularization of video surveillance systems, the demand for cascading and interconnection between video surveillance systems is increasing. The introduction of relevant standards has unified the docking interface of video surveillance systems. During construction, different manufacturers have different technical levels, and some platforms have not fully considered the scale of application. With the expansion of the scale of cameras, the increase of camera bit streams, and the expansion of the scope of application, the packet loss rate in network transmission increases, affecting the viewing quality. In particular, some video surveillance systems are even difficult to improve by upgrading the software due to some historical reasons. This embodiment can solve this problem without changing the original video surveillance system hardware and software, and achieve retransmission of lost packets to improve the viewing effect.

[0086] like Figure 2As shown, the upper-level video surveillance system in the related technology may include an upper-level SIP service and an upper-level media service, and the lower-level video surveillance system may include a lower-level SIP service and a lower-level media service. When the upper-level and lower-level video surveillance systems are connected, they need to communicate on the network, which may pass through multiple network devices. When the two systems are connected, the SIP protocol is used for signaling interaction, and the RTP protocol is used to transmit media streams. After the lower-level video surveillance system registers with the upper-level video surveillance system, the lower-level video surveillance system reports relevant video resources and directory resources to the upper-level video surveillance system, and the upper-level video surveillance system can call the lower-level video surveillance video for viewing. When the upper-level video surveillance system calls the video, it sends an INVITE transaction request to the lower-level video surveillance system, and the upper and lower levels establish a video session. The RTP protocol information is exchanged through the Session Description Protocol (SDP), including the use of the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP), the sender and the receiving IP and port information, etc. The lower-level media service thus sends the video stream and audio stream to the corresponding upper-level media service. Figure 2 The INVITE request, response OK and confirmation ACK are the interactive signaling between the upper and lower video surveillance systems in the process of establishing a video session, and the BYE request and OK are the interactive signaling between the upper and lower video surveillance systems when closing a video session.

[0087] In the related technology, RTP can use UDP, TCP active, and TCP passive methods to transmit media streams. Although the reliability of video streams can be guaranteed by TCP, it will also lead to large delays and aggravate congestion, which may cause video freezes in the upper-level video surveillance system when the network is not good. When using UDP, many manufacturers have not developed the QOS mechanism of RTP. In this way, there will be some video surveillance system environments. In the early stage after construction, due to the low pressure of the network environment and the low network packet loss rate, both UDP and TCP transmission are normal. However, with the increase of access devices, docking systems, and video users, the network pressure increases, and UDP transmission will appear due to packet loss and screen distortion, while TCP transmission will appear due to too many concurrent numbers. The phenomenon of freezes and jumps. Moreover, since the upper and lower monitoring systems belong to different manufacturers, the upper monitoring system may not be able to directly request the lower media service in the lower monitoring system to resend the lost packets.

[0088] Specifically, this embodiment can reinforce the media stream transmission of the video surveillance system based on the original socket method. Without upgrading the original video surveillance system software, when the video surveillance system transmission is transmitted using UDP, the lost packets after the video surveillance system is connected are retransmitted, thereby reducing screen distortion.

[0089] Among them, this embodiment can use the original socket technology to receive the data packets on the network card, so as to monitor and analyze the signaling data packets and video data packets in the video session between the upper-level video monitoring system and the lower-level video monitoring system. Figure 3 As shown, this embodiment can add 4 new modules: upper-level SIP signaling monitoring service, upper-level video reception monitoring service, lower-level SIP signaling monitoring service and lower-level video sending monitoring service. The red parts are the modules and execution processes added in this embodiment relative to the related technology.

[0090] Specifically, the upper-level SIP signaling monitoring service can monitor the SIP signaling of the upper-level SIP service and control the task start and stop of the upper-level video reception monitoring service. The upper-level video reception monitoring service can monitor the RTP packets received by the upper-level media service, promptly discover that the RTP packets have not been received, and issue a retransmission notification. The lower-level SIP signaling monitoring service can monitor the SIP signaling of the lower-level media service and control the task start and stop of the lower-level video sending monitoring service. The lower-level video sending monitoring service can monitor the RTP packets sent by the lower-level media service and save them for a certain period of time. After receiving the retransmission notification, it retransmits the corresponding RTP packets to the upper-level media service.

[0091] This embodiment can work together with the existing video surveillance system in the related art. The existing video surveillance system does not need to upgrade the software. It only needs to deploy the software components of this embodiment, namely the above four modules, on the corresponding servers of the existing video surveillance system.

[0092] Combination Figure 3 As shown, in the scenario where the upper and lower video surveillance systems transmit video data packets through a video session, the video session in this embodiment can be specifically a video session established between the upper media service and the lower media service, the first video data packet is a video data packet sent by the lower media service and received by the upper media service in the video session, and the second video data packet is a video data packet sent by the lower media service to the upper media service in the video session. This embodiment can regard the upper video reception monitoring service and the upper SIP signaling monitoring service as a whole, and this embodiment can be applied to the whole, and the whole executes as follows Figure 1The various processes shown in the figure implement the packet loss determination and packet loss retransmission process. Specifically, this embodiment can determine through the set packet loss mechanism, identify the packet loss and the sequence number to be retransmitted in the process of the upper-level media service receiving the first video data packet through the video session, and send the sequence number to be retransmitted to the lower-level video sending monitoring service, so that the lower-level video sending monitoring service can find the corresponding second video data packet according to the sequence number to be retransmitted, and retransmit it to the upper-level media service, so as to achieve packet loss retransmission, reduce the packet loss rate, improve the video transmission quality based on UDP of the upper and lower video monitoring systems, and thus improve the user experience. Among them, the lower-level video sending monitoring service can be regarded as a monitoring service in the lower-level video monitoring system.

[0093] Specifically, the upper-level SIP signaling monitoring service can be deployed on the server where the upper-level SIP service is located. Through the raw socket method, the SIP data sent by the upper-level SIP service can be read from the network card, and the data sent to the lower-level SIP service and port can be filtered. When the INVITE request and its response are parsed, the key fields in the SDP message carried by it are parsed (for example, the receiving IP is parsed from the c field, the video receiving port is parsed from the m field, and the SSRC is parsed from the y field. The RTP / AVP in the m field indicates the use of UDP), and a session monitoring is prepared to be started. After the ACK of the INVITE request is parsed, the relevant information of the video stream to be monitored is sent to the upper-level video receiving monitoring service, so that the upper-level video receiving monitoring service can perform session monitoring based on this information.

[0094] like Figure 4 As shown, this embodiment can monitor the SIP signaling of the upper-level SIP service through the upper-level SIP signaling monitoring service, receive data packets, and filter data packets according to the configuration information. When it is detected that the upper-level SIP service receives an INVITE request, parse SIP and SDP. When it is detected that the upper-level SIP service receives an INVITE response, parse SIP and SDP. When it is detected that the upper-level SIP service receives an ACK, parse SIP. Send a start video monitoring command to the upper-level video reception monitoring service. When it is detected that the upper-level SIP service receives a BYE request, parse SIP and send a stop video monitoring command to the upper-level video reception monitoring service. Prepare for the next round of monitoring tasks.

[0095] The content in the video stream related information is the session information of the video session. Figure 5As shown in the figure, Call-ID is used to uniquely identify a video session, VideoReceiverIP is the device IP address of the upper-level media service, VideoReceiverPort is the port of the upper-level media service, VideoSenderrIP is the device IP address of the lower-level media service, and VideoSenderPort is the port of the lower-level media service. VideoSSRC is a synchronization source identifier, which is used to uniquely identify a media stream in the RTP protocol. When the upper-level SIP signaling monitoring service parses the BYE request, it notifies the upper-level video receiving monitoring service to stop monitoring the session.

[0096] Specifically, the upper-level video receiving monitoring service can be deployed on the server where the upper-level media service is located, and the RTP data packets can be parsed and filtered according to the UDP port and SSRC provided by the upper-level SIP signaling monitoring service. Based on the sequence number and packet loss judgment mechanism of the RTP packet, the lower-level video sending monitoring service will be notified if packet loss is found.

[0097] like Figure 6 As shown, in this embodiment, the upper-level video reception monitoring service can be started, and a notification sent by the upper-level SIP signaling monitoring service is received to start a video data monitoring session. UDP packets are received, filtered by receiving IP and port, and RTP is parsed. After that, packet loss judgment is performed, and RTP retransmission notification is sent if packet loss is determined. Until the stop notification is received, the current round of monitoring stops.

[0098] Specifically, when the upper-level video reception monitoring service is judging the packet loss mechanism, each RTP packet carries a sequence number, which can be a cyclic sequence from 0 to 65535. In the actual process, since the received RTP may be out of order, it cannot be simply judged based on the order. In this embodiment, the time and queue length can be used to judge whether the packet is lost. A first queue is created, and each time a video data packet is received by the upper-level media service in the video session, the RTP sequence number is taken out and the receiving time is recorded. If the sequence number is larger than the minimum sequence number in the first queue, or the sum of the sequence number and the maximum out-of-order positive offset is larger than the maximum sequence number in the first queue, the video data packet is determined to be a newly sent data packet, and the sequence number in the video data packet is stored in the first queue, and the sequence numbers in the first queue can be sorted from small to large. If the total number of sequence numbers in the first queue reaches a preset threshold value N, or the duration of the receiving time corresponding to the minimum sequence number in the first queue from the current time exceeds the preset duration threshold, the next minimum sequence number in the first queue is subtracted from the minimum sequence number to obtain the corresponding difference. If the difference is greater than 1, indicating that there is packet loss, it is considered that there is packet loss, and each integer greater than the minimum sequence number in the first queue and less than the next minimum sequence number is determined as the sequence number corresponding to the lost data packet, that is, the sequence number to be retransmitted.

[0099] Specifically, the subordinate SIP signaling monitoring service can be deployed on the server where the subordinate SIP service is located, and its execution process can refer to the execution process of the superior SIP signaling monitoring service. The subordinate SIP signaling monitoring service can send the relevant information of the video stream that needs to be monitored, that is, the session information of the video session to the subordinate video sending monitoring service, so that the subordinate video sending monitoring service can perform session monitoring based on this information.

[0100] Specifically, the lower-level video transmission monitoring service can be deployed on the server where the lower-level media service is located, and parse and filter the RTP data packet according to the UDP port and synchronization source identifier given by the lower-level SIP signaling monitoring service, record the time of the RTP packet data, save it in the queue, and set a timeout value. If the timeout exceeds, it will be discarded. The timeout time must be no less than the packet loss detection timeout time of the upper-level video reception monitoring service. When an RTP packet retransmission request is received, the corresponding RTP packet is found from the queue according to the packet loss sequence number and sent to the upper-level media service.

[0101] like Figure 7 As shown, this embodiment can start the lower-level video transmission monitoring service, receive the notification sent by the lower-level SIP signaling monitoring service, start a video data monitoring session, receive UDP packets, filter and parse RTP, and save them in the queue. Determine whether a packet loss notification is received. If so, take out and send RTP according to the sequence number, detect data timeout, and discard the timeout data in the queue. Until receiving a stop monitoring notification, this round of monitoring stops.

[0102] The packet loss processing method in video data transmission proposed in this embodiment can be applied to detecting video data packets lost in video data transmission between an upper-level video surveillance system and a lower-level video surveillance system, determining the sequence number to be retransmitted, and instructing the lower-level video sending monitoring service in the lower-level video surveillance system to retransmit packet loss to the upper-level media service in the upper-level video surveillance system, thereby realizing packet loss detection and retransmission, reducing the packet loss rate of the upper-level media service, and ensuring the video quality of the upper-level media service.

[0103] like Figure 8 As shown, this embodiment provides a packet loss processing device in video data transmission, the device comprising: The monitoring unit 801 is used to monitor that the video receiving system receives any first video data packet in a video session, and then at least determine the target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; A first determining unit 802 is configured to determine that the first video data packet is a newly transmitted data packet if the target sequence number is greater than the minimum sequence number of each sequence number stored in the first queue, or if the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue; The storage unit 803 is used to store the target sequence number in the first queue; The monitoring unit 804 is configured to, when the total number of sequence numbers in the first queue reaches a preset threshold, subtract the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; A second determining unit 805, configured to determine that packet loss occurs if the sequence number difference is greater than 1; A third determining unit 806, configured to determine each integer greater than the minimum sequence number and less than the second minimum sequence number as a sequence number to be retransmitted; A deleting unit 807, configured to delete the smallest sequence number in the first queue; The instruction unit 808 is used to instruct the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

[0104] It should be noted that the processing of the monitoring unit 801, the first determination unit 802, the storage unit 803, the monitoring unit 804, the second determination unit 805, the third determination unit 806, the deletion unit 807 and the instruction unit 808 and the beneficial effects thereof can be respectively referred to in Figure 1 Steps S101 to S108 in the above are not described in detail.

[0105] Optionally, the monitoring unit 801 is further configured to: When monitoring that the video receiving system and the video sending system have established a video session, the session information of the video session is obtained; wherein the session information of the video session includes the address information and port information of the video receiving system, and the address information and port information of the video sending system; Monitoring data received by the video receiving system in the video session according to the address information and port information of the video receiving system and the address information and port information of the video sending system in the session information; When it is monitored that the video receiving system receives any first video data packet in the video session, at least a target sequence number in the first video data packet is determined.

[0106] Optionally, the above device further comprises: The queue creation unit is used to create an empty queue before monitoring that the video receiving system receives any first video data packet in the video session, and use the empty queue as the first queue; The storage unit 803 is also used to: after at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, determine that the first video data packet is a newly sent data packet, and store the target sequence number in the first video data packet in the first queue.

[0107] Optionally, the monitoring unit 801 is further configured to: upon monitoring that the video receiving system receives any first video data packet in the video session, determine the target sequence number in the first video data packet and the time when the video receiving system receives the first video data packet; The above device also includes: A storage unit, configured to store a target sequence number and a receiving time in the first video data packet accordingly after determining that the first video data packet is a newly transmitted data packet; The monitoring unit 804 is further configured to: when it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, or the reception duration corresponding to the minimum sequence number exceeds a preset duration threshold, subtract the minimum sequence number from the second minimum sequence number in the first queue to obtain a sequence number difference; The receiving duration is obtained by subtracting the receiving time corresponding to the minimum sequence number from the current time.

[0108] Optionally, the video sending system includes a media service and a monitoring service; the media service is used to send multiple second video data packets to the video receiving system through a video session; the monitoring service is used to monitor that the media service sends any second video data packet in the video session, and then store the monitored second video data packet in the created second queue; The instruction unit 808 is further configured to: Generate a data packet retransmission instruction according to each sequence number to be retransmitted, wherein the data packet retransmission instruction includes each sequence number to be retransmitted; The data packet retransmission instruction is sent to the monitoring service, so that the monitoring service: searches for the corresponding second video data packet in the second queue according to each to-be-retransmitted sequence number in the data packet retransmission instruction, and sends each found second video data packet to the video receiving system through the video session.

[0109] Optionally, the above device further comprises: The stopping unit is used to continuously detect the session state of the video session after monitoring the data received by the video receiving system in the video session; when it is detected that the session state of the video session is session termination, stop monitoring the data received by the video receiving system in the video session.

[0110] Optionally, the above device further comprises: The fourth determination unit is used to determine that the first video data packet is a retransmitted data packet after at least determining the target sequence number in the first video data packet, if the target sequence number is not greater than the minimum sequence number, and the sum of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, and prohibit the target sequence number from being stored in the first queue.

[0111] The packet loss processing device in video data transmission proposed in this embodiment can determine the sequence number corresponding to the lost data packet, that is, the sequence number to be retransmitted, through the new data packet determination condition and the packet loss determination mechanism in the video session established between the video receiving system and the video sending system, and instruct the video sending system to retransmit the lost video data packet to the video receiving system through the video session, thereby realizing the retransmission of the lost data packet and ensuring the video quality of the video receiving system.

[0112] The packet loss processing device in the video data transmission in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0113] The embodiment of the present invention also provides a computer device having the above Figure 8 The packet loss processing device in video data transmission is shown.

[0114] See also Fig. 9 , a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0115] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0116] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0117] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function. The data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The memory 20 may include a volatile memory, such as a random access memory. The memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive. The memory 20 may also include a combination of the above-mentioned types of memory.

[0119] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0120] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing packet loss in video data transmission, characterized in that: include: When monitoring that the video receiving system receives any first video data packet in a video session, at least determining a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; If the target sequence number is greater than the minimum sequence number of each sequence number stored in the first queue, or the sum of the target sequence number and the set maximum out-of-order positive offset is greater than the maximum sequence number in the first queue, it is determined that the first video data packet is a newly sent data packet, and the target sequence number is stored in the first queue; When it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, the minimum sequence number is subtracted from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; If the sequence number difference is greater than 1, it is determined that packet loss has occurred, and each integer greater than the minimum sequence number and less than the second minimum sequence number is determined as a sequence number to be retransmitted; The minimum sequence number is deleted from the first queue, and the video sending system is instructed to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

2. The method according to claim 1, characterized in that The monitoring that the video receiving system receives any first video data packet in the video session, then at least determining the target sequence number in the first video data packet, comprises: When monitoring that the video receiving system and the video sending system have completed the establishment of the video session, acquiring session information of the video session; wherein the session information of the video session includes address information and port information of the video receiving system, and address information and port information of the video sending system; Monitoring data received by the video receiving system in the video session according to the address information and port information of the video receiving system in the session information, and the address information and port information of the video sending system; If it is monitored that the video receiving system receives any of the first video data packets in the video session, at least a target sequence number in the first video data packet is determined.

3. The method according to claim 2, characterized in that Before monitoring that the video receiving system receives any of the first video data packets in the video session, the method further includes: Create an empty queue, and use the empty queue as the first queue; After at least determining the target sequence number in the first video data packet, if the sequence number is not stored in the first queue, or the first video data packet is the first first video data packet received by the video receiving system in the video session, then the first video data packet is determined to be a newly sent data packet, and the target sequence number in the first video data packet is stored in the first queue.

4. The method according to claim 1, characterized in that: The at least determining the target sequence number in the first video data packet comprises: Determining a target sequence number in the first video data packet and a time at which the video receiving system receives the first video data packet; After determining that the first video data packet is a newly sent data packet, the method further includes: Correspondingly saving the target sequence number and the receiving time in the first video data packet; The monitoring that the total number of sequence numbers in the first queue reaches a preset threshold, subtracting the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference, includes: If it is monitored that the total number of sequence numbers in the first queue reaches the preset threshold, or the receiving duration corresponding to the minimum sequence number exceeds the preset duration threshold, the minimum sequence number is subtracted from the second minimum sequence number to obtain the sequence number difference; The receiving duration is obtained by subtracting the receiving time corresponding to the minimum sequence number from the current time.

5. The method according to claim 1, characterized in that The video sending system includes a media service and a monitoring service; the media service is used to send a plurality of second video data packets to the video receiving system through the video session; the monitoring service is used to monitor that the media service sends any second video data packet in the video session, and then store the monitored second video data packet in the created second queue; The instructing the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session includes: Generate a data packet retransmission instruction according to each of the sequence numbers to be retransmitted, wherein the data packet retransmission instruction includes each of the sequence numbers to be retransmitted; The data packet retransmission instruction is sent to the monitoring service so that the monitoring service: searches for the corresponding second video data packet in the second queue according to each of the to-be-retransmitted sequence numbers in the data packet retransmission instruction, and sends each found second video data packet to the video receiving system through the video session.

6. The method according to claim 2, characterized in that After monitoring the data received by the video receiving system in the video session, the method further includes: Continuously detecting the session state of the video session; When it is detected that the session state of the video session is session termination, the monitoring of the data received by the video receiving system in the video session is stopped.

7. The method according to any one of claims 1 to 6, characterized in that After at least determining the target sequence number in the first video data packet, the method further includes: If the target sequence number is not greater than the minimum sequence number, and the sum of the target sequence number and the maximum out-of-order positive offset is not greater than the maximum sequence number, the first video data packet is determined to be a retransmitted data packet, and the target sequence number is prohibited from being stored in the first queue.

8. A packet loss processing device in video data transmission, characterized in that: include A monitoring unit, configured to monitor that the video receiving system receives any first video data packet in a video session, and then at least determine a target sequence number in the first video data packet; wherein the video session is established between the video receiving system and the video sending system; A first determining unit, configured to determine that the first video data packet is a newly transmitted data packet if the target sequence number is greater than a minimum sequence number among each sequence number stored in the first queue, or if a sum of the target sequence number and a set maximum out-of-order positive offset is greater than a maximum sequence number in the first queue; A storage unit, configured to store the target sequence number in the first queue; a monitoring unit, configured to, when it is detected that the total number of sequence numbers in the first queue reaches a preset threshold, subtract the minimum sequence number from the second minimum sequence number in the first queue to obtain a corresponding sequence number difference; A second determining unit, configured to determine that packet loss occurs if the sequence number difference is greater than 1; A third determining unit, configured to determine each integer greater than the minimum sequence number and less than the second minimum sequence number as a sequence number to be retransmitted; a deleting unit, configured to delete the minimum sequence number in the first queue; The instruction unit is used to instruct the video sending system to resend each video data packet corresponding to the sequence number to be resent to the video receiving system through the video session.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the packet loss processing method in video data transmission according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the packet loss processing method in video data transmission according to any one of claims 1 to 7.

Images