Video data retransmission method and device, equipment and medium

Abstract

The invention relates to the technical field of video data transmission, and provides a video data retransmission method, device and equipment and a medium, and the method comprises the steps: calculating the average fragment number of a video frame according to the fragment number at the previous N moments, and calculating a delay coefficient and a retransmission serial number difference value according to all to-be-retransmitted fragments; when the average fragment number of the video frame, the delay coefficient and the retransmission serial number difference meet a retransmission delay decision condition, determining a plurality of target sending fragments from all the fragments to be retransmitted and all the normal sending fragments in the video sending queue, and sending all the target sending fragments to a video data receiving end; and when the average fragment number of the video frame, the delay coefficient and the retransmission serial number difference value do not meet a retransmission delay decision condition, determining a plurality of target retransmission fragments from all the fragments to be retransmitted in the video transmission queue, and transmitting all the target retransmission fragments to a video data receiving end. According to the method, the retransmission delay of the video data can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of video data transmission, and in particular to a video data retransmission method, device, equipment and medium. Background Art

[0002] In order to meet the real-time nature of video transmission, the real-time video transmission system mainly uses the User Datagram Protocol (UDP) as the communication protocol of the transport layer.

[0003] Unlike the Transmission Control Protocol (TCP), the UDP protocol does not deal with network transmission problems such as packet loss, disorder, jitter, and delay during the transmission process.

[0004] Therefore, the transmission of audio and video data usually requires one or more layers of streaming protocols to handle these problems. Streaming protocols are usually encapsulated on top of UDP protocols, such as the common real-time transport protocol / real-time transport control protocol, network protocols used to protect real-time communication data, etc.

[0005] In fact, no matter which transmission protocol is used, including the TCP protocol, when packet loss occurs in network transmission, the most common method is to request data retransmission from the sender to recover the lost data.

[0006] The data retransmission process needs to go through the entire process of "receiving end request -> sending end responds and resends data -> receiving end re-receives". This process is the main reason why the transmission delay will increase when network fluctuations occur during data transmission over the network.

[0007] In real-time video transmission systems, streaming protocols usually use data retransmission as the main means to deal with network packet loss.

[0008] Since video coding mainly reduces data redundancy by "referencing" other video frames, video coding data has inter-frame dependencies.

[0009] Inter-frame dependency determines that the video transmission system needs to ensure a certain degree of data sequence and integrity. Otherwise, the video at the receiving end may appear distorted, black, pixelated, unable to be decoded, stuck, etc.

[0010] The real-time video transmission system uses data retransmission to deal with the above problems, which is the main reason for its delay. Since data retransmission technology has high recovery efficiency in most scenarios and can guarantee a certain degree of reliability, it is still indispensable in the real-time video transmission system. In the current video data retransmission technology, the number of retransmissions at the sender and the delay of the receiving queue are determined according to the packet loss rate and the round-trip delay of the transmission between the two communicating parties. The network transmission state deteriorates, increasing the number of retransmissions and buffer delay. However, under certain special network conditions, this strategy will deteriorate the network transmission channel due to excessive retransmission data, and fall into a vicious cycle of "the worse the network, the more retransmissions, further blocking the network, and the greater the delay"; in addition, the method of using the data fed back by the receiving end to obtain the actual receiving bit rate of the current receiving end and reducing the sending bit rate according to the actual receiving bit rate by the sending module can reduce the transmission rate in real time according to the network transmission state, but it cannot effectively solve the problem of increased delay caused by reducing the transmission rate, and it is easy to enter a state where the transmission rate continues to decrease and cannot be called back in time. According to the type of service, such as video calls, live broadcasts, and file transfers, the method of adjusting the data transmission path in combination with the current network transmission status can improve the network transmission link for audio and video data. However, in the scenario where the consumer end accesses the service through home broadband or operator hotspots, when the access point has a network bottleneck, the optional network access path is very limited, and it is almost impossible to switch the transmission link, so the optimization effect is very limited. The method of calculating the window retransmission threshold, deleting the data to be retransmitted that exceeds the retransmission number, reducing the current amount of retransmitted data, and then reducing the delay can control the amount of retransmitted data sent, and avoid the sudden increase in delay caused by excessive data retransmission. However, after the network fluctuation reaches a certain critical level, there is too much data that needs to be retransmitted, and the data that exceeds the retransmission threshold is postponed to the next observation window, which will still increase the cumulative video delay. It can be seen that the existing video data retransmission method has the problem of high video data retransmission delay. Summary of the invention

[0011] The present application provides a video data retransmission method, apparatus, device and medium, which can solve the problem of high delay in video data retransmission.

[0012] In a first aspect, an embodiment of the present application provides a video data retransmission method, the video data retransmission method comprising:

[0013] Obtain all the segments to be retransmitted in the video transmission queue of the video data transmitter at the current moment, and the number of segments of the video frame to be transmitted in the previous N moments; the Nth moment is the moment before the current moment;

[0014] The average number of video frame segments is calculated based on the number of segments at the previous N moments, and the delay coefficient and the difference in retransmission sequence numbers are calculated based on all the segments to be retransmitted;

[0015] When the average number of video frame fragments, the delay coefficient, and the difference in retransmission sequence numbers meet the retransmission delay decision condition, multiple target transmission fragments are determined from all the to-be-retransmitted fragments and all the normal transmission fragments in the video transmission queue, and all the target transmission fragments are sent to the video data receiving end; the normal transmission fragments are the fragments that are added to the video transmission queue for the first time;

[0016] When the average number of video frame fragments, the delay coefficient and the difference in retransmission sequence numbers do not meet the retransmission delay decision conditions, multiple target retransmission fragments are determined from all the to-be-retransmitted fragments in the video transmission queue, and all target retransmission fragments are sent to the video data receiving end.

[0017] Optionally, the average number of fragments of the video frame is calculated based on the number of fragments at the previous N moments, including:

[0018] By formula:

[0019]

[0020] Calculate the average number of video frame fragments S a ;

[0021] Among them, NF r represents the number of all video frames to be transmitted at N moments, S i Indicates the number of slices of the i-th video frame.

[0022] Optionally, the delay factor and the retransmission sequence number difference are calculated based on all the fragments to be retransmitted, including:

[0023] Obtain the number of retransmissions of each to-be-retransmitted segment, and calculate the delay factor based on all the retransmissions;

[0024] Get the sequence number of each fragment to be retransmitted, and calculate the retransmission sequence number difference based on all sequence numbers.

[0025] Optionally, the delay factor is calculated based on all retransmission times, including:

[0026] By formula:

[0027]

[0028] Calculate the delay factor F d ;

[0029] Among them, R c Represents the maximum value of all retransmission times, R m Indicates the maximum number of retransmissions.

[0030] Optionally, the retransmission sequence number difference is calculated based on all sequence numbers, including:

[0031] By formula:

[0032] M d =SNV m -SNNR m

[0033] Calculate the retransmission sequence number difference M d ;

[0034] Among them, SNV m Indicates the maximum sequence number among all sequence numbers, SNR m Indicates the smallest sequence number among all sequence numbers.

[0035] Optionally, the retransmission delay decision condition is:

[0036]

[0037] Among them, M d Indicates the difference in retransmission sequence number, F d Denotes the delay factor, F r Indicates the frame rate, S a Indicates the average number of video frame fragments, B k Indicates the bit size of the most recent key frame, and R indicates the current sending bit rate.

[0038] Optionally, multiple target sending segments are determined from all the to-be-retransmitted segments and all the normally sent segments in the video sending queue, including:

[0039] According to the priorities of all the to-be-retransmitted segments and all the normally-transmitted segments in the video transmission queue, a plurality of target transmission segments are determined from all the to-be-retransmitted segments and all the normally-transmitted segments in the video transmission queue;

[0040] Multiple target retransmission segments are determined from all the segments to be retransmitted in the video transmission queue, including:

[0041] According to the priorities of all the to-be-retransmitted fragments in the video transmission queue, a plurality of target retransmitted fragments are determined from all the to-be-retransmitted fragments in the video transmission queue.

[0042] In a second aspect, the present application provides a video data sending device, which is applied to a video data sending end, and the video data sending device includes:

[0043] The acquisition module is used to acquire all the to-be-retransmitted segments in the video transmission queue of the video data transmitting end at the current moment, and the number of segments of the video frame to be transmitted at the previous N moments; the Nth moment is the moment before the current moment;

[0044] A calculation module, used to calculate the average number of video frame fragments according to the number of fragments at the previous N moments, and calculate the delay coefficient and the difference in retransmission sequence number according to all fragments to be retransmitted;

[0045] The first sending module is used to determine a plurality of target sending segments from all the to-be-retransmitted segments and all the normally sent segments in the video sending queue when the average number of segments of the video frame, the delay coefficient and the difference in the retransmission sequence number meet the retransmission delay decision condition, and send all the target sending segments to the video data receiving end; the normally sent segments are the segments that are added to the video sending queue for the first time;

[0046] The second sending module is used to determine multiple target retransmission fragments from all the fragments to be retransmitted in the video sending queue when the average number of fragments of the video frame, the delay coefficient and the difference in the retransmission sequence number do not meet the retransmission delay decision conditions, and send all the target retransmission fragments to the video data receiving end.

[0047] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned video data retransmission method when executing the above-mentioned computer program.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned video data retransmission method is implemented.

[0049] The above solution of the present application has the following beneficial effects:

[0050] In an embodiment of the present application, by obtaining all the to-be-retransmitted fragments in the video sending queue of the video data sending end at the current moment, and the number of fragments of the video frame to be transmitted in the previous N moments, then calculating the average number of fragments of the video frame according to the number of fragments at the previous N moments, and calculating the delay coefficient and the retransmission sequence number difference according to all the to-be-retransmitted fragments, when the average number of fragments of the video frame, the delay coefficient and the retransmission sequence number difference meet the retransmission delay decision condition, determining multiple target transmission fragments from all the to-be-retransmitted fragments and all the normally transmitted fragments in the video sending queue, and sending all the target transmission fragments to the video data receiving end, when the average number of fragments of the video frame, the delay coefficient and the retransmission sequence number difference do not meet the retransmission delay decision condition, determining multiple target retransmission fragments from all the to-be-retransmitted fragments in the video sending queue, and sending all the target retransmission fragments to the video data receiving end. Among them, whether the delay decision condition is met is judged according to the average number of video frame fragments, delay coefficient and retransmission sequence number difference of the video data sending end, and different data sending processes are executed. The actual data transmission status of the video data sending end at the current moment is taken into consideration, the rationality and practicality of video data retransmission are improved, and then the video data retransmission delay is reduced.

[0051] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A flowchart of a video data retransmission method provided by an embodiment of the present application;

[0054] Figure 2 A specific process diagram of a video data retransmission method provided in one embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of a video data sending device provided in one embodiment of the present application;

[0056] Figure 4 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0057] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0058] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0059] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0060] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0061] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0062] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0063] In response to the problem of high retransmission delay of existing video data, the video data sending end of the embodiment of the present application obtains all the to-be-retransmitted fragments in the video sending queue of the video data sending end at the current moment, and the number of fragments of the video frame to be transmitted in the previous N moments, and then calculates the average number of fragments of the video frame according to the number of fragments in the previous N moments, and calculates the delay coefficient and the retransmission sequence number difference according to all the to-be-retransmitted fragments. When the average number of fragments of the video frame, the delay coefficient and the retransmission sequence number difference meet the retransmission delay decision condition, multiple target transmission fragments are determined from all the to-be-retransmitted fragments and all the normally transmitted fragments in the video sending queue, and all the target transmission fragments are sent to the video data receiving end. When the average number of fragments of the video frame, the delay coefficient and the retransmission sequence number difference do not meet the retransmission delay decision condition, multiple target retransmission fragments are determined from all the to-be-retransmitted fragments in the video sending queue, and all the target retransmission fragments are sent to the video data receiving end. Among them, whether the delay decision condition is met is judged according to the average number of video frame fragments, delay coefficient and retransmission sequence number difference of the video data sending end, and different data sending processes are executed. The actual data transmission status of the video data sending end at the current moment is taken into consideration, the rationality and practicality of video data retransmission are improved, and then the video data retransmission delay is reduced.

[0064] Next, an exemplary description is given of the video data retransmission method provided in the present application and applied to the video data sending end.

[0065] The video data sending end may be a network camera, a live streaming tool or other equipment.

[0066] like Figure 1 As shown, the video data retransmission method provided by the present application and applied to the video data sending end includes the following steps:

[0067] Step 11, obtaining all the to-be-retransmitted segments in the video transmission queue of the video data transmitting end at the current moment, and the number of segments of the video frames to be transmitted in the previous N moments.

[0068] The Nth moment is the moment before the current moment. The time interval between the N moments can be based on the sending cycle of the video data sending end. For example, if the current moment is 1 minute and 20 seconds, the sending cycle is 5 seconds, and N is 4, then the first 4 moments are 1 minute, 1 minute and 5 seconds, 1 minute and 10 seconds, and 1 minute and 15 seconds respectively. The video frames to be transmitted in the first N moments are the video frames that have been encoded and fragmented in the first N moments and added to the video sending queue. At the current moment, the fragments of these video frames may have been transmitted to the video data receiving end, or they may be in the video sending queue and have not yet been sent.

[0069] In some embodiments of the present application, all the fragments to be retransmitted in the video sending queue and the number of fragments of the video frames transmitted in the previous N moments can be obtained by accessing the software, equipment, etc. used to monitor the video sending queue in the video data sending end.

[0070] It should be noted that video frame fragmentation means that when a real-time video transmission system uses a UDP-based streaming protocol, video frame data is usually divided into smaller data packets, whose bit size is related to the maximum transmission unit of network transmission.

[0071] Exemplarily, if there are no segments to be retransmitted in the current video transmission queue, the segments are transmitted to the video data receiving end in the order of the normally transmitted segments (ie, the segments of the video frames transmitted for the first time) in the video transmission queue.

[0072] Step 12, calculating the average number of video frame segments according to the number of segments at the previous N moments, and calculating the delay coefficient and the difference in retransmission sequence number according to all the segments to be retransmitted.

[0073] In some embodiments of the present application, the above-mentioned step of calculating the average number of fragments of the video frame according to the number of fragments at the previous N moments, and calculating the delay coefficient and the difference in retransmission sequence number according to all fragments to be retransmitted includes:

[0074] The first step is to calculate the average number of video frame fragments based on the number of fragments at the previous N moments.

[0075] Specifically, through the formula:

[0076]

[0077] Calculate the average number of video frame fragments S a .

[0078] Among them, NF r represents the number of all video frames to be transmitted at N moments, S i Indicates the number of slices of the i-th video frame.

[0079] The second step is to obtain the number of retransmissions of each segment to be retransmitted, and calculate the delay coefficient based on all the retransmissions.

[0080] Specifically, through the formula:

[0081]

[0082] Calculate the delay factor F d .

[0083] Among them, R c Represents the maximum value of all retransmission times, R mIndicates the maximum number of retransmissions.

[0084] The third step is to obtain the sequence number of each fragment to be retransmitted, and calculate the retransmission sequence number difference based on all sequence numbers.

[0085] Specifically, through the formula:

[0086] M d =SNV m -SNR m

[0087] Calculate the retransmission sequence number difference M d .

[0088] Among them, SNV m Indicates the maximum sequence number among all sequence numbers, SNR m Indicates the smallest sequence number among all sequence numbers.

[0089] It should be noted that the number of retransmissions is the number of times the fragment is currently retransmitted. The fragments sent according to the normal process will not be destroyed immediately after the transmission is completed, but will be placed in the history queue for a period of time. During this period, if a retransmission request is received and a response is retransmitted, the corresponding retransmission number record will be increased. The larger this value is, the greater the delay caused by retransmitting the corresponding fragment. After the video data frame is divided into data fragments with smaller bit sizes, they are usually numbered in sequence to obtain the sequence number of the fragment. The retransmission sequence number difference calculated in this step represents the difference between the sequence number of the latest fragment to be retransmitted (i.e., the fragment to be retransmitted that entered the video sending queue most recently) and the sequence number of the oldest fragment to be retransmitted (i.e., the fragment to be retransmitted that entered the current video sending queue the earliest), which is an important feature used to measure the delay between the current receiving end and the sending end.

[0090] Step 13, when the average number of video frame fragments, the delay coefficient and the difference in retransmission sequence numbers meet the retransmission delay decision conditions, multiple target transmission fragments are determined from all the fragments to be retransmitted and all the normal transmission fragments in the video transmission queue, and all the target transmission fragments are sent to the video data receiving end.

[0091] The above-mentioned normally sent segments are segments that are added to the video sending queue for the first time.

[0092] Specifically, the retransmission delay decision conditions are:

[0093]

[0094] Among them, M d Indicates the difference in retransmission sequence number, F d Denotes the delay factor, F r Indicates the frame rate, S aIndicates the average number of video frame fragments, B k Indicates the bit size of the most recent key frame, and R indicates the current sending bit rate.

[0095] It should be noted that the ratio of the key frame size to the current transmission bit rate is the main source of delay in sending key frames. When the sender receives a large number of retransmission requests, it is usually known that a large number of packets are lost in the network. In engineering practice, other technologies, such as increasing redundancy, are usually combined to combat packet loss. Therefore, it is assumed here that key frames will not need to be retransmitted in the future to solve the packet loss problem.

[0096] The above-mentioned step of determining multiple target sending fragments from all the fragments to be retransmitted and all the normally sent fragments in the video sending queue is specifically: according to the priorities of all the fragments to be retransmitted and all the normally sent fragments in the video sending queue, multiple target sending fragments are determined from all the fragments to be retransmitted and all the normally sent fragments in the video sending queue.

[0097] It should be noted that the above-mentioned priority can be the sequence number of the fragment. The rule is that the sequence number of the fragment that enters the video sending queue first is small, and multiple fragments with earlier sequence numbers are selected as target sending fragments. The number of target sending fragments is determined according to the number of fragments sent in each cycle preset by the video data sending end.

[0098] Exemplarily, when performing this step, no response is made to the packet loss retransmission request (used to describe the need for data retransmission) received from the video data receiving end. After performing this step, if the key frame needs to be transmitted at the next moment of the current moment, the key frame is encoded and fragmented and added to the video transmission queue.

[0099] Step 14, when the average number of video frame fragments, the delay coefficient and the difference in retransmission sequence number do not meet the retransmission delay decision condition, multiple target retransmission fragments are determined from all the to-be-retransmitted fragments in the video transmission queue, and all the target retransmission fragments are sent to the video data receiving end.

[0100] The above-mentioned step of determining a plurality of target retransmission segments from all the to-be-retransmitted segments in the video transmission queue is specifically as follows:

[0101] According to the priorities of all the to-be-retransmitted fragments in the video transmission queue, a plurality of target retransmitted fragments are determined from all the to-be-retransmitted fragments in the video transmission queue.

[0102] It should be noted that the above-mentioned priority can be the sequence number of the fragment. The rule is that the sequence number of the fragment to be retransmitted that enters the video sending queue first is small, and multiple fragments to be retransmitted with the front sequence numbers are selected as target retransmitted fragments. The number of target retransmitted fragments is determined according to the number of fragments sent in each cycle preset by the video data sending end.

[0103] Exemplarily, when executing this step, priority is given to the to-be-retransmitted fragments, and the target retransmitted fragments are obtained from all the to-be-retransmitted fragments. Only when the number of all the to-be-retransmitted fragments is less than the number of the target retransmitted fragments, the normal sending of the fragments is considered.

[0104] It is worth mentioning that whether the delay decision conditions are met is judged according to the average number of video frame fragments, delay coefficient and retransmission sequence number difference at the video data sending end, and different data sending processes are executed. The actual data transmission status of the video data generating end at the current moment is taken into account, thereby improving the rationality and practicality of video data retransmission, thereby reducing the video data retransmission delay.

[0105] The video data retransmission method of the present application is exemplarily described below with reference to a specific example.

[0106] like Figure 2 As shown, first send the video data fragments to the video sending queue, then calculate the average number of video frame fragments in the observation window (i.e., N moments), determine whether there are data fragments to be retransmitted (i.e., fragments to be retransmitted), if not, send ordinary video data fragments and return to the step of sending video data fragments, if yes, calculate the delay coefficient of the retransmitted video data fragments, and calculate the difference in retransmission sequence numbers to determine whether the retransmission delay decision condition is met, if so, clear the historical sending queue, do not respond to packet loss retransmission requests, clear all video frame fragments to be sent and encode key frames, and then return to the step of sending video data fragments, otherwise, send the video data fragments to be retransmitted, and return to the step of sending video data fragments.

[0107] According to the above process, test in a simulated test environment:

[0108] Network impairment settings: 3Mbps limited bandwidth, 20% random packet loss, 30-60ms round-trip latency.

[0109] Video encoding: H.264 encoding format.

[0110] Operation method: The timing starts when the receiving end receives the video and the video screen appears. At the 15th second, network damage is enabled; at the 25th second, network damage is disabled; at the 45th second, network damage is enabled again; at the 55th second, network damage is disabled; and at the 75th second, this round of testing ends.

[0111] The test results are shown in Table 1:

[0112]

[0113] Table 1

[0114] A total of 4 tests were conducted. The before-and-after test time in the same test is within 3 minutes. base represents the unoptimized version of the baseline, opt represents the optimized version of this method, average delay, in milliseconds, the end-to-end delay of the video calculated by the transmission system. Average freeze rate, in percentage, the video freeze rate calculated by the transmission system. Average frame rate, in frames per second, the video frame rate calculated by the transmission system. PSNR represents the signal-to-noise ratio, in dB, the video encoding quality calculated by the transmission system. Average sending delay, in milliseconds, the sending delay of the sending module calculated by the transmission system. All data are calculated once per second, with each test time length of 75s as the average.

[0115] It can be seen that the delay index is significantly improved before and after optimization using the method of the present application, and the jam rate index is also improved, and there is no obvious degradation in the encoding quality. The frame rate has a slight negative benefit, but the reduction is only 1-2 frames.

[0116] The following is an exemplary description of the video data sending end provided in this application.

[0117] like Figure 3 As shown, an embodiment of the present application provides a video data sending device, which is applied to a video data sending end. The video data sending device 300 includes:

[0118] The acquisition module 301 is used to acquire all the to-be-retransmitted segments in the video transmission queue of the video data transmitting end at the current moment, and the number of segments of the video frame to be transmitted at the previous N moments; the Nth moment is the moment before the current moment;

[0119] A calculation module 302 is used to calculate the average number of video frame segments according to the number of segments at the previous N moments, and calculate the delay coefficient and the difference in retransmission sequence number according to all the segments to be retransmitted;

[0120] The first sending module 303 is used to determine a plurality of target sending segments from all the to-be-retransmitted segments and all the normal sending segments in the video sending queue when the average number of segments of the video frame, the delay coefficient and the difference in the retransmission sequence number meet the retransmission delay decision condition, and send all the target sending segments to the video data receiving end; the normal sending segment is the segment that is added to the video sending queue for the first time;

[0121] The second sending module 304 is used to determine multiple target retransmission fragments from all the to-be-retransmitted fragments in the video sending queue when the average number of fragments of the video frame, the delay coefficient and the difference in the retransmission sequence number do not meet the retransmission delay decision condition, and send all the target retransmission fragments to the video data receiving end.

[0122] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0123] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0124] like Figure 4 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0125] Specifically, when the processor D100 executes the computer program D102, it obtains all the to-be-retransmitted fragments in the video sending queue of the video data sending end at the current moment, and the number of fragments of the video frame to be transmitted in the previous N moments, and then calculates the average number of fragments of the video frame according to the number of fragments in the previous N moments, and calculates the delay coefficient and the retransmission sequence number difference according to all the to-be-retransmitted fragments. When the average number of fragments of the video frame, the delay coefficient, and the retransmission sequence number difference meet the retransmission delay decision condition, multiple target transmission fragments are determined from all the to-be-retransmitted fragments and all the normally transmitted fragments in the video sending queue, and all the target transmission fragments are sent to the video data receiving end. When the average number of fragments of the video frame, the delay coefficient, and the retransmission sequence number difference do not meet the retransmission delay decision condition, multiple target retransmission fragments are determined from all the to-be-retransmitted fragments in the video sending queue, and all the target retransmission fragments are sent to the video data receiving end. Among them, whether the delay decision condition is met is judged according to the average number of video frame fragments, delay coefficient and retransmission sequence number difference of the video data sending end, and different data sending processes are executed. The actual data transmission status of the video data sending end at the current moment is taken into consideration, the rationality and practicality of video data retransmission are improved, and then the video data retransmission delay is reduced.

[0126] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0127] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0128] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0129] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the video data retransmission method device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video data retransmission method, characterized in that: Applied to a video data transmitting end, the video data retransmission method comprises: Obtain all the segments to be retransmitted in the video transmission queue of the video data transmitter at the current moment, and the number of segments of the video frame to be transmitted in the previous N moments; the Nth moment is the moment before the current moment; Calculate the average number of video frame fragments according to the number of fragments at the first N moments, and calculate the delay coefficient and the difference in retransmission sequence number according to all fragments to be retransmitted; When the average number of video frame fragments, the delay coefficient and the difference in retransmission sequence numbers meet the retransmission delay decision condition, multiple target transmission fragments are determined from all the to-be-retransmitted fragments and all the normally transmitted fragments in the video transmission queue, and all the target transmission fragments are sent to the video data receiving end; the normally transmitted fragments are fragments that are added to the video transmission queue for the first time; When the average number of video frame fragments, the delay coefficient and the retransmission sequence number difference do not meet the retransmission delay decision condition, multiple target retransmission fragments are determined from all the to-be-retransmitted fragments in the video sending queue, and all the target retransmission fragments are sent to the video data receiving end.

2. The video data retransmission method according to claim 1, characterized in that: The calculating the average number of fragments of the video frame according to the number of fragments at the first N moments includes: By formula: Calculate the average number of video frame fragments S a ; Among them, NF r S represents the number of all video frames to be transmitted at N moments. i Indicates the number of slices of the i-th video frame.

3. The video data retransmission method according to claim 1, characterized in that: The step of calculating the delay coefficient and the retransmission sequence number difference according to all the fragments to be retransmitted includes: Obtain the number of retransmissions of each to-be-retransmitted segment, and calculate the delay factor based on all the retransmissions; Get the sequence number of each fragment to be retransmitted, and calculate the retransmission sequence number difference based on all sequence numbers.

4. The video data retransmission method according to claim 3, characterized in that: The calculating of the delay coefficient based on all retransmission times includes: By formula: Calculate the delay factor F d ; Among them, R c Represents the maximum value of all retransmission times, R m Indicates the maximum number of retransmissions.

5. The video data retransmission method according to claim 3, characterized in that: The calculating of the retransmission sequence number difference based on all sequence numbers includes: By formula: M d =SNV m -SNR m Calculate the retransmission sequence number difference M d ; Among them, SNV m Indicates the maximum sequence number among all sequence numbers, SNR m Indicates the smallest sequence number among all sequence numbers.

6. The video data retransmission method according to claim 1, characterized in that: The retransmission delay decision condition is: Among them, M d Indicates the difference in retransmission sequence number, F d Denotes the delay factor, F r Indicates the frame rate, S a Indicates the average number of video frame fragments, B k Indicates the bit size of the most recent key frame, and R indicates the current sending bit rate.

7. The video data retransmission method according to claim 1, characterized in that: The step of determining a plurality of target transmission segments from all the to-be-retransmitted segments and all the normally transmitted segments in the video transmission queue comprises: According to the priorities of all the to-be-retransmitted segments and all the normally-transmitted segments in the video transmission queue, a plurality of target transmission segments are determined from all the to-be-retransmitted segments and all the normally-transmitted segments in the video transmission queue; The step of determining a plurality of target retransmission segments from all the to-be-retransmitted segments in the video transmission queue comprises: According to the priorities of all the to-be-retransmitted segments in the video transmission queue, a plurality of target retransmitted segments are determined from all the to-be-retransmitted segments in the video transmission queue.

8. A video data sending device, characterized in that: Applied to a video data sending end, the video data sending device comprises: The acquisition module is used to acquire all the to-be-retransmitted segments in the video transmission queue of the video data transmitting end at the current moment, and the number of segments of the video frame to be transmitted at the previous N moments; the Nth moment is the moment before the current moment; A calculation module, used to calculate the average number of video frame fragments according to the number of fragments at the first N moments, and calculate the delay coefficient and the difference in retransmission sequence number according to all fragments to be retransmitted; A first sending module is used to determine a plurality of target sending segments from all the to-be-retransmitted segments and all the normally sent segments in the video sending queue when the average number of segments of the video frame, the delay coefficient and the difference of the retransmission sequence number meet the retransmission delay decision condition, and send all the target sending segments to the video data receiving end; the normally sent segments are segments that are added to the video sending queue for the first time; The second sending module is used to determine multiple target retransmission fragments from all the fragments to be retransmitted in the video sending queue when the average number of fragments of the video frame, the delay coefficient and the difference between the retransmission sequence numbers do not meet the retransmission delay decision condition, and send all the target retransmission fragments to the video data receiving end.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the video data retransmission method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video data retransmission method according to any one of claims 1 to 7 is implemented.

Images