A method, device and electronic device for predicting packet loss

Through the signal quality evaluation and packet loss prediction model combined with ACK packets and round trip delay, the packet loss problem in video playback of smart cameras is solved, improving the accuracy and stability of video transmission, and avoiding lag and delay.

CN118890526BActive Publication Date: 2024-12-06ZHEJIANG DAHUA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411388531.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2024-12-06
Estimated Expiration
2044-09-30

AI Technical Summary

Technical Problem

When smart cameras are connected to Wi-Fi networks, due to factors such as installation location, wireless interference and network delay, packet loss problems in network transmission layer, resulting in problems such as stuttering and delay in video playback. The existing technology adopts heuristic algorithms. Packet loss recognition is performed, with low accuracy.

Method used

The trained signal quality evaluation model obtains the signal quality value of the current network environment. If it is less than the preset threshold, input the target feature parameter set to packet loss prediction model for packet loss prediction, and determine packet loss by combining ACK packets and round trip delays. interval, adjust the transmission strategy to avoid video lag and delay.

Benefits of technology

It improves the accuracy of packet loss prediction, significantly reduces the lag and delay problems in video playback, and ensures the continuity and integrity of video transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118890526B_ABST
    Figure CN118890526B_ABST
Patent Text Reader

Abstract

The present application discloses a data packet loss prediction method, device and electronic device, which relate to the field of data processing technology and are used to improve the accuracy of data packet loss prediction. The method comprises: first, based on the transmitted video data, a signal quality value under the current network environment is obtained through a trained signal quality assessment model; then, if the signal quality value is less than a preset threshold, the target feature parameter set is input into the trained packet loss prediction model to predict the packet loss of the video data, and a first packet loss interval is obtained. Through the above method, the packet loss prediction is performed using feature data of multiple dimensions, which can improve the accuracy of the model in predicting packet loss, and thus can improve problems such as video playback freeze and picture delay caused by packet loss.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and electronic device for predicting data packet loss. Background Art

[0002] At present, smart cameras are widely used for real-time monitoring in various scenarios. Smart cameras can be connected to the public cloud platform through Wi-Fi networks, and users can access the cloud platform on their terminal devices to view real-time video or alarm recordings in the monitoring area.

[0003] However, when smart cameras are connected to a Wi-Fi network, they are prone to packet loss at the network transmission layer due to factors such as installation location, wireless interference, and network delay, which can cause problems such as video playback freezes, frame loss, and delays.

[0004] In the related art, an acknowledgement character (ACK) is usually used for packet loss identification. However, this method uses a heuristic algorithm, that is, using experience or a fixed formula to identify packet loss, which has the problem of low packet loss identification accuracy. Summary of the invention

[0005] The embodiments of the present application provide a data packet loss prediction method, device and electronic device to improve the prediction accuracy of data packet loss, improve video playback quality, and avoid freezes, delays, and the like.

[0006] In a first aspect, the present application provides a method for predicting data packet loss, the method comprising:

[0007] Based on the sent video data, the signal quality value in the current network environment is obtained through the trained signal quality assessment model; wherein the sent video data consists of N video packets, N is a positive integer greater than or equal to 1, and the signal quality assessment model is trained based on the wireless signal characteristic parameters corresponding to the historical video playback quality data;

[0008] If the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to predict packet loss for the video data to obtain a first packet loss interval; wherein the target feature parameter set includes: video feature parameters of the sent video data, network feature parameters of the video sending end, and wireless signal feature parameters.

[0009] Through the above method, packet loss prediction is performed on data packet transmission based on multi-dimensional feature parameters, which improves the accuracy of packet loss prediction, and enables the video sender to adjust the transmission strategy in time to avoid problems such as video freeze and picture delay.

[0010] In an optional implementation, if the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data, and before obtaining the first packet loss interval, the method further includes:

[0011] Determine whether an ACK message sent by the video receiving end is received;

[0012] If yes, the second packet loss interval and the received video packet interval are parsed from the ACK message.

[0013] Through the above method, based on the ACK message fed back by the video receiving end, the packet loss situation in the current network environment can be determined.

[0014] In an optional implementation manner, after parsing the second packet loss interval and the received video packet interval from the ACK message, the method further includes:

[0015] Get the round-trip time (RTT) corresponding to the current network environment;

[0016] If the RTT is greater than a preset threshold, a third packet loss interval is determined according to the N video packets and the received video packet interval.

[0017] Through the above method, according to the round-trip time RTT corresponding to the current network environment and the received video packet interval, the packet loss situation when the RTT is greater than the threshold can be predicted.

[0018] In an optional implementation manner, after determining the third packet loss interval according to the N video packets and the received video packet interval, the method further includes:

[0019] If the RTT is greater than a preset threshold, the union of the first packet loss interval, the second packet loss interval, and the third packet loss interval is determined as the first target retransmission interval; or,

[0020] If the RTT is less than the preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the second target retransmission interval, or the first packet loss interval is determined as the second target retransmission interval; or,

[0021] If the signal quality value is greater than a preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the third target retransmission interval;

[0022] If the signal quality value is greater than the preset threshold and an ACK message is received, the intersection of the first packet loss interval and the third packet loss interval and the union of the second packet loss interval are determined as the fourth target retransmission interval.

[0023] Through the above method, different packet loss intervals can be determined in different situations, and video packets can be retransmitted based on the determined packet loss intervals, which can significantly reduce video freezes, delays, etc.

[0024] In an optional implementation, inputting the target feature parameter set into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain a first packet loss interval includes:

[0025] Extract features from video data to obtain video features, network features, and wireless signal features;

[0026] Perform feature fusion on video features, network features and wireless signal features to obtain fusion features;

[0027] Packet loss prediction is performed based on the fusion features to obtain a first packet loss interval.

[0028] In a second aspect, the present application provides a data packet loss prediction device, the device comprising:

[0029] A processing module, configured to obtain a signal quality value under a current network environment based on the sent video data through a trained signal quality assessment model; wherein the sent video data is composed of N video packets, N is a positive integer greater than or equal to 1, and the signal quality assessment model is trained based on wireless signal characteristic parameters corresponding to historical video playback quality data;

[0030] The prediction module is used to input the target feature parameter set into the trained packet loss prediction model to predict the packet loss of the video data and obtain the first packet loss interval if the signal quality value is less than a preset threshold; wherein the target feature parameter set includes: video feature parameters of the sent video data, network feature parameters of the video sending end, and wireless signal feature parameters.

[0031] In an optional implementation, if the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data, and before obtaining the first packet loss interval, the processing module is further used to:

[0032] Determine whether an ACK message sent by the video receiving end is received;

[0033] If yes, the second packet loss interval and the received video packet interval are parsed from the ACK message.

[0034] In an optional implementation manner, after parsing the second packet loss interval and the received video packet interval from the ACK message, the processing module is further used to:

[0035] Get the round-trip time (RTT) corresponding to the current network environment;

[0036] If the RTT is greater than a preset threshold, a third packet loss interval is determined according to the N video packets and the received video packet interval.

[0037] In an optional implementation, after determining the third packet loss interval according to the N video packets and the received video packet interval, the processing module is further configured to:

[0038] If the RTT is greater than a preset threshold, the union of the first packet loss interval, the second packet loss interval, and the third packet loss interval is determined as the first target retransmission interval; or,

[0039] If the RTT is less than the preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the second target retransmission interval, or the first packet loss interval is determined as the second target retransmission interval; or,

[0040] If the signal quality value is greater than a preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the third target retransmission interval;

[0041] If the signal quality value is greater than the preset threshold and an ACK message is received, the intersection of the first packet loss interval and the third packet loss interval and the union of the second packet loss interval are determined as the fourth target retransmission interval.

[0042] In an optional implementation, when the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain the first packet loss interval, the prediction module is specifically used to:

[0043] Perform feature fusion on video features, network features and wireless signal features to obtain fusion features;

[0044] Packet loss prediction is performed based on the fusion features to obtain the first packet loss interval.

[0045] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the data packet loss prediction method described in the first aspect above.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium comprising a program code. When the program code is executed on an electronic device, the program code is used to enable the electronic device to execute the steps of the data packet loss prediction method described in the first aspect above.

[0047] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the data packet loss prediction method as described in the first aspect.

[0048] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0050] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0051] Figure 2 A schematic diagram of an implementation flow of a data packet loss prediction method provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of a data packet loss prediction device provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0055] It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. A and B are connected, which can represent two situations: A and B are directly connected and A and B are connected through C. In addition, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0056] In addition, the collection, dissemination, and use of data in the technical solution of this application comply with the requirements of relevant national laws and regulations.

[0057] The following is a brief introduction to the design concept of the embodiment of the present application:

[0058] At present, smart cameras are widely used for real-time monitoring in various scenarios. Smart cameras can be connected to the public cloud platform through Wi-Fi networks, and users can access the cloud platform on their terminal devices to view real-time video or alarm recordings in the monitoring area.

[0059] However, when smart cameras are connected to a Wi-Fi network, they are prone to packet loss at the network transmission layer due to factors such as installation location, wireless interference, and network delay, which can cause problems such as video playback freezes, frame loss, and delays.

[0060] In the related art, ACK messages are usually used for packet loss identification. However, this method uses a heuristic algorithm, that is, it uses experience or a fixed formula to identify packet loss, which has the problem of low packet loss identification accuracy. Specifically, since the Wi-Fi network may be interfered with in actual application scenarios, resulting in large network fluctuations, using a fixed formula for packet loss identification based on the ACK message will cause the network parameters fed back by the ACK message to be inaccurate due to network fluctuations, thereby affecting the accuracy of packet loss identification. In addition, when the Wi-Fi network transmits data in an open space, it is easily blocked by obstructions (such as walls, etc.) and interfered by other electromagnetic waves, which will also cause delays in the ACK message, thereby affecting the accuracy of packet loss identification.

[0061] In view of this, the present application provides a method for predicting packet loss, which includes: first, in response to a video transmitter sending video data to a video receiver, a signal quality value in the current network environment is obtained through a trained signal quality assessment model; wherein, the video data is composed of N video packets, N is a positive integer greater than or equal to 1, and the signal quality assessment model is obtained by training the wireless signal characteristic parameters corresponding to the historical video playback quality data; then, if the signal quality value is less than a preset threshold, the target characteristic parameter set is input into the trained packet loss prediction model to predict the packet loss of the video data, and a first packet loss interval is obtained; wherein, the target characteristic parameter set includes video characteristic parameters, network characteristic parameters, and wireless signal characteristic parameters. Through the above method, the packet loss prediction of video data is performed using characteristic parameters of multiple dimensions, which can improve the accuracy of the prediction model in predicting packet loss, and the data transmission strategy can be adjusted based on the prediction results to avoid problems such as video playback freeze and picture delay caused by packet loss.

[0062] The following describes a method for predicting data packet loss provided by an exemplary embodiment of the present application with reference to the accompanying drawings.

[0063] See also Figure 1 , which is a schematic diagram of possible application scenarios in the embodiment of the present application. The application scenario diagram includes a smart camera and terminal devices (including terminal device 121, terminal device 122...terminal device 12n).

[0064] The terminal device 120 includes but is not limited to mobile phones, tablet computers, laptops, desktop computers, e-readers, intelligent voice interaction devices, vehicle-mounted terminals and other devices; various software can be installed on the terminal device, such as applications and applets, etc., which are not specifically limited in this application.

[0065] For further information, see Figure 2 FIG. 1 is a schematic diagram of an implementation process of a method for predicting packet loss provided by an embodiment of the present application. The specific implementation process of the method is as follows:

[0066] S1: Based on the sent video data, the signal quality value in the current network environment is obtained through the trained signal quality assessment model.

[0067] In the embodiment of the present application, when the video transmitter sends video data to the video receiver, the complete video data is divided into multiple video packets according to a preset size for transmission. For example, if a video data segment contains 200 frames of images, the video data can be divided into video packets of 50 frames and transmitted in sequence. In addition, the transmitted video data is the video data that has not been successfully received by the video receiver.

[0068] Therefore, when the video transmitter sends video data to the video receiver, the signal quality value under the current network environment can be obtained through the trained signal quality evaluation model.

[0069] Specifically, when no data is sent, the wireless signal strength RSSI and other parameters corresponding to the wireless network may be good, but when the wireless network carries video data for transmission, the network conditions of the wireless network may change significantly. Therefore, in the embodiment of the present application, when training the signal quality assessment model, the historical video playback data when the network carries video data for transmission and the wireless signal characteristic parameters corresponding to the historical video playback data are used as the training data set to train the signal quality assessment model to be trained.

[0070] Specifically, in an embodiment of the present application, historical video playback quality data including picture delay, freeze rate, bit rate, etc. is first obtained.

[0071] Furthermore, the video playback quality data is filtered according to the requirements of the actual application scenario. Optionally, the video playback quality data within a period of time is filtered according to a preset threshold. For example, the video playback quality data of the past 7 days is obtained for filtering.

[0072] For example, the threshold of the image delay is set to 5 milliseconds, so as to filter out data with an image delay of less than 5 milliseconds, the threshold of the freeze rate is set to 5%, so as to filter out data with a freeze rate of less than 5%, and the threshold of the bit rate is set to 500 kbps, so as to filter out data with a bit rate of less than 500 kbps. Then, the filtered data with image delay, freeze rate, and bit rate below the preset threshold is used as a training data set.

[0073] Of course, other video playback quality data may also be selected, and this application does not make any specific limitations here.

[0074] In the embodiment of the present application, the training data set may be fitted by nonlinear fitting, and the fitting result may be used as a signal quality assessment model. For example, nonlinear fitting may be performed by nonlinear regression or nonlinear least squares method. Specifically, it may be a curve relationship model or an equation relationship model between the wireless signal characteristic parameter and the signal quality value.

[0075] In the embodiment of the present application, when the video transmitter sends video data to the video receiver, the wireless signal characteristic parameters may be obtained, which may include wireless signal strength RSSI, channel utilization, wireless packet loss rate, wireless negotiation rate, wireless receiving signal-to-noise ratio, etc.

[0076] Furthermore, when the video sending end sends video data to the video receiving end, the video sending end evaluates the signal quality by acquiring wireless signal characteristic parameters and video quality data, and obtains the signal quality value under the current network environment.

[0077] Through the above method, the signal quality is evaluated based on the current wireless signal parameters, so that when the current network environment is poor, the transmission strategy can be adjusted in time to ensure the video quality.

[0078] S2: If the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain a first packet loss interval.

[0079] In the embodiment of the present application, if the signal quality value of the current network environment is less than the preset threshold, it means that the current network environment is already at a poor level, and there is a possibility of packet loss when sending video data. For example, the preset threshold of the signal quality value is set to 0.8, and when the signal quality value is less than 0.8, there is a possibility of packet loss.

[0080] In an optional implementation, when the video sending end sends video data to the video receiving end, it sends video packets of a fixed size to the video receiving end in sequence. When the video receiving end receives the video packet sent by the video sending end, it sends an ACK message to the video sending end to confirm that the video packet has been correctly received and can further send subsequent video packets.

[0081] Therefore, after the video transmitter sends a packet to the video receiver, it also needs to determine whether an ACK message sent by the video receiver is received.

[0082] If the video receiving end receives the ACK message sent by the video sending end, the second packet loss interval and the received video packet interval are parsed from the ACK message.

[0083] For example, there is currently a segment of video data that is divided into 100 video packets. At this time, the video transmitter has sent video packets No. 1-20 to the video receiver. After receiving these video packets, the video receiver will send an ACK message to the video transmitter to feedback the current reception situation. If the current network environment is good, video packets No. 1-20 will be successfully received by the video receiver, and there will be no packet loss. If the current network environment is poor, some video packets in video packets No. 1-20 may be lost. For example, video packets No. 5-9 are lost, and then the video receiver uses an ACK message to feedback to the video transmitter that video packets No. 1-4 and video packets No. 10-20 are successfully received, and video packets No. 5-9 are lost. Therefore, the second packet loss interval is video packets No. 5-9, and the received video packet interval is video packets No. 1-4 and No. 10-20.

[0084] Through the above method, the packet loss interval can be accurately determined, which is convenient for subsequent processing and reduces screen freezes and the like.

[0085] Furthermore, in an optional implementation, when the video transmitter sends a video data packet to the video receiver, it is further necessary to obtain the round-trip time (RTT) corresponding to the current network environment. If the RTT is greater than a preset threshold, some video packets may be lost. For example, if the preset threshold is 1 second, and the video transmitter does not receive feedback from the video receiver after sending the video packet to the video receiver for 1 second, some video packets may be lost.

[0086] For example, the video sender has sent video packets No. 1-100 to the video receiver. After successfully receiving some video packets, the video receiver will feedback the received video packet interval to the video sender through an ACK message. For example, the current video receiver has successfully received video packets No. 1-80. If the video packets No. 81-100 have not fed back the reception status to the video sender through an ACK message after the RTT is greater than the preset threshold, the video packets No. 81-100 are determined as the third packet loss interval.

[0087] Furthermore, if the signal quality value corresponding to the current network environment is less than a preset threshold or the ACK message sent by the video receiving end is not received, the video sending end predicts the packet loss interval.

[0088] In the embodiment of the present application, when performing packet loss prediction, it is a time-series related prediction task. Therefore, a recurrent neural network (RNN) can be selected for model training to obtain the packet loss prediction model used in the present application.

[0089] Specifically, when training the packet loss prediction model, the video feature parameters corresponding to the video data, the network feature parameters corresponding to the video transmitter, and the wireless signal feature parameters can be obtained from the historical data recorded during the video transmission process. The video feature parameters may include: video packet transmission delay, video packet transmission time, video bit rate, video I frame to P frame ratio, the network feature parameters may include: round-trip delay RTT, congestion window, packet loss rate, retransmission rate, etc., the wireless signal feature parameters may include: wireless signal strength RSSI, channel utilization, wireless packet loss rate, wireless negotiation rate, wireless receiving signal-to-noise ratio, and corresponding data packet loss data. In order to ensure the accuracy of packet loss prediction, it is necessary to mark the video feature parameters, network feature parameters, and wireless signal feature parameters generated when the lost data packets and the lost data packets are sent, and then use the above data as a training data set.

[0090] Then the obtained training data set is divided into a training set, a validation set, and a test set according to a preset ratio. For example, the data set can be divided according to a ratio of 6:2:2. Further, a suitable neural network architecture is designed, the weights and biases are initialized, and the data in the training set is input into the neural network for training, and the output result is obtained by calculation through the output layer. The difference between the predicted value and the true value is calculated using a loss function, and the network parameters are updated by gradient descent or other optimization algorithms. During the training process, if the loss function value is greater than the expected value, the model parameters are adjusted, and then the loss function is recalculated. In each training process, the model that performs best on the validation set is selected. Finally, the final model is evaluated using the test set. If the performance of the test set does not meet the requirements, it is necessary to continue training with the training set, adjust the model architecture or parameters, and finally pass the validation set test. The neural network model is used as the packet loss prediction model of this application.

[0091] Furthermore, in an optional implementation, when using a trained packet loss prediction model for prediction, the video transmitter can obtain video feature parameters, wireless signal feature parameters, and network feature parameters corresponding to the transmitted data in real time, thereby forming a target feature parameter set.

[0092] The target feature parameter set is input into the trained packet loss prediction model. First, the video features, network features, and wireless signal features are normalized so that the numerical ranges of different features are unified or scaled to similar scales, thereby eliminating the scale differences between features and the dimension effects between different features. Furthermore, the video features, network features, and wireless signal features are fused to obtain fused features, and then the prediction results are obtained based on the fused features.

[0093] In an embodiment of the present application, the packet loss prediction model predicts packet loss for video data sent from the video transmitter to the video receiver, and the output result is a probability distribution value for different intervals. For example, 100 video packets have been sent, among which video packets No. 1-20 have been successfully received by the video receiver, and packet loss prediction is performed for the remaining video packets No. 21-100. For another example, the result output by the packet loss prediction model is the packet loss probability value for each interval of the remaining video packets: the packet loss probability of video packets No. 21-50 is 20%, the packet loss probability of video packets No. 51-77 is 36%, the packet loss probability of video packets No. 78-88 is 25%, and the packet loss probability of video packets No. 89-100 is 88%.

[0094] Further, a packet loss interval with a packet loss probability greater than a preset threshold is selected as the first packet loss interval. For example, if the preset packet loss probability threshold is set to 80%, then video packets No. 89-100 are the first packet loss interval.

[0095] In an optional implementation, after determining the first packet loss interval, the second packet loss interval, and the third packet loss interval based on the method provided in the embodiment of the present application, the current transmission strategy can be adjusted according to the current data packet loss situation.

[0096] Specifically, if the RTT in the current network environment is greater than a preset threshold, the target retransmission interval is determined based on the first packet loss interval, the second packet loss interval, and the third packet loss interval. For example, if the first packet loss interval is A, the second packet loss interval is B, and the third packet loss interval is C, then in this embodiment of the application, the union of interval A, interval B, and interval C is used as the target retransmission interval, that is, the target retransmission interval is .

[0097] It should be noted that, at this time, since the video transmitter feeds back the second packet loss interval to the video receiver through the ACK message, and when the RTT is greater than the preset threshold, other video packets may be lost, that is, there is a third packet loss interval, which also includes the first packet loss interval predicted according to the target feature parameter set. In order to better transmit the video, ensure the video quality, and reduce the picture delay, freeze, etc., all possible packet loss intervals are retransmitted.

[0098] Furthermore, if the RTT is less than a preset threshold and the video receiving end does not receive an ACK message fed back by the video sending end, a second target retransmission interval is determined based on the first packet loss interval and the third packet loss interval. In the embodiment of the present application, the second target retransmission interval can be the first packet loss interval, or it can be the intersection of the first packet loss interval and the third packet loss interval, that is, the second target retransmission interval is Or C.

[0099] Furthermore, if the signal quality corresponding to the current network environment is greater than a preset threshold and no ACK message is received, a third target retransmission interval is determined based on the first packet loss zone set and the third packet loss interval. In an embodiment of the present application, the third target retransmission interval may be the intersection of the first packet loss interval and the third packet loss interval, that is, the third target retransmission interval is .

[0100] Furthermore, if the signal quality value corresponding to the current network environment is greater than the preset threshold and an ACK message is received, the intersection of the first packet loss interval and the third packet loss interval can be taken as the union with the second packet loss interval to determine the fourth target retransmission interval, that is, the fourth target retransmission interval is .

[0101] After different target retransmission intervals, the video transmitter can resend corresponding data to the video receiver according to the target retransmission interval.

[0102] Through the above method, different transmission strategies can be adjusted in time according to different network conditions to ensure that there will be no video loss due to packet loss, thereby ensuring the continuity and integrity of the video.

[0103] Further, based on the same technical concept, the embodiment of the present application provides a data packet loss prediction device, which is used to implement the above method flow of the embodiment of the present application. Figure 3 As shown, the device includes: a processing module 301 and a prediction module 302, wherein:

[0104] The processing module 301 is used to obtain the signal quality value in the current network environment based on the sent video data through the trained signal quality evaluation model; wherein the sent video data is composed of N video packets, N is a positive integer greater than or equal to 1, and the signal quality evaluation model is trained based on the wireless signal characteristic parameters corresponding to the historical video playback quality data;

[0105] The prediction module 302 is used to input the target feature parameters into the trained packet loss prediction model to predict the packet loss of the video data and obtain the first packet loss interval if the signal quality value is less than a preset threshold; wherein the target feature parameter set includes: video feature parameters of the sent video data, network feature parameters of the video sending end, and wireless signal feature parameters.

[0106] In an optional implementation, if the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data, and before obtaining the first packet loss interval, the processing module 301 is further used to:

[0107] Determine whether an ACK message sent by the video receiving end is received;

[0108] If yes, the second packet loss interval and the received video packet interval are parsed from the ACK message.

[0109] In an optional implementation manner, after parsing the second packet loss interval and the received video packet interval from the ACK message, the processing module 301 is further configured to:

[0110] Get the round-trip time (RTT) corresponding to the current network environment;

[0111] If the RTT is greater than a preset threshold, a third packet loss interval is determined according to the N video packets and the received video packet interval.

[0112] In an optional implementation, after determining the third packet loss interval according to the N video packets and the received video packet interval, the processing module 301 is further configured to:

[0113] If the RTT is greater than a preset threshold, the union of the first packet loss interval, the second packet loss interval, and the third packet loss interval is determined as the first target retransmission interval; or,

[0114] If the RTT is less than the preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the second target retransmission interval, or the first packet loss interval is determined as the second target retransmission interval; or,

[0115] If the signal quality value is greater than a preset threshold and no ACK message is received, the intersection of the first packet loss interval and the third packet loss interval is determined as the third target retransmission interval;

[0116] If the signal quality value is greater than the preset threshold and an ACK message is received, the intersection of the first packet loss interval and the third packet loss interval and the union of the second packet loss interval are determined as the fourth target retransmission interval.

[0117] In an optional implementation, when the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain the first packet loss interval, the prediction module 302 is specifically used to:

[0118] Perform feature fusion on video features, network features and wireless signal features to obtain fusion features;

[0119] Packet loss prediction is performed based on the fusion features to obtain the first packet loss interval.

[0120] Based on the same technical concept, the embodiment of the present application also provides an electronic device, which can implement the data packet loss prediction method process provided in the above embodiment of the present application. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device. Figure 4 As shown, the electronic device may include:

[0121] At least one processor 401, and a memory 402 connected to the at least one processor 401. The specific connection medium between the processor 401 and the memory 402 is not limited in the embodiment of the present application. Figure 4 In the example, the processor 401 and the memory 402 are connected via the bus 400. The bus 400 is Figure 4 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 400 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 401 can also be called a controller, and there is no limitation on the name.

[0122] In the embodiment of the present application, the memory 402 stores instructions that can be executed by at least one processor 401. The at least one processor 401 can execute a packet loss prediction method discussed above by executing the instructions stored in the memory 402. The processor 401 can implement Figure 4 The functions of each module in the device shown.

[0123] Among them, the processor 401 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 402 and calling the data stored in the memory 402, the various functions of the device and process data, the device can be monitored as a whole.

[0124] In one possible design, the processor 401 may include one or more processing units, and the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0125] Processor 401 may be a general-purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of a packet loss prediction method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in a processor.

[0126] The memory 402 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, and the like. The memory 402 is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 402 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0127] By programming the processor 401, the code corresponding to the frequency offset estimation method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The steps of a method for predicting packet loss in the embodiment shown are as follows: How to design and program the processor 401 is a technique known to those skilled in the art and will not be described in detail here.

[0128] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a data packet loss prediction method discussed above.

[0129] In some possible implementations, the present application also provides various aspects of a data packet loss prediction method, which can also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a data packet loss prediction method according to various exemplary embodiments of the present application described above in this specification.

[0130] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0131] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a server, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] Program code for performing the operations of the present application may be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for predicting packet loss, characterized in that: The method comprises: Based on the sent video data, a signal quality value in the current network environment is obtained through a trained signal quality assessment model; wherein the sent video data consists of N video packets, N is a positive integer greater than or equal to 1, and the signal quality assessment model is trained based on wireless signal characteristic parameters corresponding to historical video playback quality data; If the signal quality value is less than a preset threshold, the target feature parameter set is input into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain a first packet loss interval; wherein the target feature parameter set includes: video feature parameters of the sent video data, network feature parameters of the video sending end, and wireless signal feature parameters.

2. The method according to claim 1, characterized in that Before the step of inputting the target feature parameter set into a trained packet loss prediction model to perform packet loss prediction on the video data if the signal quality value is less than a preset threshold and obtaining a first packet loss interval, the method further includes: Determine whether an ACK message sent by the video receiving end is received; If yes, the second packet loss interval and the received video packet interval are parsed from the ACK message.

3. The method according to claim 2, characterized in that After parsing the second packet loss interval and the received video packet interval from the ACK message, the method further includes: Get the round-trip time (RTT) corresponding to the current network environment; If the RTT is greater than a preset threshold, a third packet loss interval is determined according to the N video packets and the received video packet interval.

4. The method according to claim 3, characterized in that After determining a third packet loss interval according to the N video packets and the received video packet interval, the method further includes: If the RTT is greater than a preset threshold, determining the union of the first packet loss interval, the second packet loss interval, and the third packet loss interval as a first target retransmission interval; or, If the RTT is less than a preset threshold and the ACK message is not received, determining the second target retransmission interval according to the intersection of the first packet loss interval and the third packet loss interval, or determining the first packet loss interval as the second target retransmission interval; or, If the signal quality value is greater than a preset threshold and the ACK message is not received, determining the intersection of the first packet loss interval and the third packet loss interval as a third target retransmission interval; If the signal quality value is greater than a preset threshold and the ACK message is received, the intersection of the first packet loss interval and the third packet loss interval and the union of the second packet loss interval are determined as a fourth target retransmission interval.

5. The method according to claim 1, characterized in that The step of inputting the target feature parameter set into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain a first packet loss interval includes: Performing feature fusion on the video feature, the network feature, and the wireless signal feature to obtain a fusion feature; Packet loss prediction is performed based on the fusion features to obtain a first packet loss interval.

6. A packet loss prediction device, characterized in that: The device comprises: A processing module, configured to obtain a signal quality value in a current network environment based on the sent video data through a trained signal quality assessment model; wherein the sent video data is composed of N video packets, N is a positive integer greater than or equal to 1, and the signal quality assessment model is trained based on wireless signal characteristic parameters corresponding to historical video playback quality data; A prediction module is used to input a target feature parameter set into a trained packet loss prediction model to perform packet loss prediction on the video data to obtain a first packet loss interval if the signal quality value is less than a preset threshold; wherein the target feature parameter set includes: video feature parameters of the sent video data, network feature parameters of the video sending end, and wireless signal feature parameters.

7. The device according to claim 6, characterized in that Before the step of inputting the target feature parameter set into a trained packet loss prediction model to perform packet loss prediction on the video data if the signal quality value is less than a preset threshold and obtaining a first packet loss interval, the processing module is further used to: Determine whether an ACK message sent by the video receiving end is received; If yes, the second packet loss interval and the received video packet interval are parsed from the ACK message.

8. The device according to claim 7, characterized in that After parsing the second packet loss interval and the received video packet interval from the ACK message, the processing module is further used to: Get the round-trip time (RTT) corresponding to the current network environment; If the RTT is greater than a preset threshold, a third packet loss interval is determined according to the N video packets and the received video packet interval.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Packet loss prediction method, device, equipment, medium and product

    CN118802588A