H.265 coding video packet loss image recovery method based on machine learning

Abstract

The invention relates to the technical field of image processing, and discloses an H.265 coded video packet loss image recovery method based on machine learning, which comprises the following steps: acquiring packet loss characteristics of a receiving end at the current moment, inputting a packet loss prediction model pre-constructed based on characteristic dimension expansion, and outputting a packet loss characteristic prediction value at the next moment; determining whether the receiving end sends key frame request information to the sending end or not based on a comparison strategy of different thresholds and packet loss feature predicted values under different network data, so as to obtain the key frame at the current moment for continuing decoding; and if recovery cannot be realized through the key frame request, introducing a packet loss recovery mechanism of intra-frame and inter-frame prediction for different lost image frame types, and performing image recovery of the lost area by using different packet loss image recovery strategies. According to the method, a complete technical chain from prediction, decision making to repairing is constructed, and the modern video coding standard and an intelligent learning mechanism are deeply combined, so that active sensing and accurate repairing of the video packet loss are realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for recovering lost images from H.265 encoded video based on machine learning. Background Technology

[0002] In the complex network environment of video transmission, packet loss is a common occurrence. Although H.265, as a new generation of high-efficiency video coding standard, greatly improves compression efficiency, its video quality stability is severely challenged in the face of packet loss.

[0003] Traditional video packet loss recovery methods have many shortcomings. For example, relying on accumulated packet loss or reaching the maximum number of retransmissions before requesting keyframes to recover lost data fails to adequately consider the current network conditions. Requesting keyframes during network congestion increases the network load, leading to further network degradation, inefficient packet loss recovery, and issues such as video stuttering and screen tearing, severely impacting the user viewing experience. With the continuous expansion of video applications such as online live streaming, remote medical video diagnosis, and video conferencing, the demand for stable and high-quality H.265 video transmission in complex network environments is becoming increasingly urgent, necessitating a new packet loss recovery method to address these issues. Summary of the Invention

[0004] This invention aims to provide a method for recovering lost images from H.265 encoded video using machine learning. By constructing a complete technology chain from prediction and decision-making to repair, and deeply integrating modern video coding standards and intelligent learning mechanisms, it achieves proactive perception and accurate repair of video packet loss.

[0005] The basic solution provided by this invention is: a method for recovering lost images from H.265 encoded video based on machine learning.

[0006] S1, obtain the packet loss characteristics of the receiving end at the current moment, input the pre-built packet loss prediction model based on the feature dimension expansion, and output the packet loss characteristic prediction value of the next moment;

[0007] S2, based on the comparison strategy of different thresholds and packet loss feature prediction values ​​under different network data, determines whether the receiver should send key frame request information to the sender in order to obtain the key frame at the current moment for continued decoding;

[0008] S3. If recovery cannot be achieved through keyframe requests, intra-frame and inter-frame prediction packet loss repair mechanisms are introduced for different types of lost image frames, and image recovery of lost areas is performed using different packet loss image recovery strategies.

[0009] The working principle and advantages of this invention are as follows:

[0010] This invention constructs a complete technology chain from prediction and decision-making to repair, deeply integrating modern video coding standards and intelligent learning mechanisms to achieve proactive perception and accurate repair of video packet loss.

[0011] This invention constructs a packet loss prediction model based on feature dimension expansion to proactively predict the changing trend of network state. By integrating multiple network features (such as latency, jitter, packet loss burstiness, etc.) and / or content features, the model constructs a more comprehensive state description, improves the accuracy of prediction, and provides a valuable time window and decision basis for subsequent intelligent decision-making.

[0012] This invention employs a prediction-driven adaptive decision-making mechanism, combining predictive information with dynamic decision thresholds to achieve an intelligent and preventative resource scheduling strategy. By comparing different thresholds under different network data with predicted packet loss characteristics, a context-aware decision-making process is achieved. During network congestion, unnecessary keyframe requests are effectively suppressed to prevent further network degradation; conversely, when packet loss severely impacts video continuity, keyframes are requested promptly to interrupt error propagation, and intra-frame and inter-frame predictions are used for repair. This mechanism, which dynamically adjusts current decision rules based on predicted future states, significantly enhances adaptability to complex heterogeneous networks, ensures basic smoothness of video transmission, greatly optimizes network bandwidth utilization, and improves the overall stability and efficiency of the system.

[0013] This invention abandons traditional single or simple filling strategies in terms of restoration. For different image frame types in H.265 encoding, it innovatively introduces a differentiated intra-frame and inter-frame prediction collaborative restoration mechanism. It creatively reverses the application of core video encoding compression tools (intra-frame / inter-frame prediction) to the field of packet loss recovery, achieving layered, content-aware, and refined restoration. For I-frames or cases where a good reference frame is unavailable, intra-frame prediction is used to mine spatial correlations for restoration. For P / B frames, inter-frame prediction is prioritized, searching for matching information from the correct reference frame in the temporal dimension. This fully utilizes the spatial texture similarity and temporal motion continuity of the video sequence for content-aware pixel prediction and reconstruction. This restoration method, which deeply integrates coding characteristics, can extremely effectively restore details closer to the original image, significantly mitigating common problems in traditional methods such as image blurring, blocky distortion, and mosaic effects, thereby fundamentally improving the visual fidelity of packet loss recovery.

[0014] Ultimately, this invention, through a closed-loop system of prediction-guided decision-making and decision-driven repair, maintains high-quality and stable video footage even under demanding network conditions where data retransmission is not feasible. This not only effectively curbs precipitous drops in video quality caused by packet loss but also provides users with a consistently smooth, clear, and natural viewing experience, demonstrating its significant value in improving the quality of video services. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a method for recovering lost images from H.265 encoded video based on machine learning, provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram illustrating the process of model training, model prediction, and keyframe request provided in an embodiment of the present invention.

[0017] Figure 3 This is a schematic diagram illustrating the process of image restoration based on intra-frame and inter-frame prediction packet loss repair mechanism provided in an embodiment of the present invention. Detailed Implementation

[0018] The following detailed explanation illustrates the specific implementation methods:

[0019] Example 1

[0020] The basics are as follows: Figure 1 As shown: A method for recovering lost images from H.265 encoded video based on machine learning, comprising:

[0021] S1, obtain the packet loss characteristics of the receiving end at the current moment, input the pre-built packet loss prediction model based on the feature dimension expansion, and output the packet loss characteristic prediction value of the next moment;

[0022] S2, based on the comparison strategy of different thresholds and packet loss feature prediction values ​​under different network data, determines whether the receiver should send key frame request information to the sender in order to obtain the key frame at the current moment for continued decoding;

[0023] S3. If recovery cannot be achieved through keyframe requests, intra-frame and inter-frame prediction packet loss repair mechanisms are introduced for different types of lost image frames, and image recovery of lost areas is performed using different packet loss image recovery strategies.

[0024] Specifically:

[0025] In S1, the receiving end monitors network conditions in real time to detect packet loss. When receiving H.265 encoded video data packets, the receiving end determines whether packet loss exists by comparing the continuity of RTP (Real-Time Transport Protocol) packet sequence numbers. If a sequence number interruption or jump is detected, packet loss is determined to have occurred, and relevant basic information about the packet loss, such as the time of packet loss and the number of packets lost, is recorded.

[0026] Packet loss features are calculated based on relevant basic information about packet loss, including packet loss rate and packet recovery rate. To reduce the frequency of packet loss feature calculation and provide more stable detection results, RTP packets received within a preset time period (e.g., 100ms) are grouped into a single RTP packet group for calculation.

[0027] Packet loss rate calculation: For each RTP packet group, the receiver records the number N packets lost at time k. k Simultaneously, by using the difference in sequence numbers and the sequence number of the first RTP packet, the total number M of RTP packets that should theoretically be received can be calculated. k Then the packet loss rate at time k is:

[0028]

[0029] Packet loss recovery rate calculation: Set a target waiting time, and count the number R of packets lost at time k that are recovered within that target waiting time. k Calculate the packet loss recovery rate at time k:

[0030]

[0031] like Figure 2 As shown, in S1, a packet loss prediction model is pre-built:

[0032] 1. Feature dimensions are expanded. In addition to the basic packet loss rate and packet loss recovery rate, auxiliary network state features are introduced, including: network bandwidth fluctuation value (the standard deviation of bandwidth over the past 10 packet groups); mean round-trip time (RTT) (the moving average of the last 5 RTT measurements); packet jitter (the variance of the arrival time difference between adjacent packet groups); frame type distribution (the proportion of I / P / B frames in the past GOP); feature normalization uses min-max standardization.

[0033]

[0034] Where f is the original eigenvalue, f min and f max These are the eigenvalues ​​for offline statistical analysis.

[0035] 2. Predictive model architecture, employing an improved machine learning LSTM (Long Short-Term Memory) model, includes:

[0036] Input layer: Receives a 7-dimensional feature vector (2 core features + 5 auxiliary features);

[0037] Hidden layers: 2 layers of LSTM units (64 neurons in the first layer and 32 neurons in the second layer), with a Dropout layer added after each layer (dropout rate = 0.2) to suppress overfitting;

[0038] Output layer: 2 neurons (outputting the predicted packet loss rate and the predicted packet recovery rate respectively), with the activation function using the Sigmoid function to restrict the output to the range [0,1].

[0039] Collect packet loss information over historical time periods, including packet loss rate, packet loss recovery rate, and five extended dimensions, and input this data into the initial prediction model.

[0040] 3. Prediction model training:

[0041] 1) Dataset construction: Collect packet loss sequences for 10 typical network scenarios (such as 4G / 5G / WiFi, congestion / smoothness), generate 1000 samples with a duration of 10 minutes for each scenario, and divide them into training set and validation set in a 7:3 ratio.

[0042] 2) Loss function, using the weighted mean square error algorithm:

[0043] Loss = α.MSE(L pred ,L ture )+β.MSE(R pred ,R ture )

[0044] Where α and β are the mean square error weights of packet loss rate and packet recovery rate, respectively, MSE(L pred ,L ture MSE(R) pred ,R ture ) are the mean square errors of packet loss rate and packet loss recovery rate, respectively. pred L is the predicted packet loss rate. ture R represents the actual packet loss rate. pred R is the predicted packet loss recovery rate. ture The actual value of the packet loss recovery rate can be used; α = 0.6, β = 0.4 can be adopted to strengthen the prediction weight of the packet loss rate.

[0045] 3) Optimizer: Adam optimizer (initial learning rate 0.001, decaying by 10% every 50 rounds);

[0046] 4) Early stopping mechanism: Training is stopped when the validation set loss does not decrease for 10 consecutive rounds, and the optimal model is saved.

[0047] Through training, the model learns the patterns of packet loss information changing over time, yielding theoretical results for historical time periods. Based on these theoretical results and actual packet loss information, biases and information are determined, leading to the acquisition of coefficients for the initial prediction model. These coefficients are then used to update the parameters of the initial prediction model, resulting in a preliminarily optimized prediction model.

[0048] 4. Predict packet loss information in the next moment and correct the prediction model:

[0049] 1) Sliding window sampling: Taking the current time k as the endpoint, the feature sequence of the past 20 packets (about 2 seconds) is taken as input;

[0050] 2) Model Inference: Input the feature sequence into the pre-trained LSTM model, and output the predicted values ​​of packet loss parameters for the next 3 packet groups (300ms), i.e., the packet loss rate L. k+1 and packet loss recovery rate R rate,k+1 .

[0051] 3) Prediction Correction: Kalman filtering is used to smooth the continuous prediction results.

[0052]

[0053] in, Let z be the state vector representing packet loss rate / packet loss recovery rate. k K represents the model's predicted value. k Here, H is the Kalman gain, and H is the observation matrix used to map the packet loss rate component in the state vector to the observable packet loss rate value. Its value is [1, 0], indicating that only the current packet loss rate observation value z is used. k Correct the system state.

[0054] Outlier handling: When the predicted value deviates from the historical mean by more than 3 times the standard deviation, confidence assessment is triggered: calculate the mutation index of the feature sequence (L2 norm of the first derivative of the feature). If the index is >0.8, it is determined to be a network mutation, and the backup prediction mode (linear extrapolation based on the most recent 3 packet groups) is activated.

[0055] Model adaptive update: Prediction error is statistically analyzed hourly, and incremental training is initiated when the mean absolute error (MAE) > 0.05.

[0056]

[0057] The model parameters were fine-tuned using 500 newly collected packet data sets (the learning rate was reduced to 1 / 10 of the initial value), retaining historical knowledge while adapting to network changes.

[0058] The packet loss information at time k (packet loss rate L) k and packet loss recovery rate R rate,kThe data is input into the optimized prediction model to obtain the predicted packet loss information at time k+1, including the predicted packet loss rate L. k+1 and packet loss recovery rate R rate,k+1 .

[0059] In S2, the dynamic target threshold is:

[0060] The target threshold includes the packet loss rate threshold T at time k+1. L and packet loss recovery rate threshold T R The threshold setting can be adjusted according to different network environments and video quality requirements. For example, when the network is relatively stable, a relatively low threshold can be set to control packet loss recovery more precisely; when the network fluctuates greatly, the threshold can be appropriately increased to ensure video smoothness.

[0061] like Figure 2 As shown, in S2, the keyframe request decision is:

[0062] If the packet loss information at time k+1 reaches the target threshold, a keyframe request message is sent to the sender. Specifically, when the predicted packet loss rate L... k+1 ≥T L Or the predicted packet loss recovery rate R rate,k+1 ≤T R If the network condition is deemed poor and packet loss is severely impacting video quality, a keyframe request is sent to the sender to retrieve the keyframe for the current moment. When the receiver requests a keyframe, it discards all buffered video data packets from the previous Group of Pictures (GOP) and uses the latest retrieved keyframe as the reference frame to continue receiving and processing subsequent video streams.

[0063] In S3, frame type determination: For H.265 bitstreams that cannot be recovered through the keyframe retransmission mechanism (i.e., recovered through keyframe request), the receiver performs frame type determination on the decoded H.265 video ES (basic stream) data. The frame types in H.265 include IDR frames (instantaneous decoding refresh frames), I frames (internal coding frames), B frames (bidirectional prediction frames), and P frames (forward prediction frames).

[0064] like Figure 3 As shown in S3, when packet loss is detected by frame type determination and the current frame is an I-frame or an IDR frame (not the first frame data and without buffered IDR frames), the lost image is recovered through an intra-frame predicted packet loss recovery mechanism. Specifically:

[0065] Intra-frame prediction mode selection: For I-frames or IDR frames with packet loss, the optimal intra-frame prediction mode is selected based on H.265 intra-frame prediction modes, such as planar mode, DC mode, and angle mode, combined with information from adjacent received pixel blocks. For edge regions, the angle mode better fits the pixel change trend; for flat regions, the DC mode is used. The optimal mode is determined by calculating the error between the predicted pixel and the adjacent received pixel under different modes (using the mean square error (MSE) algorithm).

[0066] Pixel value prediction and padding: Based on the selected intra-frame prediction mode, the pixel values ​​of the lost packet area are predicted and filled. Taking the angle mode as an example, the value of the lost pixel is predicted based on the direction and gradient information of neighboring pixels. For a 4*4 pixel block, the predicted value of each pixel in the pixel block is obtained through weighted calculation using the linear relationship of the surrounding received pixels, thereby completing the image restoration of the lost packet area.

[0067] like Figure 3 As shown in S3, when the frame type determines that the current frame of packet loss is a P frame or a B frame, the lost image is recovered through the inter-frame prediction packet loss recovery mechanism.

[0068] Reference frame selection: In H.265 encoding, P-frames typically use the previous frame as the reference frame, while B-frames use both the previous and next frames as preceding and following reference frames. For a P-frame with packet loss, if the previous frame was received completely, it is used as the reference frame; if the previous frame also had packet loss, the most suitable frame that has been received completely is selected as the reference frame based on historical packet loss information and prediction models (such as the aforementioned machine learning-based prediction model).

[0069] Motion estimation and compensation: Using reference frame information, a motion estimation algorithm (Block Matching Algorithm, BMA) is used to find the pixel block in the reference frame that best matches the packet loss area, thus determining the motion vector. Taking block matching-based motion estimation as an example, the lost P-frame or B-frame is divided into multiple 16*16 macroblocks. Each macroblock is traversed within the search window of the reference frame, and the matching error at each position is calculated (using absolute error and SAD algorithms). The position with the smallest matching error is found, and its offset relative to the current macroblock is the motion vector. Based on the motion vector, pixel values ​​are obtained from the corresponding pixel block in the reference frame, and motion compensation is performed on the packet loss area to complete image restoration.

[0070] like Figure 3 As shown in S3, when the frame type determines that the current frame of packet loss is a B frame, in addition to the above-mentioned unidirectional motion estimation and compensation, bidirectional prediction optimization is also performed using the backward reference frame.

[0071] By obtaining motion vectors and predicted pixel values ​​from forward and backward reference frames respectively, and fusing the forward and backward prediction results using methods such as weighted averaging, a more accurate reconstructed image is obtained. For example, based on the reliability of the forward and backward predictions (measured by the size of the matching error), different weights are set to perform weighted calculations on the pixel values ​​of the forward and backward predictions, resulting in the final pixel values ​​used to reconstruct the packet loss area of ​​the B-frame.

[0072] This embodiment provides a method for recovering lost video packets based on machine learning H.265 encoded video. By constructing a complete technology chain from prediction and decision-making to repair, and deeply integrating modern video coding standards and intelligent learning mechanisms, it achieves proactive perception and accurate repair of video packet loss.

[0073] Example 2

[0074] Unlike Example 1, for a lost P-frame, if the previous frame also experienced packet loss, the most suitable fully received frame is selected as the reference frame based on historical packet loss information and a prediction model (such as the aforementioned machine learning-based prediction model). Specifically:

[0075] 1. Construct a "candidate reference frame set"

[0076] The system scans the decoding buffer, collecting all video frames that have been fully received and successfully decoded. This set typically includes: the most recent received frame (one or more complete P-frames or B-frames / keyframes closest to it before the lost P-frame) and the keyframe (the most recent complete I-frame, which is the most reliable alternative because it does not depend on any other frame).

[0077] 2. Feature extraction and similarity assessment

[0078] The goal is to select the "best" frame from multiple candidate frames. The core criteria for decision-making are temporal relevance and content similarity. The following analysis methods can be used:

[0079] Temporal proximity analysis: Calculate the timestamp difference (Δt) between each candidate frame and the lost frame P. The smaller the difference, the smaller the scene change usually is, and the higher the correlation.

[0080] Content motion trend prediction: This is a key step in introducing machine learning models. The system inputs historical packet loss information (such as packet loss patterns in the past N frames and network jitter) and video content features (such as the average motion vector amplitude calculated in the received frames and scene complexity) into a pre-trained prediction model.

[0081] Model Task: The model outputs a prediction of the "scene consistency probability" or "motion continuity score" between the lost P-frame and each candidate frame. By learning from historical data, it can determine which reference frame at a given time point will yield better reconstruction results under similar network and content features.

[0082] 3. Comprehensive decision-making and reference frame selection

[0083] Combining the two main factors from step 3, a final decision score is formed:

[0084] Decision score = α × (1 / Δt) + β × similarity score output by the prediction model

[0085] Here, α and β are weighting coefficients used to balance temporal proximity and the confidence level of the model prediction.

[0086] The candidate frame with the highest decision score is selected and designated as the "best alternative reference frame".

[0087] This embodiment provides a method for recovering lost images from H.265 encoded video based on machine learning. It offers an optimized repair strategy for lost P-frames if the previous frame also has packet loss, thereby improving the applicability of this method.

[0088] Example 3

[0089] Unlike Examples 1 and 2, this method dynamically adjusts the packet loss rate threshold and packet loss recovery rate threshold based on the current packet loss volatility. The core strategy is: the more stable the network, the more "fine" and "aggressive" the threshold settings; the greater the network volatility, the more "lenient" and "conservative" the threshold settings. The specific process includes:

[0090] First, set a basic threshold for packet loss rate and a basic threshold for packet loss recovery rate based on different network data and video quality.

[0091] Secondly, during real-time application, the standard deviation of packet loss rate Std_Loss_k within the preset time period preceding time k is calculated. Based on the basic threshold of packet loss rate TLbase (e.g., set to 5%) and the basic threshold of packet loss recovery rate TRbase (e.g., set to 80%), the thresholds of packet loss rate TL{k+1} and packet loss recovery rate TR{k+1} at time k+1 are calculated using the standard deviation of packet loss rate.

[0092] If Std_Loss_k < 1% (the network is very stable), then TL{k+1} = TLbase - 1% = 4%.

[0093] Reason: The network is stable, allowing for the setting of a relatively strict threshold. Once the packet loss rate slightly rises to 4%, the protection mechanism is immediately triggered to prevent a sudden collapse in quality, achieving "fine-grained control".

[0094] If 1% <= Std_Loss_k < 2% (the network has slight fluctuations), then TL{k+1} = TLbase = 5%.

[0095] Reason: Under normal circumstances, the default threshold is used.

[0096] If Std_Loss_k >= 2% (network fluctuations are large), then TL{k+1} = TLbase + 2% = 7%.

[0097] Reason: Networks are inherently unstable, and occasional spikes in packet loss are normal. Setting the threshold too low will cause frequent policy switching, leading to video stuttering and drastic resolution changes. Appropriately increasing the threshold can "filter out" some temporary fluctuations, ensuring smooth video playback.

[0098] The adjustment logic for the packet loss recovery rate threshold is the opposite of the adjustment logic for the packet loss rate threshold:

[0099] If Std_Loss_k < 1% (the network is very stable), then TR{k+1} = TRbase + 5% = 85%.

[0100] Reason: Network stability places higher demands on recovery capabilities. We only confidently upgrade video quality when the recovery rate is very high (85%). This avoids blindly upgrading when the recovery mechanism is only "barely usable".

[0101] If 1% <= Std_Loss_k < 2% (the network has slight fluctuations), then TR{k+1} = TRbase = 80%.

[0102] If Std_Loss_k >= 2% (network fluctuations are large), then TR{k+1} = TRbase - 10% = 70%.

[0103] Reason: Under poor network conditions, the requirements for recovery capabilities are reduced. As long as the recovery rate can reach 70%, we believe there is an opportunity to try to maintain or slightly improve the quality, rather than simply downgrading. This gives the system a chance to "survive" in adversity.

[0104] This embodiment provides a method for recovering lost images from H.265 encoded video packets based on machine learning. The determination of the threshold can adapt to changes in the network environment, making an optimal balance between video quality and smoothness, and driving the video transmission strategy to adjust more intelligently.

[0105] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A method for recovering a lost packet image of an H.265 encoded video based on machine learning, characterized in that, The method comprises the following steps: S1, obtaining the packet loss characteristics of the current time of the receiving end, inputting the packet loss prediction model based on the characteristic dimension expansion, and outputting the packet loss characteristic prediction value of the next time; S2, based on the comparison strategy of different threshold values and packet loss characteristic prediction values under different network data, determining whether the receiving end sends key frame request information to the sending end to obtain the current time key frame for continuous decoding; S3, if the key frame request cannot be recovered, for different lost image frame types, the packet loss repair mechanism of intra-frame and inter-frame prediction is introduced, and different packet loss image recovery strategies are used for image recovery of the lost area.

2. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S1, the receiving end detects the network status in real time to detect packet loss, and after determining that packet loss occurs, an RTP packet group is combined by receiving RTP packets in a preset time period, and the packet loss characteristics are calculated.

3. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S1, the packet loss characteristics include packet loss rate and packet loss recovery rate; the expanded characteristics are network state auxiliary characteristics, including network bandwidth fluctuation value, round-trip delay average, data packet jitter, frame type distribution and normalized characteristics.

4. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S2, when the comparison condition of at least one packet loss characteristic prediction value and its corresponding threshold value is met, the receiving end sends key frame request information to the sending end.

5. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S2, when the receiving end requests the key frame, all buffered video data packets in the previous group of pictures (GOP) are discarded, and the latest obtained key frame is used as the reference frame to continue receiving and processing the subsequent video stream.

6. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S3, for the H265 code stream that cannot be recovered by the key frame request, the receiving end performs frame type judgment on the decoded H.265 video basic stream data.

7. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S3, when the frame type judgment determines that the current frame with packet loss is an intra-coded frame or an instant decoding refresh frame, the lost image is recovered through the intra-frame prediction packet loss recovery mechanism; wherein, the instant decoding refresh frame is located at the middle position of the video sequence and neither the sending end nor the receiving end has effective cache of the instant decoding refresh frame; When the frame type judgment determines that the current frame with packet loss is a forward prediction frame or a bidirectional prediction frame, the lost image is recovered through the inter-frame prediction packet loss recovery mechanism.

8. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S3, when the frame type judgment determines that the current frame with packet loss is an intra-coded frame or an instant decoding refresh frame, for the lost area of the intra-coded frame or the instant decoding refresh frame, the optimal intra-frame prediction mode is selected based on the information of the adjacent received pixel blocks, and the pixel values of the lost area are predicted and filled to complete the image recovery of the lost area.

9. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S3, when the frame type judgment determines that the current frame with packet loss is a forward prediction frame, the previous frame is used as the reference frame, and the image recovery of the lost area is completed through one-way motion estimation and compensation.

10. The machine learning based H.265 encoded video packet loss image recovery method of claim 1, wherein, In S3, when the frame type judgment determines that the current frame with packet loss is a bidirectional prediction frame, the previous frame and the next frame are used as the front and rear reference frames respectively, and the image recovery of the lost area is completed through bidirectional motion estimation and compensation.

Images