A method for automatically repairing damaged video images using AI

Abstract

The present invention provides a method for automatically repairing damaged video images using AI, which relates to the technical field of video image repair. After identifying damaged frames in a video file to be repaired, the method generates corresponding image repair tasks, and inserts the image repair tasks into a repair task queue in chronological order based on the starting timing information of the damaged frames in each image repair task. When repairing the video, the image repair tasks are taken out from the head of the repair task queue and processed to obtain corresponding repaired frames. Finally, the repaired frames are inserted into corresponding positions in the video file to be repaired to replace the corresponding damaged frames. In this way, the method can generate corresponding repaired frames based on the context information of the damaged frames with the help of AI technology when the frame content is completely lost. The method has the characteristics of high repair efficiency and the ability to effectively guarantee video quality and user viewing experience.

Description

Technical Field

[0001] The present invention relates to the technical field of video image restoration, and in particular to a method for automatically restoring damaged video images using AI. Background Art

[0002] Video broadly refers to various technologies that capture, record, process, store, transmit, and reproduce a series of static images using electrical signals. When continuous image changes exceed 24 frames per second, the human eye cannot distinguish single static frames due to the persistence of vision principle; instead, the visual effect appears smooth and continuous, and this continuous image is called video. Video technology is widely used in modern society, encompassing nearly every industry and field, such as entertainment and media, education, security and surveillance, and entertainment and creativity. Due to various factors, such as network, hardware, software, and storage, video frames may be damaged or lost, affecting video quality and the viewing experience. Current video image restoration technologies primarily fill in missing regions within each video frame. Missing regions primarily include damaged or accidentally occluded areas within the image. In other words, these technologies primarily repair portions of each frame. However, repairing missing regions in a frame relies on the surrounding content. When the entire frame is missing or damaged, restoration based on the frame's own content is difficult. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for automatically repairing damaged video frames using AI. Based on the contextual information of the frames before and after the damaged frame, the damaged frame is automatically repaired using AI technology to ensure video quality and viewing experience.

[0004] To achieve the above-mentioned object of the invention, the present invention provides a method for automatically repairing damaged video images using AI, the method comprising:

[0005] S101: Obtain a video file to be repaired, and identify damaged frames in the video file to be repaired;

[0006] S102: Generate a corresponding image restoration task for each damaged frame, wherein the image restoration task includes start timing information and stop timing information of the damaged frame;

[0007] S103, initializing a repair task queue, and inserting the image repair tasks into the repair task queue in chronological order according to the start timing information of the damaged frames in each image repair task;

[0008] S104, taking out the image repair task from the head of the repair task queue, processing the image repair task, and obtaining a repaired frame image;

[0009] S105: inserting the repaired frame into the video file to be repaired, replacing the corresponding damaged frame, according to the start timing information and the stop timing information of the damaged frame in the image repair task;

[0010] S106. Execute steps S104 to S105 in a loop until the repair task queue is empty.

[0011] Furthermore, damaged frames in the video file to be repaired are identified, specifically by manually marking the damaged frames in the video file; or automatically identifying the damaged frames in the video file through a frame recognition module.

[0012] Furthermore, the damaged frames in the video file are automatically identified by the frame recognition module, specifically including:

[0013] S201, dividing the video file into a plurality of frames, each frame corresponding to a time point;

[0014] S202, recording the frame picture at a time point before the frame picture as the previous frame picture, recording the frame picture at a time point after the frame picture as the subsequent frame picture, and recognizing the semantics of the previous frame picture, the frame picture, and the subsequent frame picture respectively;

[0015] S203, respectively calculating the difference between the semantics of the frame and the semantics of the previous frame and the semantics of the subsequent frame to obtain a previous semantic difference value and a subsequent semantic difference value;

[0016] S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

[0017] Furthermore, the damaged frames in the video file are automatically identified by the frame identification module, which specifically includes:

[0018] S301, identifying the video file type, where the video file type includes a monitoring type and a creation type;

[0019] S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the following semantic difference value is greater than a preset difference threshold, determine whether the frame picture semantics is a transition semantics;

[0020] S303: If the frame semantics are transition semantics, the frame is not determined to be a damaged frame.

[0021] Furthermore, the damaged frames in the video file are automatically identified by the frame identification module, which specifically includes:

[0022] S401, after dividing the video file into a plurality of frames, pre-processing the obtained plurality of frames;

[0023] S402, using a motion estimation algorithm to calculate a motion vector between two adjacent frames in a plurality of frames;

[0024] S403, calculating the mean of the motion vectors of all frames, and using the mean as a statistical threshold;

[0025] S404: Determine whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold;

[0026] S405 , judging whether the frame image may be a damaged frame image based on the judgment results of S204 and S404 .

[0027] Furthermore, the image restoration task is processed to obtain the restored frame image, which specifically includes:

[0028] S501: Calculate the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information;

[0029] S502, acquiring a previous normal frame image according to the damaged frame start timing information, and acquiring a subsequent normal frame image according to the damaged frame stop timing information;

[0030] S503, performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame;

[0031] S504: inputting the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame;

[0032] S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

[0033] Furthermore, the semantics of the preceding and subsequent normal frames are input into the semantic generation network for processing to obtain the semantic prediction results of the damaged frame, which specifically includes:

[0034] S601, encoding the semantics of the preceding normal frame and the semantics of the following normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame;

[0035] S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction result of the damaged frame.

[0036] Furthermore, the semantic prediction results of the damaged frame and the duration of the frame damage are input into the image generation model to generate multiple frames corresponding to the semantic prediction results, which specifically includes the following operations:

[0037] S701, performing semantic enhancement on the semantic prediction result;

[0038] S702, upsampling the semantic prediction result after semantic enhancement to obtain a mask of the same size as the final generated frame image;

[0039] S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of the frame damage through the perceptron of the image generation model.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention provides a method for automatically repairing damaged video images using AI. First, after identifying damaged frames in a video file to be repaired, corresponding image repair tasks are generated, and according to the starting timing information of the damaged frames in each image repair task, the image repair tasks are inserted into a repair task queue in chronological order. When repairing the video, the image repair tasks are taken out from the head of the repair task queue and processed to obtain corresponding repaired frames. Finally, the repaired frames are inserted into corresponding positions in the video file to be repaired to replace the corresponding damaged frames. In this way, the method can generate corresponding repaired frames based on the context information of the damaged frames with the help of AI technology when the frame content is completely lost. The method has the characteristics of high repair efficiency and the ability to effectively guarantee video quality and user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 This is a schematic diagram of the overall process of a method for automatically repairing damaged video images using AI provided by an embodiment of the present invention.

[0044] Figure 2This is a schematic diagram of the working process of the frame recognition module provided by an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram of the transition frame image recognition process provided by an embodiment of the present invention.

[0046] Figure 4 This is a schematic diagram of the damaged frame image recognition process provided by an embodiment of the present invention.

[0047] Figure 5 This is a schematic diagram of the image restoration task processing flow provided by an embodiment of the present invention.

[0048] Figure 6 This is a schematic diagram of the semantic generation network workflow provided by an embodiment of the present invention.

[0049] Figure 7 1 is a schematic diagram of the workflow of the image generation model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The principles and features of the present invention are described below with reference to the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0051] Reference Figure 1 This embodiment provides a method for automatically repairing damaged video images using AI, the method comprising the steps of:

[0052] S101: Acquire a video file to be repaired, and identify damaged frames in the video file to be repaired.

[0053] In this embodiment, damaged frames in the video file to be repaired are identified, specifically by manually marking the damaged frames in the video file; or automatically identifying the damaged frames in the video file using a frame recognition module. In other words, the method provided in this embodiment can be achieved by either manually identifying and marking damaged frames in the video file, or automatically identifying damaged frames using a frame recognition module. In some implementations, manual and automatic recognition can be combined. For example, after automatically identifying damaged frames in a video file using a frame recognition module, the recognition results can be manually confirmed and re-identified to improve recognition accuracy.

[0054] S102 : Generate a corresponding image repair task for each damaged frame, where the image repair task includes start timing information and stop timing information of the damaged frame.

[0055] It is understandable that the damaged frame in the video may be a single frame or multiple consecutive frames. The damaged frame start timing information is used to describe the timing information of the frame in the video where the image damage begins, that is, at which time point in the video the frame appears. Correspondingly, the damaged frame termination timing information is used to describe at which time point in the video the damaged frame turns into a normal image. When there is only one damaged frame or the damaged frame is discontinuous, the start timing information and termination timing information corresponding to the same damaged frame are the same; when the damaged frame is a continuous multiple frames, the start timing information and termination timing information respectively describe the start time node and termination time node of the continuous multiple frames of damaged images in the video.

[0056] S103: Initialize a repair task queue, and insert the image repair tasks into the repair task queue in chronological order according to the starting timing information of the damaged frames in each image repair task.

[0057] S104: Take out the image repair task from the head of the repair task queue, process the image repair task, and obtain a repaired frame image.

[0058] S105 , inserting the repaired frame into the video file to be repaired to replace the corresponding damaged frame according to the start timing information and the stop timing information of the damaged frame in the image repair task.

[0059] S106. Execute steps S104 to S105 in a loop until the repair task queue is empty.

[0060] As a possible implementation method, refer to Figure 2 , the frame recognition module is used to automatically identify damaged frames in the video file, specifically including the following operations:

[0061] S201: Divide the video file into multiple frames, each frame corresponding to a time point.

[0062] S202: Record the frame picture at a time point before the frame picture as the preceding frame picture, and record the frame picture at a time point after the frame picture as the succeeding frame picture, and recognize the semantics of the preceding frame picture, the frame picture, and the succeeding frame picture respectively.

[0063] S203 , respectively calculating differences between the semantics of the frame and the semantics of the preceding frame and the semantics of the succeeding frame to obtain a preceding semantic difference value and a succeeding semantic difference value.

[0064] S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

[0065] Under normal circumstances, the semantic changes between adjacent frames in a continuous sequence are generally minor. However, if a frame is damaged or lost, the semantics of that frame may suddenly change significantly compared to its preceding and following frames. This implementation compares the semantics of each frame in the video with those of its preceding and following frames. If the semantic difference between a frame and its preceding or following frames exceeds a preset difference threshold, the frame is considered to be potentially damaged.

[0066] Exemplarily, the difference between the semantics of a frame and the semantics of a preceding frame and a succeeding frame can be calculated by directly calculating the semantic similarity of the two frames using a preset semantic similarity model to determine the difference. Alternatively, the difference in semantics between the two frames can be calculated using a word embedding method, specifically by using a pre-trained word embedding model to convert semantics into vectors and calculating the distance between the two semantic vectors. Commonly used distance metrics include Euclidean distance and cosine similarity. It is understandable that the calculation of semantic differences can also be achieved through other methods, which are not specifically limited in this embodiment.

[0067] As a further optional embodiment, refer to Figure 3 , automatically identify damaged frames in video files through the frame recognition module, specifically including:

[0068] S301. Identify the video file type, which includes a monitoring type and a creation type.

[0069] S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the subsequent semantic difference value is greater than a preset difference threshold, determine whether the frame picture semantics is a transition semantics.

[0070] S303: If the frame semantics are transition semantics, the frame is not determined to be a damaged frame.

[0071] For different types of videos, the content and form of their presentation are different, and the semantic transformation forms between consecutive frames are also different. For creative video files, in order to enhance the richness of the content to attract viewers, the creator may add some transition pictures. The appearance of transition pictures may cause the semantics of consecutive frames to suddenly change significantly, but it does not mean that the video frames are lost or damaged. For surveillance video files, they are usually used to monitor fixed locations, and the content semantics of their consecutive frames often do not suddenly change significantly. Therefore, for this type of video file, if the content semantics of a certain frame suddenly change significantly compared to the content semantics of the frames before and after it, it means that the frame may be damaged or lost. This embodiment identifies the corresponding video file type while judging a certain frame as possibly being a damaged frame. If it is a creative video file, it further judges whether the frame semantics of the possibly damaged frame are transition semantics. If they are transition semantics, they are not judged as damaged frames to avoid misjudgment and improve recognition accuracy.

[0072] As a further optional embodiment, refer to Figure 4 , automatically identify damaged frames in video files through the frame recognition module, specifically including:

[0073] S401: After dividing a video file into a plurality of frames, pre-processing is performed on the obtained plurality of frames.

[0074] Exemplarily, the obtained multiple frames are pre-processed, including grayscale conversion, noise reduction, etc., to improve the accuracy of subsequent processing.

[0075] S402: Calculate a motion vector between two adjacent frames in a plurality of frames using a motion estimation algorithm.

[0076] S403: Calculate the mean value of the motion vectors of all frames, and use the mean value as a statistical threshold.

[0077] S404: Determine whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold.

[0078] S405 , judging whether the frame image may be a damaged frame image based on the judgment results of S204 and S404 .

[0079] Motion estimation refers to the detection and estimation of the movement of objects between consecutive video frames. By calculating the pixel displacement between adjacent frames, a motion vector map can be generated. The motion vector represents the direction and distance of movement of a pixel block from one frame to another. These vectors can be used to describe the movement of objects between frames. By analyzing the statistical characteristics of the motion vectors, abnormal frames can be detected. For example, if the motion vector of a frame is significantly different from that of other frames, or the motion vector of a certain area changes suddenly, this may indicate that the frame is damaged or lost. The motion estimation algorithm can use block matching, optical flow method, etc., which is not specifically limited in this embodiment.

[0080] In this embodiment, the frame recognition module not only identifies whether a frame may be a damaged frame by judging the semantic difference between the frame and its preceding and following adjacent frames, but also calculates the motion vectors between adjacent frames in the video file through a motion estimation algorithm, and calculates their mean as a statistical threshold, and further judges whether the frame is a damaged frame by judging whether the motion vector between adjacent frames is greater than the statistical threshold. That is, the method can further improve the recognition accuracy by comprehensively judging whether the frame is a damaged frame by combining semantics and motion vectors.

[0081] As a possible implementation method, refer to Figure 5 , process the image restoration task and obtain the restored frame image, which specifically includes the following operations:

[0082] S501: Calculate the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information.

[0083] S502: Acquire a previous normal frame image according to the damaged frame start timing information, and acquire a subsequent normal frame image according to the damaged frame stop timing information.

[0084] S503 , performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame.

[0085] S504: Input the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame.

[0086] S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

[0087] This implementation first uses a semantic generation network to predict the intermediate semantics of the damaged frame based on the contextual semantics of the preceding and subsequent frames. The semantic predictions for the damaged frame output by the semantic generation network, along with the duration of the frame damage, are then input into an image generation model to generate multiple frames corresponding to the semantic predictions. In other words, if the damaged frames are multiple consecutive frames, the same number of frames will be generated to completely replace the damaged frames in the video file.

[0088] As a further optional embodiment, refer to Figure 6 , the semantics of the preceding and following normal frames are input into the semantic generation network for processing to obtain the semantic prediction results of the damaged frame, specifically including:

[0089] S601: Encode the semantics of the preceding normal frame and the semantics of the subsequent normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame.

[0090] In this embodiment, the semantic generation network includes an encoder and a decoder. The encoder can use a bidirectional long short-term memory network, which can capture the semantic relationships of long texts. Through the bidirectional long short-term memory network encoding process, a contextual hidden vector matrix of the semantics of the damaged frame can be obtained, including the hidden layer vectors of the forward long short-term memory network and the hidden layer vectors of the backward long short-term memory network.

[0091] S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction result of the damaged frame.

[0092] For example, the decoder of a semantic generation network can employ a Transformer network. By incorporating a self-attention mechanism, the decoder can more easily capture long-range semantic relationships when decoding and predicting the semantics of corrupted frames. The Transformer network uses a multi-head attention mechanism, with each attention mechanism performing parallel computations and the outputs of each attention mechanism concatenated. This multi-head attention mechanism allows the network to focus on multiple features at different locations.

[0093] As a further optional embodiment, refer to Figure 7 , the semantic prediction results of the damaged frame and the duration of the frame damage are input into the image generation model to generate multiple frames corresponding to the semantic prediction results, including the following operations:

[0094] S701: Perform semantic enhancement on the semantic prediction result.

[0095] In this implementation, after the decoder outputs the semantic prediction result, it is encoded by the encoder to obtain a feature representation of the semantic prediction result. This feature representation is then randomly sampled to achieve semantic enhancement. Semantic enhancement can improve the generator's robustness to perturbations, thereby improving model performance.

[0096] S702 : Up-sampling the semantic prediction result after semantic enhancement to obtain a mask of the same size as the final generated frame image.

[0097] In this implementation, to better align the semantic prediction results with the generated frame space, the semantically enhanced semantic prediction results are combined with a mask. The mask is a rough outline generated based on the image's spatial features, used to guide subsequent semantic understanding and image refinement. For example, the mask can be generated using a CNN network.

[0098] S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of the frame damage through the perceptron of the image generation model.

[0099] The semantic prediction result represents the content of the frames between the previous frame and the subsequent frame. If the damaged frame is a continuous multi-frame picture, the semantic prediction result represents the content of the continuous multi-frame pictures. Therefore, it is necessary to constrain the perceptron by the duration of frame damage so that it generates multiple frames corresponding to the duration of frame damage to reflect the semantic prediction result, rather than using a single frame or a small number of frames to express the same semantics, so that the transition of the repaired video file is more natural and accurate.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatically repairing damaged video images using AI, characterized in that: The method comprises: S101: Obtain a video file to be repaired, and identify damaged frames in the video file to be repaired; S102: Generate a corresponding image restoration task for each damaged frame, wherein the image restoration task includes start timing information and stop timing information of the damaged frame; S103, initializing a repair task queue, and inserting the image repair tasks into the repair task queue in chronological order according to the start timing information of the damaged frames in each image repair task; S104, taking out the image repair task from the head of the repair task queue, processing the image repair task, and obtaining a repaired frame image; S105: inserting the repaired frame into the video file to be repaired, replacing the corresponding damaged frame, according to the start timing information and the stop timing information of the damaged frame in the image repair task; S106, looping through steps S104 to S105 until the repair task queue is empty; Identifying damaged frames in the video file to be repaired, specifically: automatically identifying damaged frames in the video file through a frame recognition module; The frame recognition module automatically identifies damaged frames in video files, including: S201, dividing the video file into a plurality of frames, each frame corresponding to a time point; S202, recording the frame picture at a time point before the frame picture as the previous frame picture, recording the frame picture at a time point after the frame picture as the subsequent frame picture, and recognizing the semantics of the previous frame picture, the frame picture, and the subsequent frame picture respectively; S203, respectively calculating the difference between the semantics of the frame and the semantics of the previous frame and the semantics of the subsequent frame to obtain a previous semantic difference value and a subsequent semantic difference value; S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

2. The method of automatically repairing damaged video images using AI according to claim 1, characterized in that: The frame recognition module automatically identifies damaged frames in video files, including: S301, identifying the video file type, where the video file type includes a monitoring type and a creation type; S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the following semantic difference value is greater than a preset difference threshold, determine whether the frame picture semantics is a transition semantics; S303: If the frame semantics are transition semantics, the frame is not determined to be a damaged frame.

3. The method of automatically repairing damaged video images using AI according to claim 1, characterized in that: The frame recognition module automatically identifies damaged frames in video files, including: S401, after dividing the video file into a plurality of frames, pre-processing the obtained plurality of frames; S402, using a motion estimation algorithm to calculate a motion vector between two adjacent frames in a plurality of frames; S403, calculating the mean of the motion vectors of all frames, and using the mean as a statistical threshold; S404: Determine whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold; S405 , judging whether the frame image may be a damaged frame image based on the judgment results of S204 and S404 .

4. The method of automatically repairing damaged video images using AI according to claim 1, characterized in that: Process the image restoration task to obtain the restored frame image, specifically including: S501: Calculate the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information; S502, acquiring a previous normal frame image according to the damaged frame start timing information, and acquiring a subsequent normal frame image according to the damaged frame stop timing information; S503, performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame; S504: inputting the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame; S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

5. The method of automatically repairing damaged video images using AI according to claim 4, characterized in that: The semantics of the preceding and subsequent normal frames are input into the semantic generation network for processing to obtain the semantic prediction results of the damaged frame, specifically including: S601, encoding the semantics of the preceding normal frame and the semantics of the following normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame; S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction result of the damaged frame.

6. The method of automatically repairing damaged video images using AI according to claim 4, characterized in that: The semantic prediction results of the damaged frame and the duration of the frame damage are input into the image generation model to generate multiple frames corresponding to the semantic prediction results. The specific operations include the following: S701, performing semantic enhancement on the semantic prediction result; S702, upsampling the semantic prediction result after semantic enhancement to obtain a mask of the same size as the final generated frame image; S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of the frame damage through the perceptron of the image generation model.

Images