Method for automatically repairing damaged video picture by AI (Artificial Intelligence)

Abstract

The invention provides a method for automatically repairing a damaged video picture through AI, and relates to the technical field of video picture repairing, and the method comprises the steps: generating a corresponding picture repairing task after recognizing a damaged frame picture in a to-be-repaired video file, and repairing the damaged frame picture according to the initial time sequence information of the damaged frame picture in each picture repairing task. The method comprises the following steps: inserting a picture repairing task into a repairing task queue according to a time sequence, when a video is repaired, starting to take out the picture repairing task from the head of the repairing task queue and processing the picture repairing task to obtain a corresponding repairing frame picture, and finally inserting the repairing frame picture into a corresponding position of a to-be-repaired video file. According to the method, the corresponding damaged frame picture is replaced, so that the corresponding repaired frame picture can be generated according to the context information of the damaged frame picture by means of the AI technology under the condition that the content of the frame picture is completely lost, and the method has the characteristics that the repairing efficiency is high, and the video quality and the watching experience of a user can be effectively guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of video picture restoration, and in particular to a method for automatically restoring damaged video pictures using AI. Background Art

[0002] Video refers to various technologies that capture, record, process, store, transmit and reproduce a series of static images in the form of electrical signals. When the continuous image changes more than 24 frames per second, according to the principle of visual persistence, the human eye cannot distinguish a single static picture; it looks like a smooth and continuous visual effect, and such a continuous picture is called video. Video technology has been widely used in modern society, covering almost all industries and fields, such as entertainment and media, education, security and monitoring, entertainment and creativity, etc. Due to network, hardware, software, storage and other reasons, video frames may be damaged or lost, affecting the quality of the video and viewing experience. The current video image repair technology mainly fills in the missing area information of each frame in the video. The missing area mainly includes the damaged area or the area accidentally blocked in the image. That is to say, the technology mainly repairs part of each frame of the video, and the repair of the missing area of ​​the frame depends on the surrounding content of the missing area. When the entire frame content is missing or damaged, it is difficult to repair it based on the content of the frame itself. Summary of the invention

[0003] The purpose of the present invention is to provide a method for automatically repairing damaged video frames using AI, which automatically repairs the damaged video frames using AI technology based on contextual information of frames before and after the damaged frames to ensure video quality and viewing experience.

[0004] To achieve the above-mentioned object of the invention, the present invention provides a method for automatically repairing damaged video images using AI, the method comprising:

[0005] S101, obtaining a video file to be repaired, and identifying damaged frames in the video file to be repaired;

[0006] S102, generating a corresponding image repair task for each damaged frame, wherein the image repair task includes start timing information and stop timing information of the damaged frame;

[0007] S103, initializing a repair task queue, and inserting the image repair tasks into the repair task queue in chronological order according to the starting timing information of the damaged frame images in each image repair task;

[0008] S104, taking out the image repair task from the head of the repair task queue, processing the image repair task, and obtaining a repaired frame image;

[0009] S105, inserting the repaired frame into the video file to be repaired to replace the corresponding damaged frame according to the start timing information and the stop timing information of the damaged frame in the image repair task;

[0010] S106. Execute steps S104 to S105 in a loop until the repair task queue is empty.

[0011] Furthermore, the damaged frames in the video file to be repaired are identified, specifically: manually marking the damaged frames in the video file; or automatically identifying the damaged frames in the video file through a frame recognition module.

[0012] Furthermore, the damaged frames in the video file are automatically identified by the frame identification module, specifically including:

[0013] S201, dividing the video file into a plurality of frames, each frame corresponding to a time point;

[0014] S202, recording a frame picture at a time point before the frame picture as a preceding frame picture, recording a frame picture at a time point after the frame picture as a succeeding frame picture, and respectively identifying the semantics of the preceding frame picture, the frame picture, and the succeeding frame picture;

[0015] S203, respectively calculating the difference between the semantics of the frame picture and the semantics of the previous frame picture and the semantics of the subsequent frame picture to obtain a previous semantic difference value and a subsequent semantic difference value;

[0016] S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

[0017] Furthermore, the damaged frames in the video file are automatically identified by a frame identification module, which specifically includes:

[0018] S301, identifying the video file type, where the video file type includes a monitoring type and a creation type;

[0019] S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than a preset difference threshold, determining whether the frame picture semantics is a transition semantics;

[0020] S303: If the frame picture semantics is transition semantics, the frame picture is not determined to be a damaged frame picture.

[0021] Furthermore, the damaged frames in the video file are automatically identified by a frame identification module, which specifically includes:

[0022] S401, after dividing the video file into a plurality of frames, preprocessing the obtained plurality of frames;

[0023] S402, using a motion estimation algorithm to calculate a motion vector between two adjacent frames of a plurality of frames;

[0024] S403, calculating the mean value of the motion vectors of all frames, and taking the mean value as a statistical threshold;

[0025] S404, determining whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold;

[0026] S405 , judging whether the frame picture may be a damaged frame picture in combination with the judgment results of S204 and S404 .

[0027] Further, the image restoration task is processed to obtain the restored frame image, which specifically includes:

[0028] S501, calculating the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information;

[0029] S502, acquiring a previous normal frame picture according to the damaged frame start timing information, and acquiring a subsequent normal frame picture according to the damaged frame stop timing information;

[0030] S503, performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame;

[0031] S504, inputting the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame;

[0032] S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

[0033] Furthermore, the semantics of the preceding normal frame and the semantics of the succeeding normal frame are input into the semantic generation network for processing to obtain the semantic prediction result of the damaged frame, which specifically includes:

[0034] S601, encoding the semantics of the preceding normal frame and the semantics of the succeeding normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame;

[0035] S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction results of the damaged frame.

[0036] Furthermore, the semantic prediction result of the damaged frame and the duration of the frame damage are input into the image generation model to generate multiple frames corresponding to the semantic prediction result, which specifically includes the following operations:

[0037] S701, performing semantic enhancement on the semantic prediction result;

[0038] S702, up-sampling the semantic prediction result after semantic enhancement to obtain a mask with the same size as the final generated frame image;

[0039] S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of frame damage through the perceptron of the image generation model.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention provides a method for automatically repairing damaged video images with AI. First, after identifying damaged frame images in a video file to be repaired, a corresponding image repair task is generated, and according to the starting timing information of the damaged frame images in each image repair task, the image repair task is inserted into a repair task queue in chronological order. When repairing the video, the image repair task is taken out from the head of the repair task queue and processed to obtain a corresponding repaired frame image. Finally, the repaired frame image is inserted into a corresponding position of the video file to be repaired to replace its corresponding damaged frame image. In this way, the method can generate a corresponding repaired frame image according to the context information of the damaged frame image with the help of AI technology when the frame image content is completely lost. The method has the characteristics of high repair efficiency and the ability to effectively guarantee video quality and user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a schematic diagram of the overall process of a method for automatically repairing damaged video images using AI provided by an embodiment of the present invention.

[0044] Figure 2It is a schematic diagram of the working process of the frame recognition module provided by an embodiment of the present invention.

[0045] Figure 3 It is a schematic diagram of the transition frame picture recognition process provided by an embodiment of the present invention.

[0046] Figure 4 It is a schematic diagram of a damaged frame image recognition process provided by an embodiment of the present invention.

[0047] Figure 5 It is a schematic diagram of the picture repair task processing flow provided by an embodiment of the present invention.

[0048] Figure 6 It is a schematic diagram of the workflow of the semantic generation network provided by an embodiment of the present invention.

[0049] Figure 7 It is a schematic diagram of the workflow of the image generation model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0051] Reference Figure 1 This embodiment provides a method for automatically repairing damaged video images with AI, the method comprising the steps of:

[0052] S101: Acquire a video file to be repaired, and identify damaged frames in the video file to be repaired.

[0053] In this embodiment, the damaged frames in the video file to be repaired are identified, specifically: the damaged frames in the video file are manually marked; or the damaged frames in the video file are automatically identified through a frame recognition module. That is to say, the method provided in this embodiment can be either manually identifying and marking damaged frames in the video file, or automatically identifying damaged frames through a frame recognition module. In some implementations, it can also be implemented in a combination of manual identification and automatic identification. For example, after the damaged frames in the video file are automatically identified through a frame recognition module, the identification results are manually confirmed and re-identified, thereby improving the identification accuracy.

[0054] S102: Generate a corresponding image repair task for each damaged frame, where the image repair task includes start timing information and stop timing information of the damaged frame.

[0055] It can be understood that the damaged frame in the video may be one frame or multiple consecutive frames. The damaged frame start timing information is used to describe the timing information of the frame in the video where the picture damage begins to appear, that is, at which time point in the video the frame appears. Correspondingly, the termination timing information of the damaged frame is used to describe at which time point in the video the damaged frame turns into a normal picture. When there is only one damaged frame or the damaged frame is discontinuous, the start timing information and the termination timing information corresponding to the same damaged frame are the same; when the damaged frame is a continuous multiple-frame, the start timing information and the termination timing information respectively describe the start time node and the termination time node of the continuous multiple-frame damaged pictures in the video.

[0056] S103, initializing a repair task queue, and inserting the image repair tasks into the repair task queue in chronological order according to the starting timing information of the damaged frame images in each image repair task.

[0057] S104, taking out the image repair task from the head of the repair task queue, processing the image repair task, and obtaining a repaired frame image.

[0058] S105 . Insert the repaired frame into the video file to be repaired to replace the corresponding damaged frame according to the start timing information and the stop timing information of the damaged frame in the image repair task.

[0059] S106. Execute steps S104 to S105 in a loop until the repair task queue is empty.

[0060] As a possible implementation method, refer to Figure 2 , the damaged frames in the video file are automatically identified through the frame identification module, which specifically includes the following operations:

[0061] S201, dividing a video file into a plurality of frames, each frame corresponding to a time point.

[0062] S202: Record the frame picture at a time point before the frame picture as the preceding frame picture, and record the frame picture at a time point after the frame picture as the succeeding frame picture, and recognize the semantics of the preceding frame picture, the frame picture, and the succeeding frame picture respectively.

[0063] S203 , respectively calculating differences between the semantics of the frame and the semantics of the preceding frame and the semantics of the succeeding frame to obtain a preceding semantics difference value and a succeeding semantics difference value.

[0064] S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

[0065] Under normal circumstances, in a continuous frame, the semantic changes of the content of two adjacent frames are generally small. If a frame is damaged or lost, the semantics of the frame may suddenly change greatly compared with the adjacent frames before and after it. This implementation compares the semantics of each frame in the video with the semantics of its previous and subsequent frames. When the semantic difference between the frame and the semantics of the previous or subsequent frames is greater than a preset difference threshold, the frame is considered to be a damaged frame.

[0066] Exemplarily, the difference between the semantics of a frame and the semantics of a preceding frame and a succeeding frame can be calculated by directly calculating the semantic similarity of the two frames using a preset semantic similarity model to determine the difference, or by calculating the difference between the two semantics using a word embedding method, specifically using a pre-trained word embedding model to convert the semantics into a vector, and calculating the distance between the two semantic vectors. Commonly used distance metrics include Euclidean distance, cosine similarity, etc. It can be understood that the calculation of semantic differences can also be achieved in other ways, which are not specifically limited in this embodiment.

[0067] As a further optional embodiment, refer to Figure 3 , the damaged frames in the video file are automatically identified through the frame identification module, which specifically includes:

[0068] S301. Identify the video file type, where the video file type includes a monitoring type and a creation type.

[0069] S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than a preset difference threshold, it is determined whether the frame picture semantics is a transition semantics.

[0070] S303: If the frame picture semantics is transition semantics, the frame picture is not determined to be a damaged frame picture.

[0071] For different types of videos, the content and form of the presentation are different, and the semantic transformation forms between continuous frames are also different. For video files of the creation type, in order to enhance the richness of the content to attract the audience, the creator may add some transition pictures. The appearance of transition pictures may cause the semantics of the continuous frames to suddenly change greatly, but it does not mean that the video frames are lost or damaged. For video files of the monitoring type, it is usually to monitor a fixed location, and the content semantics of its continuous frames often do not suddenly change greatly. Therefore, for this type of video file, if the content semantics of a certain frame suddenly changes greatly compared to the content semantics of the frames before and after it, it means that the frame may be damaged or lost. This implementation method identifies the corresponding video file type while judging a certain frame as a possible damaged frame. If it is a video file of the creation type, it further judges whether the frame semantics of the frame that may be damaged is the transition semantics. If it is the transition semantics, it is not judged as a damaged frame to avoid misjudgment and improve the recognition accuracy.

[0072] As a further optional embodiment, refer to Figure 4 , the damaged frames in the video file are automatically identified through the frame identification module, which specifically includes:

[0073] S401: After dividing a video file into a plurality of frames, pre-processing is performed on the obtained plurality of frames.

[0074] Exemplarily, the obtained multiple frames are pre-processed, including grayscale conversion, noise reduction, etc., to improve the accuracy of subsequent processing.

[0075] S402: Calculate a motion vector between two adjacent frames in a plurality of frames using a motion estimation algorithm.

[0076] S403: Calculate the mean value of the motion vectors of all frames, and use the mean value as a statistical threshold.

[0077] S404: Determine whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold.

[0078] S405 , judging whether the frame picture may be a damaged frame picture in combination with the judgment results of S204 and S404 .

[0079] Motion estimation refers to detecting and estimating the motion of an object between consecutive video frames. By calculating the pixel displacement between adjacent frames, a motion vector map can be generated. The motion vector represents the direction and distance of movement of a pixel block from one frame to another. These vectors can be used to describe the motion of an object between frames. By analyzing the statistical characteristics of the motion vectors, abnormal frames can be detected. For example, if the motion vector of a frame is significantly different from other frames, or the motion vector of a certain area changes suddenly, this may indicate that the frame is damaged or lost. The motion estimation algorithm may use block matching, optical flow, etc., which is not specifically limited in this embodiment.

[0080] In this embodiment, the frame recognition module not only identifies whether a frame may be a damaged frame by judging the semantic difference between the frame and its preceding and succeeding adjacent frames, but also calculates the motion vectors between adjacent frames in the video file through a motion estimation algorithm, and calculates the mean thereof as a statistical threshold, and further determines whether the frame is a damaged frame by judging whether the motion vector between adjacent frames is greater than the statistical threshold. That is, the method can further improve the recognition accuracy by comprehensively judging whether a frame is a damaged frame by combining semantics and motion vectors.

[0081] As a possible implementation method, refer to Figure 5 , process the image repair task and obtain the repaired frame image, which specifically includes the following operations:

[0082] S501: Calculate the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information.

[0083] S502: Acquire a previous normal frame image according to the damaged frame start timing information, and acquire a subsequent normal frame image according to the damaged frame stop timing information.

[0084] S503, performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame.

[0085] S504: input the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame.

[0086] S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

[0087] This implementation method first predicts the intermediate semantics of the damaged frame through a semantic generation network based on the contextual semantics of the preceding and succeeding frames of the damaged frame, and then inputs the semantic prediction results of the damaged frame output by the semantic generation network and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction results. In other words, when the damaged frame is a continuous multiple frame, the same number of multiple frames will also be generated to completely replace the damaged frame in the video file.

[0088] As a further optional embodiment, refer to Figure 6 , input the semantics of the previous normal frame and the subsequent normal frame into the semantic generation network for processing, and obtain the semantic prediction results of the damaged frame, which specifically include:

[0089] S601, encoding the semantics of the preceding normal frame and the semantics of the succeeding normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame.

[0090] In this implementation, the semantic generation network includes an encoder and a decoder. The encoder may use a bidirectional long short-term memory network, which can capture the semantic relationship of long texts. Through the bidirectional long short-term memory network encoding process, a context hidden vector matrix of the semantics of the damaged frame can be obtained, including the hidden layer vectors of the forward long short-term memory network and the hidden layer vectors of the backward long short-term memory network.

[0091] S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction results of the damaged frame.

[0092] For example, the decoder of the semantic generation network can use a Transformer network. By introducing a self-attention mechanism, the decoder can more easily capture the long-distance semantic relationship when decoding and predicting the semantics of the damaged frame. The Transformer network uses a multi-head attention mechanism, each attention mechanism is calculated in parallel, and finally the output of each attention mechanism is spliced. The multi-head attention mechanism allows the network to focus on multiple features at different positions.

[0093] As a further optional embodiment, refer to Figure 7 , input the semantic prediction results of the damaged frame and the duration of the frame damage into the image generation model, and generate multiple frames corresponding to the semantic prediction results, which specifically includes the following operations:

[0094] S701: Perform semantic enhancement on the semantic prediction result.

[0095] In this implementation, after the decoder outputs the semantic prediction result, the encoder encodes it to obtain a feature representation of the semantic prediction result, and then randomly samples the feature representation of the semantic prediction result to achieve semantic enhancement. Semantic enhancement can improve the robustness of the generator to disturbances, thereby improving model performance.

[0096] S702: up-sample the semantic prediction result after semantic enhancement to obtain a mask of the same size as the final generated frame image.

[0097] In this implementation, in order to better fit the semantic prediction result with the space of the generated frame, the semantic prediction result after semantic enhancement is combined with the mask. The mask is a roughly generated outline based on the image space features, which is used to know the subsequent semantic understanding and image fine generation. Exemplarily, the mask can be generated by a CNN network.

[0098] S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of frame damage through the perceptron of the image generation model.

[0099] The semantic prediction result represents the content of the frames between the previous frame and the subsequent frame. If the damaged frame is a continuous multi-frame, the semantic prediction result represents the content of the continuous multi-frame. Therefore, it is necessary to constrain the perceptron by the duration of frame damage so that it generates multiple frames corresponding to the duration of frame damage to reflect the semantic prediction result, rather than expressing the same semantics through a single frame or a small number of frames, so that the transition of the repaired video file is more natural and accurate.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically repairing damaged video images using AI, characterized in that: The method comprises: S101, obtaining a video file to be repaired, and identifying damaged frames in the video file to be repaired; S102, generating a corresponding image repair task for each damaged frame, wherein the image repair task includes start timing information and stop timing information of the damaged frame; S103, initializing a repair task queue, and inserting the image repair tasks into the repair task queue in chronological order according to the starting timing information of the damaged frame images in each image repair task; S104, taking out the image repair task from the head of the repair task queue, processing the image repair task, and obtaining a repaired frame image; S105, inserting the repaired frame into the video file to be repaired to replace the corresponding damaged frame according to the start timing information and the stop timing information of the damaged frame in the image repair task; S106. Execute steps S104 to S105 in a loop until the repair task queue is empty.

2. According to claim 1, the method for automatically repairing damaged video images using AI is characterized in that: The damaged frames in the video file to be repaired are identified, specifically: the damaged frames in the video file are manually marked; or the damaged frames in the video file are automatically identified by a frame recognition module.

3. The method of automatically repairing damaged video images using AI according to claim 2, characterized in that: The frame recognition module is used to automatically identify damaged frames in video files, including: S201, dividing the video file into a plurality of frames, each frame corresponding to a time point; S202, recording a frame picture at a time point before the frame picture as a preceding frame picture, recording a frame picture at a time point after the frame picture as a succeeding frame picture, and respectively identifying the semantics of the preceding frame picture, the frame picture, and the succeeding frame picture; S203, respectively calculating the difference between the semantics of the frame picture and the semantics of the previous frame picture and the semantics of the subsequent frame picture to obtain a previous semantic difference value and a subsequent semantic difference value; S204: Determine whether the preceding semantic difference value and the succeeding semantic difference value are greater than a preset difference threshold. If at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than the preset difference threshold, determine that the frame may be a damaged frame.

4. The method of automatically repairing damaged video images using AI according to claim 3, characterized in that: The damaged frames in the video file are automatically identified through the frame identification module, which specifically includes: S301, identifying the video file type, where the video file type includes a monitoring type and a creation type; S302: If the video file type is a creation type, when at least one of the preceding semantic difference value and the succeeding semantic difference value is greater than a preset difference threshold, determining whether the frame picture semantics is a transition semantics; S303: If the frame picture semantics is transition semantics, the frame picture is not determined to be a damaged frame picture.

5. The method of automatically repairing damaged video images using AI according to claim 3, characterized in that: The damaged frames in the video file are automatically identified through the frame identification module, which specifically includes: S401, after dividing the video file into a plurality of frames, preprocessing the obtained plurality of frames; S402, using a motion estimation algorithm to calculate a motion vector between two adjacent frames of a plurality of frames; S403, calculating the mean value of the motion vectors of all frames, and taking the mean value as a statistical threshold; S404, determining whether a motion vector between two adjacent frames among the plurality of frames is greater than a statistical threshold; S405 , judging whether the frame picture may be a damaged frame picture in combination with the judgment results of S204 and S404 .

6. The method of AI automatic repair of damaged video images according to claim 1, characterized in that: Process the image restoration task and obtain the restored frame image, which includes: S501, calculating the duration of frame damage according to the damaged frame start timing information and the damaged frame stop timing information; S502, acquiring a previous normal frame picture according to the damaged frame start timing information, and acquiring a subsequent normal frame picture according to the damaged frame stop timing information; S503, performing semantic recognition on the previous normal frame to obtain the semantics of the previous normal frame, and performing semantic recognition on the next normal frame to obtain the semantics of the next normal frame; S504, inputting the semantics of the preceding normal frame and the semantics of the succeeding normal frame into a semantic generation network for processing to obtain a semantic prediction result of the damaged frame; S505: Input the semantic prediction result of the damaged frame and the duration of the frame damage into the image generation model to generate multiple frames corresponding to the semantic prediction result, and each frame corresponds to a time point in the duration of the frame damage.

7. The method of automatically repairing damaged video images using AI according to claim 6, characterized in that: The semantics of the preceding normal frame and the following normal frame are input into the semantic generation network for processing to obtain the semantic prediction results of the damaged frame, which specifically include: S601, encoding the semantics of the preceding normal frame and the semantics of the succeeding normal frame through an encoder in a semantic generation network to obtain a context hidden vector matrix of the semantics of the damaged frame; S602: Using the context hidden vector matrix as the input of the decoder of the semantic generation network, the decoder captures the semantic information of the previous frame and the subsequent frame through the attention mechanism and gradually generates the semantic prediction results of the damaged frame.

8. The method of AI automatic repair of damaged video images according to claim 6 is characterized in that: The semantic prediction results of the damaged frame and the duration of the frame damage are input into the image generation model to generate multiple frames corresponding to the semantic prediction results, which specifically includes the following operations: S701, performing semantic enhancement on the semantic prediction result; S702, up-sampling the semantic prediction result after semantic enhancement to obtain a mask with the same size as the final generated frame image; S703: Map the semantic prediction result and the mask to the same semantic space, and obtain the final output frame image under the constraint of the duration of frame damage through the perceptron of the image generation model.

Images