Video repair method, apparatus, device, and storage medium

Abstract

The application discloses a video repairing method and device, equipment and a storage medium, and belongs to the computer technical field. The method comprises the following steps: acquiring an image set corresponding to a first video, the image set comprising a plurality of to-be-repaired images and a plurality of complete images, the complete image being an image that does not need to be repaired; performing a first repairing operation on the plurality of to-be-repaired images based on the plurality of complete images, to obtain a first image corresponding to each to-be-repaired image; sequentially performing a second repairing operation on each pixel block included in the first image based on a first target sequence, to obtain a second image, the first target sequence being a sequence obtained according to the size of the repairing value of each pixel block; and obtaining a first target video corresponding to the first video based on a plurality of second images corresponding to the plurality of to-be-repaired images. The method performs twice repairing operation on any to-be-repaired image in the plurality of to-be-repaired images, improves the accuracy of the image repairing result, and further improves the repairing quality of the video.

Description

Video restoration methods, devices, equipment and storage media Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video restoration method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of computer technology, video restoration technology has also become increasingly mature. How to repair damaged videos and how to improve the restoration effect are technical problems that need to be solved. Summary of the Invention

[0003] This application provides a video restoration method, apparatus, device, and storage medium, which can be used to improve the accuracy of video restoration results. The technical solution is as follows:

[0004] On one hand, embodiments of this application provide a video restoration method, the method comprising:

[0005] Obtain the image set corresponding to the first video, the image set including multiple images to be repaired and multiple complete images, the complete images being images that do not need to be repaired;

[0006] A first repair operation is performed on the multiple complete images to obtain a first image corresponding to each image to be repaired.

[0007] The second repair operation is performed on each pixel block of the first image in sequence based on the first target order to obtain the second image. The first target order is the order obtained by arranging the repair values ​​of each pixel block.

[0008] Based on the multiple second images corresponding to the multiple images to be repaired, the repaired first target video corresponding to the first video is obtained.

[0009] In one possible implementation, the first repair operation on the plurality of images to be repaired based on the plurality of complete images, to obtain a first image corresponding to each image to be repaired, includes:

[0010] For any one of the plurality of images to be repaired, if the image adjacent to any one of the images to be repaired is a first complete image, then a first repair operation is performed on any one of the images to be repaired based on the first complete image to obtain a first image corresponding to any one of the images to be repaired, wherein the first complete image belongs to the plurality of complete images.

[0011] In one possible implementation, the first repair operation on the plurality of images to be repaired based on the plurality of complete images, to obtain a first image corresponding to each image to be repaired, includes:

[0012] For multiple consecutive images to be repaired among the multiple images to be repaired, a first repair operation is performed on the first image to be repaired based on a second complete image adjacent to the first image to be repaired among the multiple consecutive images to be repaired, to obtain a first image corresponding to the first image to be repaired;

[0013] A first repair operation is performed on a second image adjacent to the first image to be repaired based on the first image to be repaired. The second image to be repaired belongs to the plurality of consecutive images to be repaired, and so on, until the first images corresponding to the plurality of consecutive images to be repaired are obtained respectively.

[0014] In one possible implementation, the first repair operation on the plurality of images to be repaired based on the plurality of complete images, to obtain a first image corresponding to each image to be repaired, includes:

[0015] For any one of the plurality of images to be repaired, a second optical flow between the fourth complete image and any image to be repaired is determined based on a first optical flow between the third complete image and the fourth complete image; a first repair operation is performed on any image to be repaired based on the second optical flow to obtain a first image corresponding to any image to be repaired. The third complete image is located before the fourth complete image in the playback order of the first video, and the fourth complete image is located before any image to be repaired in the playback order of the first video.

[0016] Alternatively, a fourth optical flow is determined between the fifth complete image and any image to be repaired based on a third optical flow between the fifth complete image and the sixth complete image; a first repair operation is performed on the image to be repaired based on the fourth optical flow to obtain a first image corresponding to the image to be repaired, wherein the sixth complete image is located after the fifth complete image in the playback order of the first video, and the fifth complete image is located after the image to be repaired in the playback order of the first video;

[0017] Alternatively, a sixth optical flow between the seventh complete image and any image to be repaired is determined based on a fifth optical flow between the seventh complete image and the eighth complete image, or a seventh optical flow between the eighth complete image and any image to be repaired is determined; a first repair operation is performed on any image to be repaired based on the sixth optical flow or the seventh optical flow to obtain a first image corresponding to the image to be repaired, wherein the seventh complete image is located before any image to be repaired in the playback order of the first video, and the eighth complete image is located after any image to be repaired in the playback order of the first video.

[0018] In one possible implementation, after performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired, the method further includes:

[0019] For the first pixel blocks whose repair values ​​are less than the first threshold among the repair values ​​of each pixel block included in the first image, the third repair operation is performed on the first pixel blocks in sequence based on the second target order to obtain the third image. The second target order is the order obtained by arranging the first pixel blocks according to the size of their repair values.

[0020] Based on the multiple third images corresponding to the multiple images to be repaired, the repaired second target video corresponding to the first video is obtained.

[0021] In one possible implementation, before performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired, the method further includes:

[0022] An image detection model is obtained, which is trained on a video including the image to be repaired and the complete image;

[0023] Based on the image detection model, multiple images to be repaired and multiple complete images are determined in the image set corresponding to the first video.

[0024] In one possible implementation, the first repair operation on the plurality of images to be repaired based on the plurality of complete images, to obtain a first image corresponding to each image to be repaired, includes:

[0025] Obtain the image restoration model;

[0026] The image restoration model performs a first restoration operation on the multiple images to be restored based on the multiple complete images, thereby obtaining a first image corresponding to each image to be restored.

[0027] On the other hand, a video restoration apparatus is provided, the apparatus comprising:

[0028] The acquisition module is used to acquire an image set corresponding to the first video. The image set includes multiple images to be repaired and multiple complete images, wherein the complete images are images that do not need to be repaired.

[0029] The repair module is used to perform a first repair operation on the multiple images to be repaired based on the multiple complete images, so as to obtain a first image corresponding to each image to be repaired;

[0030] The repair module is further configured to perform a second repair operation on each pixel block included in the first image in sequence based on a first target order to obtain a second image, wherein the first target order is the order obtained by arranging the repair values ​​of each pixel block.

[0031] The determining module is used to obtain the repaired first target video corresponding to the first video based on the multiple second images corresponding to the multiple images to be repaired.

[0032] In one possible implementation, the repair module is configured to, for any one of the plurality of images to be repaired, if an image adjacent to the image to be repaired is a first complete image, perform a first repair operation on the image to be repaired based on the first complete image to obtain a first image corresponding to the image to be repaired, wherein the first complete image belongs to the plurality of complete images.

[0033] In one possible implementation, the repair module is configured to, for multiple consecutive images to be repaired among the plurality of images to be repaired, perform a first repair operation on the first image to be repaired based on a second complete image adjacent to the first image to be repaired among the plurality of consecutive images to be repaired, to obtain a first image corresponding to the first image to be repaired; and perform a first repair operation on a second image to be repaired adjacent to the first image based on the first image corresponding to the first image to be repaired, wherein the second image to be repaired belongs to the plurality of consecutive images to be repaired, and so on, until a first image corresponding to each of the plurality of consecutive images to be repaired is obtained.

[0034] In one possible implementation, the repair module is configured to, for any one of the plurality of images to be repaired, determine a second optical flow between the fourth complete image and the image to be repaired based on a first optical flow between the third complete image and the fourth complete image; perform a first repair operation on the image to be repaired based on the second optical flow to obtain a first image corresponding to the image to be repaired, wherein the third complete image is located before the fourth complete image in the playback order of the first video, and the fourth complete image is located before the image to be repaired in the playback order of the first video;

[0035] Alternatively, a fourth optical flow is determined between the fifth complete image and any image to be repaired based on a third optical flow between the fifth complete image and the sixth complete image; a first repair operation is performed on the image to be repaired based on the fourth optical flow to obtain a first image corresponding to the image to be repaired, wherein the sixth complete image is located after the fifth complete image in the playback order of the first video, and the fifth complete image is located after the image to be repaired in the playback order of the first video;

[0036] Alternatively, a sixth optical flow between the seventh complete image and any image to be repaired is determined based on a fifth optical flow between the seventh complete image and the eighth complete image, or a seventh optical flow between the eighth complete image and any image to be repaired is determined; a first repair operation is performed on any image to be repaired based on the sixth optical flow or the seventh optical flow to obtain a first image corresponding to the image to be repaired, wherein the seventh complete image is located before any image to be repaired in the playback order of the first video, and the eighth complete image is located after any image to be repaired in the playback order of the first video.

[0037] In one possible implementation, the repair module is further configured to perform a third repair operation on the first pixel blocks corresponding to the repair values ​​of the pixel blocks included in the first image that are less than a first threshold, based on a second target order, to obtain a third image, wherein the second target order is the order obtained by arranging the first pixel blocks according to the size of the repair values ​​of the first pixel blocks;

[0038] The determining module is further configured to obtain the repaired second target video corresponding to the first video based on the multiple third images corresponding to the multiple images to be repaired.

[0039] In one possible implementation, the acquisition module is further configured to acquire an image detection model, which is trained on a video including the image to be repaired and the complete image;

[0040] The determining module is further configured to determine, based on the image detection model, multiple images to be repaired and multiple complete images in the image set corresponding to the first video.

[0041] In one possible implementation, the acquisition module is used to acquire an image restoration model;

[0042] The repair module is used to perform a first repair operation on the multiple images to be repaired based on the multiple complete images according to the image repair model, so as to obtain a first image corresponding to each image to be repaired.

[0043] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement any of the video restoration methods described above.

[0044] On the other hand, a computer-readable storage medium is also provided, wherein at least one computer program is stored therein, the at least one computer program being loaded and executed by a processor to enable a computer to implement any of the video restoration methods described above.

[0045] On the other hand, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the video restoration methods described above.

[0046] The technical solution provided in this application has at least the following beneficial effects:

[0047] In this embodiment, a first repair operation is first performed on multiple images to be repaired based on multiple complete images to obtain a first image; then, a second repair operation is performed on each pixel block in the first image according to the order of pixel block repair values ​​to obtain a second image; and the repaired video is obtained based on the second images corresponding to the multiple images to be repaired. In this method, each image to be repaired undergoes two repair operations, which improves the accuracy of the image repair results and further enhances the quality of the video repair. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0050] Figure 2 is a flowchart of a video restoration method provided in an embodiment of this application;

[0051] Figure 3 is a schematic diagram of multiple images corresponding to a first video provided in an embodiment of this application;

[0052] Figure 4 is a schematic diagram of the positional relationship between an image to be repaired and a complete image provided in an embodiment of this application;

[0053] Figure 5 is a schematic diagram of the positional relationship between the image to be repaired and the complete image provided in another embodiment of this application;

[0054] Figure 6 is a schematic diagram of the positional relationship between the image to be repaired and the complete image provided in another embodiment of this application;

[0055] Figure 7 is a structural schematic diagram of a video restoration device provided in an embodiment of this application;

[0056] Figure 8 is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0057] Figure 9 is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0059] This application provides a video restoration method. Please refer to Figure 1, which shows a schematic diagram of the implementation environment of the method provided in this application embodiment. The implementation environment may include: a terminal 11 and a server 12.

[0060] The terminal 11 has an application or webpage installed that can repair videos. When the application or webpage needs to repair a video, the method provided in this embodiment can be used. Furthermore, the server 12 can store the videos that need repair, and the terminal 11 can retrieve the videos from the server 12. Alternatively, the terminal 11 can also store the videos that need repair on its own device.

[0061] Optionally, terminal 11 can be any electronic product capable of human-computer interaction with the user through one or more methods such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device, such as PC (Personal Computer), mobile phone, smartphone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car system, smart TV, smart speaker, etc. Server 12 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Terminal 11 and server 12 establish a communication connection through wired or wireless network.

[0062] Those skilled in the art should understand that the above-described terminal 11 and server 12 are merely examples. Other existing or future terminals or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0063] This application provides a video restoration method, which can be applied to the implementation environment shown in Figure 1. As shown in Figure 2, taking the application of this method to a terminal as an example, the method includes the following steps 201 to 204.

[0064] In step 201, an image set corresponding to the first video is obtained. The image set includes multiple images to be repaired and multiple complete images. The complete images are those that do not need to be repaired.

[0065] The first video is the video that needs to be repaired. This application embodiment does not limit the type of the first video. For example, the first video can be a food video, a craft video, a travel video, or a video taken by a dashcam, etc. This application embodiment also does not limit the duration of the first video; the duration can be set based on experience or determined according to actual repair needs.

[0066] In this embodiment, the terminal may obtain the first video by receiving a first video uploaded by a user or by obtaining the first video from a video platform; this embodiment does not limit the method. Furthermore, the first video may refer to a complete video including the segment to be repaired, or it may be a portion of a complete video including the segment to be repaired, obtained by cropping.

[0067] Regardless of which of the above methods the terminal uses to acquire the first video, once the terminal acquires the first video, it can further determine the image set corresponding to the first video. Typically, a first video contains multiple video frames, and each video frame corresponds to one image. These multiple images are displayed sequentially to obtain a dynamic first video. Therefore, a first video corresponds to one image set, and an image set includes multiple images.

[0068] In an exemplary embodiment, the multiple images included in the image set can be arranged in a certain order in the image set. For example, the multiple images can be arranged from front to back according to the playback order of the video, or they can be arranged from back to front according to the playback order of the video.

[0069] In this embodiment, the image set includes multiple images to be repaired and multiple complete images. A complete image is an image that does not require repair; that is, the content of a complete image is undamaged or the complete image is free of defects. In the method provided in this embodiment, repair operations are performed on multiple images to be repaired based on multiple complete images, as detailed in subsequent steps and will not be repeated here.

[0070] After the terminal obtains the image set corresponding to the first video, it needs to further distinguish between the images to be repaired and the complete images in the image set. For example, this step can be implemented by calling an image detection model, such as calling an image detection model to determine multiple images to be repaired and multiple complete images in the image set based on the image set corresponding to the first video.

[0071] In one possible implementation, determining multiple images to be repaired and multiple complete images in an image set includes: acquiring an image detection model, which is trained on a video including the images to be repaired and the complete images; and determining multiple images to be repaired and multiple complete images in the image set corresponding to the first video based on the image detection model.

[0072] In one possible implementation, an initial image detection model can be trained to obtain an image detection model, which is then used to determine the image to be repaired and the complete image. In another possible implementation, the initial image detection model can be trained using a training set. This training set includes multiple image sets corresponding to multiple first videos. These first videos can be of any type and can be of the same or different types.

[0073] For any image set from the image sets corresponding to multiple first videos, after inputting this image set into the initial image detection model, the initial image detection model determines the detection result, which indicates the image to be repaired and the complete image detected by the initial image detection model. Then, based on the difference between the detection result and the true result, the result loss is obtained. The true result is the accurate image to be repaired and the complete image in this image set; for example, the true result can be obtained through manual annotation. Furthermore, this application embodiment does not limit the method of obtaining the result loss. For example, the cross-entropy or mean squared error loss between the detection result and the true result can be used as the result loss.

[0074] After obtaining the resulting loss, the parameters in the initial image detection model are updated based on the resulting loss to obtain the trained initial image detection model. Then, it is determined whether the trained initial image detection model meets the training termination condition. If the trained initial image detection model meets the training termination condition, it is used as the image detection model to be repaired, and is used to predict the image to be repaired and the complete image in the image set corresponding to the first video.

[0075] If the initial image detection model after training does not meet the training termination condition, the initial image detection model after training will continue to be updated in the manner described above, and so on, until an initial image detection model that meets the training termination condition is obtained. The initial image detection model that meets the training termination condition will then be used as the image detection model.

[0076] The termination conditions for training mentioned above are set based on experience or can be flexibly adjusted according to the application scenario, and this application embodiment does not limit them. For example, the termination conditions for the trained initial image detection model to meet training include, but are not limited to, any one of the following: the number of model parameter updates performed when obtaining the trained initial image detection model reaches a threshold, the result loss is less than a loss threshold, or the result loss converges.

[0077] In step 202, a first repair operation is performed on multiple images to be repaired based on multiple complete images to obtain a first image corresponding to each image to be repaired.

[0078] Since the positional relationship between the image to be repaired and the complete image is different, the first repair method is also different. Therefore, the first repair operation is performed on multiple images to be repaired based on multiple complete images to obtain the first image corresponding to each image to be repaired, including but not limited to the following method one or method two.

[0079] Method 1: For any image to be repaired among multiple images to be repaired, if the image adjacent to any image to be repaired is the first complete image, then perform the first repair operation on any image to be repaired based on the first complete image to obtain the first image corresponding to any image to be repaired. The first complete image belongs to multiple complete images.

[0080] This application does not limit the position of the first complete image relative to any image to be repaired in the embodiments. For example, the first complete image may refer to a complete image that is located before any image to be repaired in the playback order of the video and is adjacent to any image to be repaired, or it may refer to a complete image that is located after any image to be repaired in the playback order of the video and is adjacent to any image to be repaired.

[0081] Because the first complete image is adjacent to the first image to be repaired, the first repair operation can be performed directly on the first image to be repaired based on the first complete image.

[0082] Method 2: For multiple consecutive images to be repaired among multiple images to be repaired, based on the second complete image adjacent to the first image to be repaired among multiple consecutive images to be repaired, a first repair operation is performed on the first image to be repaired to obtain the first image corresponding to the first image to be repaired; based on the first image corresponding to the first image to be repaired, a first repair operation is performed on the second image to be repaired adjacent to the first image, the second image to be repaired belonging to multiple consecutive images to be repaired, and so on, until the first image corresponding to each of the multiple consecutive images to be repaired is obtained.

[0083] Figure 3 is a schematic diagram of multiple images corresponding to a first video. In Figure 3, images A, B, C, D, and E are five consecutive images in the playback order of the first video. Among them, images B, C, and D are three consecutive images to be repaired, while images A and E are complete images. In this case, to repair image C, it is necessary to first repair image B (the first image to be repaired) based on image A (the second complete image) to obtain the repaired image B, and then repair image C (the second image to be repaired) based on the repaired image B to obtain the repaired image C. This repair method can also be called a forward repair operation.

[0084] In some embodiments, image D (the first image to be repaired) can be repaired based on image E (the second complete image) to obtain the repaired image D. Then, image C (the second image to be repaired) can be repaired based on the repaired image D to obtain the repaired image C. This repair method can also be called a reverse repair operation.

[0085] Furthermore, the final restored image C can be obtained based on the restoration results of image C from restored image B and image C from restored image D. For example, restoring image C based on restored image B yields restoration result C1, and restoring image C based on restored image D yields restoration result C2. By taking a weighted average of restoration results C1 and C2, the final restored image C can be obtained. This method combines forward and reverse restoration operations, which can improve the restoration quality of image C.

[0086] In one possible implementation, optical flow inpainting can be used to repair the image to be repaired based on the complete image. Optical flow is a method that uses the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames. That is, the optical flow between two adjacent frames can be determined based on the optical flow between two first adjacent frames, where the first two adjacent frames are located after the second adjacent frames in the video playback order.

[0087] Based on the different positional relationships between the complete image and the image to be repaired, the first repair operation on multiple images to be repaired based on multiple complete images can be performed in any of the following methods three to five.

[0088] Method 3: For any one of the multiple images to be repaired, determine the second optical flow between the fourth complete image and any image to be repaired based on the first optical flow between the third complete image and the fourth complete image; perform a first repair operation on any image to be repaired based on the second optical flow to obtain a first image corresponding to any image to be repaired, wherein the third complete image is located before the fourth complete image in the playback order of the first video, and the fourth complete image is located before any image to be repaired in the playback order of the first video.

[0089] This situation can be illustrated in Figure 4, which is a schematic diagram of the positional relationship between the image to be repaired and the complete image. In Figure 4, images F, G, and H are three consecutive images in the playback order of the first video, with images F and G being complete images and image H being the image to be repaired. For example, optical flow features can be extracted from images F (the third complete image) and G (the fourth complete image) to obtain the first optical flow between them. Then, based on the first optical flow, the second optical flow between image G (the fourth complete image) and image H (any image to be repaired) can be determined. After determining the second optical flow, the motion information of the object in image H can be determined, and then image H can be repaired to obtain the first image corresponding to image H.

[0090] Method 4: Determine the fourth optical flow between the fifth complete image and any image to be repaired based on the third optical flow between the fifth complete image and the sixth complete image; perform a first repair operation on any image to be repaired based on the fourth optical flow to obtain a first image corresponding to any image to be repaired. The sixth complete image is located after the fifth complete image in the playback order of the first video, and the fifth complete image is located after any image to be repaired in the playback order of the first video.

[0091] This situation can be illustrated in Figure 5, which is a schematic diagram of the positional relationship between the image to be repaired and the complete image. In Figure 5, images I, J, and K are three consecutive images in the playback order of the first video, and images J and K are complete images, while image I is the image to be repaired. For example, optical flow features can be extracted from images J (the fifth complete image) and K (the sixth complete image) to obtain the third optical flow between them. Then, based on the third optical flow, the fourth optical flow between image J (the fifth complete image) and image I (any image to be repaired) can be determined. After determining the fourth optical flow, the motion information of the object in image I can be determined, and then image I can be repaired to obtain the first image corresponding to image I.

[0092] Method 5: Determine the sixth optical flow between the seventh complete image and any image to be repaired based on the fifth optical flow between the seventh complete image and the eighth complete image, or determine the seventh optical flow between the eighth complete image and any image to be repaired; perform a first repair operation on any image to be repaired based on the sixth or seventh optical flow to obtain a first image corresponding to any image to be repaired, wherein the seventh complete image is located before any image to be repaired in the playback order of the first video, and the eighth complete image is located after any image to be repaired in the playback order of the first video.

[0093] This situation can be illustrated in Figure 6, which is a schematic diagram of the positional relationship between the image to be repaired and the complete image. In Figure 6, images L, M, and N are three consecutive images in the playback order of the first video, with images L and N being complete images and image M being the image to be repaired. For example, optical flow features can be extracted from images L (the seventh complete image) and N (the eighth complete image) to obtain the fifth optical flow between them. Then, based on the fifth optical flow, the sixth optical flow between image L (the seventh complete image) and image M (any image to be repaired) can be determined. After determining the sixth optical flow, the motion information of the object in image M can be determined, and then image M can be repaired to obtain the first image corresponding to image M. Alternatively, the seventh optical flow between image N (the eighth complete image) and image M (any image to be repaired) can be determined based on the fifth optical flow, and then the motion information of the object in image M can be determined based on the seventh optical flow, and then image M can be repaired to obtain the first image corresponding to image M.

[0094] In an exemplary embodiment, the method of performing a first repair operation on multiple images to be repaired based on multiple complete images to obtain a first image corresponding to each image to be repaired can be as follows: obtaining an image repair model; performing a first repair operation on multiple images to be repaired based on multiple complete images according to the image repair model to obtain a first image corresponding to each image to be repaired.

[0095] In one possible implementation, an image restoration model can be obtained by training an initial image restoration model. After obtaining the image restoration model, multiple images to be restored are input into the image restoration model, and the image restoration model can output the first image corresponding to each image to be restored obtained after the first restoration operation.

[0096] For example, the initial image inpainting model can be trained using a training set, which includes multiple images to be inpainted. Otherwise, the training method for the image inpainting model is basically the same as the training principle of the image detection model described earlier, and will not be repeated here.

[0097] In step 203, the second repair operation is performed on each pixel block of the first image in sequence based on the first target order to obtain the second image. The first target order is the order obtained by arranging the repair values ​​of each pixel block.

[0098] Typically, an image can be divided into multiple pixel blocks. In this embodiment, the first image is uniformly divided to obtain multiple pixel blocks of the same size. Then, a second repair operation is performed on each pixel block in a first target order. The first target order is the order obtained by arranging the repair values ​​of each pixel block according to their magnitude; this application does not limit the arrangement method. For example, the repair values ​​of each pixel block can be arranged from largest to smallest to obtain the first target order, or they can be arranged from smallest to largest. Furthermore, the repair value of each pixel block can be determined by: obtaining a repair value determination model; determining the repair value of each pixel block based on the repair value determination model, where the repair value indicates the degree of repair for each pixel block, and the repair value and the degree of repair are positively correlated.

[0099] In one possible implementation, the initial repair value determination model can be trained to obtain the repair value determination model. Then, the multiple pixel blocks obtained after division can be input into the repair value determination model, and the repair value determination model can output the repair value of each pixel block.

[0100] For example, an initial restoration value determination model can be trained using a training set. The training set includes multiple pixel blocks and their corresponding ground truth restoration values. These pixel blocks are then input into the initial restoration value determination model, which outputs the detected restoration values ​​for each pixel block. A loss function is obtained based on the difference between the detected restoration values ​​and the ground truth restoration values. After obtaining the loss function, the training method for the restoration value determination model is essentially the same as the training principles of the image detection and image restoration models described earlier, and will not be repeated here.

[0101] This application does not limit the form in which the repair value is expressed. For example, the repair value can be expressed as a percentage or as a natural number. The repair value is used to indicate the degree of repair of a pixel block, and the larger the repair value, the higher the degree of repair of the pixel block.

[0102] After determining the first target order based on the repair values ​​of each pixel block, the second repair operation needs to be performed on each pixel block sequentially according to the first target order. That is, the second repair operation is performed on the pixel blocks in descending order of their repair values. A larger repair value indicates a better repair effect, making subsequent repairs easier and more accurate. Furthermore, when performing the second repair operation on pixel blocks with smaller repair values, in addition to repairing based on adjacent images, repair can also be performed based on adjacent pixel blocks located within the same image to be repaired. In this case, if the accuracy of adjacent pixel blocks is higher, the accuracy of the second repair operation on that pixel block will also be higher. Therefore, performing the second repair operation on each pixel block in descending order of its repair values ​​can improve the overall accuracy of the repair results for each pixel block, enhance the image repair quality, and ultimately improve the video repair quality.

[0103] Furthermore, in some embodiments, besides performing the second repair operation on each pixel block of any image to be repaired, the second repair operation can also be performed only on pixel blocks whose repair values ​​are less than a threshold. That is, the second repair operation is not performed on pixel blocks whose repair values ​​are greater than the threshold. Therefore, in one possible implementation, this step includes: for the first pixel blocks corresponding to repair values ​​less than a first threshold among the repair values ​​of the pixel blocks included in the first image, performing the third repair operation on the first pixel blocks sequentially based on a second target order to obtain a third image, wherein the second target order is the order obtained by arranging the first pixel blocks according to the size of their repair values.

[0104] In this embodiment, the form of the first threshold is not limited; it can be a percentage, a natural number, etc., as long as the form of the first threshold is the same as the form of the repair value of the pixel block. Furthermore, the size of the first threshold can be set empirically or flexibly adjusted according to the actual application scenario. For example, taking the first threshold as a percentage, it can be 90%, 80%, etc.

[0105] For pixel blocks with a repair value greater than the first threshold, the pixel block can be considered to have met the repair target, and no further repair operation (third repair operation) is needed. The third repair operation can then be performed only on pixel blocks with a repair value less than the first threshold, making the repair results more accurate and improving repair efficiency. Otherwise, the principle of the third repair operation is the same as that of the second repair operation described above, and will not be repeated here.

[0106] In step 204, based on the multiple second images corresponding to the multiple images to be repaired, the repaired first target video corresponding to the first video is obtained.

[0107] Multiple second images are images obtained by performing the first repair operation and the second repair operation on multiple images to be repaired respectively. By replacing multiple images to be repaired with multiple second images, the repaired first target video corresponding to the first video can be generated.

[0108] Regarding the method described in step 203 above, which only performs the third repair operation on the first pixel block whose repair value is less than the threshold, multiple images to be repaired correspond to multiple third images. Similarly, by replacing multiple images to be repaired with multiple third images, the repaired second target video corresponding to the first video can be generated.

[0109] In this embodiment, a first repair operation is first performed on multiple images to be repaired based on multiple complete images to obtain a first image; then, a second repair operation is performed on each pixel block in the first image according to the order of pixel block repair values ​​to obtain a second image; and the repaired video is obtained based on the second images corresponding to the multiple images to be repaired. In this method, each image to be repaired undergoes two repair operations, which improves the accuracy of the image repair results and further enhances the quality of the video repair.

[0110] Referring to Figure 7, an embodiment of this application provides a video restoration device, which includes:

[0111] The acquisition module 701 is used to acquire the image set corresponding to the first video. The image set includes multiple images to be repaired and multiple complete images. The complete images are those that do not need to be repaired.

[0112] Repair module 702 is used to perform a first repair operation on multiple images to be repaired based on multiple complete images, so as to obtain a first image corresponding to each image to be repaired;

[0113] The repair module 702 is also used to perform a second repair operation on each pixel block included in the first image in sequence based on the first target order to obtain a second image. The first target order is the order obtained by arranging the repair values ​​of each pixel block.

[0114] The determination module 703 is used to obtain the repaired first target video corresponding to the first video based on multiple second images corresponding to multiple images to be repaired.

[0115] In one possible implementation, the repair module 702 is used to perform a first repair operation on any image to be repaired based on the first complete image if any image adjacent to any image to be repaired is a first complete image, thereby obtaining a first image corresponding to any image to be repaired, wherein the first complete image belongs to multiple complete images.

[0116] In one possible implementation, the repair module 702 is used to perform a first repair operation on a first image to be repaired based on a second complete image adjacent to the first image to be repaired in a plurality of consecutive images to be repaired, to obtain a first image corresponding to the first image to be repaired; and to perform a first repair operation on a second image to be repaired adjacent to the first image based on the first image corresponding to the first image to be repaired, wherein the second image to be repaired belongs to a plurality of consecutive images to be repaired, and so on, until a first image corresponding to each of the plurality of consecutive images to be repaired is obtained.

[0117] In one possible implementation, the repair module 702 is used to determine, for any one of a plurality of images to be repaired, a second optical flow between the fourth complete image and any image to be repaired based on a first optical flow between the third complete image and the fourth complete image; and to perform a first repair operation on any image to be repaired based on the second optical flow to obtain a first image corresponding to any image to be repaired, wherein the third complete image is located before the fourth complete image in the playback order of the first video, and the fourth complete image is located before any image to be repaired in the playback order of the first video.

[0118] Alternatively, a fourth optical flow between the fifth complete image and any image to be repaired is determined based on the third optical flow between the fifth complete image and the sixth complete image; a first repair operation is performed on any image to be repaired based on the fourth optical flow to obtain a first image corresponding to any image to be repaired, wherein the sixth complete image is located after the fifth complete image in the playback order of the first video, and the fifth complete image is located after any image to be repaired in the playback order of the first video.

[0119] Alternatively, based on the fifth optical flow between the seventh complete image and the eighth complete image, determine the sixth optical flow between the seventh complete image and any image to be repaired, or determine the seventh optical flow between the eighth complete image and any image to be repaired; perform a first repair operation on any image to be repaired based on the sixth or seventh optical flow to obtain a first image corresponding to any image to be repaired, wherein the seventh complete image is located before any image to be repaired in the playback order of the first video, and the eighth complete image is located after any image to be repaired in the playback order of the first video.

[0120] In one possible implementation, the repair module 702 is further configured to perform a third repair operation on the first pixel blocks corresponding to the repair values ​​of the repair values ​​of the pixel blocks included in the first image that are less than the first threshold, based on a second target order, to obtain a third image. The second target order is the order obtained by arranging the repair values ​​of the first pixel blocks according to their magnitude.

[0121] The determining module 703 is also used to obtain the repaired second target video corresponding to the first video based on the multiple third images corresponding to the multiple images to be repaired.

[0122] In one possible implementation, the acquisition module 701 is also used to acquire an image detection model, which is trained on a video including the image to be repaired and the complete image.

[0123] The determination module 703 is also used to determine multiple images to be repaired and multiple complete images in the image set corresponding to the first image based on the image detection model.

[0124] In one possible implementation, module 701 is used to acquire an image restoration model;

[0125] The repair module 702 is used to perform a first repair operation on multiple images to be repaired based on multiple complete images according to the image repair model, so as to obtain a first image corresponding to each image to be repaired.

[0126] In this embodiment, a first repair operation is first performed on multiple images to be repaired based on multiple complete images to obtain a first image; then, a second repair operation is performed on each pixel block in the first image according to the order of pixel block repair values ​​to obtain a second image; and the repaired video is obtained based on the second images corresponding to the multiple images to be repaired. In this method, each image to be repaired undergoes two repair operations, which improves the accuracy of the image repair results and further enhances the quality of the video repair.

[0127] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0128] Figure 8 is a schematic diagram of a server structure provided in an embodiment of this application. The server can vary significantly due to differences in configuration or performance, and may include one or more processors 801 and one or more memories 802. The processor 801 may be a Central Processing Unit (CPU). The one or more memories 802 store at least one computer program, which is loaded and executed by the one or more processors 801 to enable the server to implement the video repair methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0129] Figure 9 is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may be, for example, a smartphone, tablet computer, laptop computer, or desktop computer. The terminal may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0130] Typically, a terminal includes a processor 1501 and a memory 1502.

[0131] Processor 1501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0132] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1502 are used to store at least one instruction, which is executed by the processor 1501 to cause the terminal to implement the video repair method provided in the method embodiments of this application.

[0133] In some embodiments, the terminal may also optionally include: a peripheral device interface 1503 and at least one peripheral device. The processor 1501, memory 1502, and peripheral device interface 1503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, an audio circuit 1507, a positioning assembly 1508, and a power supply 1509.

[0134] Peripheral interface 1503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1501 and memory 1502. In some embodiments, processor 1501, memory 1502 and peripheral interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1501, memory 1502 and peripheral interface 1503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0135] The radio frequency (RF) circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1504 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0136] Display screen 1505 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1505 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1501 for processing. In this case, display screen 1505 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1505 can be a single screen, located on the front panel of the terminal; in other embodiments, display screen 1505 can be at least two screens, respectively located on different surfaces of the terminal or in a folded design; in other embodiments, display screen 1505 can be a flexible display screen, located on a curved or folded surface of the terminal. Furthermore, display screen 1505 can be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 1505 can be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0137] The camera assembly 1506 is used to acquire images or videos. Optionally, the camera assembly 1506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0138] The audio circuit 1507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1501 for processing, or input to the radio frequency circuit 1504 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1501 or the radio frequency circuit 1504 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1507 may also include a headphone jack.

[0139] The positioning component 1508 is used to locate the current geographical location of the terminal in order to enable navigation or LBS (Location Based Service).

[0140] Power supply 1509 is used to power the various components in the terminal. Power supply 1509 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1509 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0141] In some embodiments, the terminal further includes one or more sensors 1510. The one or more sensors 1510 include, but are not limited to: an accelerometer 1511, a gyroscope 1512, a pressure sensor 1513, a fingerprint sensor 1514, an optical sensor 1515, and a proximity sensor 1516.

[0142] Accelerometer 1511 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by the terminal. For example, accelerometer 1511 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1501 can control display screen 1505 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1511. Accelerometer 1511 can also be used for games or for acquiring user motion data.

[0143] The gyroscope sensor 1512 can detect the terminal's orientation and rotation angle. The gyroscope sensor 1512, in conjunction with the accelerometer sensor 1511, can collect the user's 3D movements on the terminal. Based on the data collected by the gyroscope sensor 1512, the processor 1501 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0144] The pressure sensor 1513 can be disposed on the side bezel of the terminal and / or the lower layer of the display screen 1505. When the pressure sensor 1513 is disposed on the side bezel of the terminal, it can detect the user's grip signal on the terminal, and the processor 1501 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1513. When the pressure sensor 1513 is disposed on the lower layer of the display screen 1505, the processor 1501 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1505. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0145] The fingerprint sensor 1514 is used to collect a user's fingerprint. The processor 1501 identifies the user based on the fingerprint collected by the fingerprint sensor 1514, or vice versa. When the user's identity is identified as trusted, the processor 1501 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1514 can be located on the front, back, or side of the terminal. When the terminal has physical buttons or a manufacturer's logo, the fingerprint sensor 1514 can be integrated with the physical buttons or the manufacturer's logo.

[0146] Optical sensor 1515 is used to collect ambient light intensity. In one embodiment, processor 1501 can control the display brightness of display screen 1505 based on the ambient light intensity collected by optical sensor 1515. Specifically, when the ambient light intensity is high, the display brightness of display screen 1505 is increased; when the ambient light intensity is low, the display brightness of display screen 1505 is decreased. In another embodiment, processor 1501 can also dynamically adjust the shooting parameters of camera assembly 1506 based on the ambient light intensity collected by optical sensor 1515.

[0147] The proximity sensor 1516, also known as a distance sensor, is typically installed on the front panel of the terminal. The proximity sensor 1516 is used to detect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1516 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1501 controls the display screen 1505 to switch from a screen-on state to a screen-off state; when the proximity sensor 1516 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1501 controls the display screen 1505 to switch from a screen-off state to a screen-on state.

[0148] Those skilled in the art will understand that the structure shown in Figure 9 does not constitute a limitation on the terminal, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0149] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the video restoration methods described above.

[0150] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the video restoration methods described above.

[0151] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0152] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the video restoration methods described above.

[0153] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the first video and other data involved in this application were obtained with full authorization.

[0154] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0155] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the above exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0156] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A video restoration method, characterized in that, The method includes: acquiring an image set corresponding to a first video, the image set including multiple images to be repaired and multiple complete images, the complete images being images that do not require repair; acquiring an image detection model, the image detection model being trained on a video including images to be repaired and complete images; determining multiple images to be repaired and multiple complete images in the image set corresponding to the first video based on the image detection model; performing a first repair operation on the multiple images to be repaired based on the multiple complete images to obtain a first image corresponding to each image to be repaired; uniformly dividing the first image to obtain multiple pixel blocks of the same size, and performing a second repair operation on each pixel block included in the first image in sequence according to a first target order to obtain a second image, the first target order being the order obtained by arranging the repair values ​​of each pixel block from largest to smallest, the repair value being used to indicate the repair degree of each pixel block, and the repair value and the repair degree being positively correlated; replacing the multiple images to be repaired based on the multiple second images corresponding to the multiple images to be repaired respectively to obtain a repaired first target video corresponding to the first video.

2. The method according to claim 1, characterized in that, The step of performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired includes: for any image to be repaired among the plurality of images to be repaired, if the image adjacent to any image to be repaired is a first complete image, then performing a first repair operation on any image to be repaired based on the first complete image to obtain a first image corresponding to any image to be repaired, wherein the first complete image belongs to the plurality of complete images.

3. The method according to claim 1, characterized in that, The step of performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired includes: for a plurality of consecutive images to be repaired among the plurality of images to be repaired, performing a first repair operation on the first image to be repaired based on a second complete image adjacent to the first image to be repaired among the plurality of consecutive images to be repaired, to obtain a first image corresponding to the first image to be repaired; performing a first repair operation on a second image to be repaired adjacent to the first image based on the first image corresponding to the first image to be repaired, wherein the second image to be repaired belongs to the plurality of consecutive images to be repaired, and so on, until a first image corresponding to each of the plurality of consecutive images to be repaired is obtained.

4. The method according to claim 1, characterized in that, The first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired includes: for any image to be repaired among the plurality of images to be repaired, determining a second optical flow between the fourth complete image and any image to be repaired based on a first optical flow between the third complete image and the fourth complete image; performing a first repair operation on any image to be repaired based on the second optical flow to obtain a first image corresponding to any image to be repaired, wherein the third complete image is located before the fourth complete image in the playback order of the first video, and the fourth complete image is located before any image to be repaired in the playback order of the first video; or, determining a fourth optical flow between the fifth complete image and any image to be repaired based on a third optical flow between the fifth complete image and the sixth complete image; performing a first repair operation on any image to be repaired based on the fourth optical flow. The image is repaired by performing a first repair operation to obtain a first image corresponding to any image to be repaired. The sixth complete image is located after the fifth complete image in the playback order of the first video, and the fifth complete image is located after any image to be repaired in the playback order of the first video. Alternatively, the sixth optical flow between the seventh complete image and any image to be repaired or the seventh optical flow between the eighth complete image and any image to be repaired is determined based on the fifth optical flow between the seventh complete image and the eighth complete image. The first repair operation is performed on any image to be repaired based on the sixth optical flow or the seventh optical flow to obtain a first image corresponding to any image to be repaired. The seventh complete image is located before any image to be repaired in the playback order of the first video, and the eighth complete image is located after any image to be repaired in the playback order of the first video.

5. The method according to claim 1, characterized in that, The step of performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images to obtain a first image corresponding to each image to be repaired includes: obtaining an image repair model; and performing a first repair operation on the plurality of images to be repaired based on the plurality of complete images according to the image repair model to obtain a first image corresponding to each image to be repaired.

6. A video restoration device, characterized in that, The apparatus includes: an acquisition module for acquiring an image set corresponding to a first video, the image set including multiple images to be repaired and multiple complete images, the complete images being images that do not require repair; the acquisition module is further configured to acquire an image detection model, the image detection model being trained on a video including images to be repaired and complete images; a determination module for determining multiple images to be repaired and multiple complete images in the image set corresponding to the first video based on the image detection model; a repair module for performing a first repair operation on the multiple images to be repaired based on the multiple complete images, obtaining a first image corresponding to each image to be repaired; the repair module is further configured to uniformly divide the first image into multiple pixel blocks of the same size, and sequentially perform a second repair operation on each pixel block included in the first image according to a first target order, the first target order being the order obtained by arranging the repair values ​​of each pixel block from largest to smallest, the repair value being used to indicate the repair degree of each pixel block, and the repair value and the repair degree being positively correlated; the determination module is further configured to replace the multiple images to be repaired based on the multiple second images corresponding to the multiple images to be repaired, obtaining a repaired first target video corresponding to the first video.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the computer device to implement the video restoration method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the video restoration method as described in any one of claims 1 to 5.

Images