Video processing method and apparatus, and device, computer-readable storage medium and product

Abstract

Provided in the embodiments of the present disclosure are a video processing method and apparatus, and a device, a computer-readable storage medium and a product. The video processing method comprises: acquiring a video to be processed, and identifying at least one defect to be repaired that corresponds to the current video frame in the video to be processed; on the basis of the current video frame and at least one associated video frame that corresponds to the current video frame, determining motion information that corresponds to the current video frame; on the basis of the motion information, performing a repair operation on the at least one defect to be repaired that corresponds to the current video frame, so as to obtain a repaired first video frame; and on the basis of a preset restoration algorithm, restoring the visual effect of the first video frame, so as to obtain a target video frame, and on the basis of at least one target video frame, obtaining a repaired target video. Therefore, a repaired target video does not comprise defective regions and presents a better visual effect, thereby improving the video quality of the repaired target video.

Description

Video processing method, device, equipment, computer-readable storage medium and product

[0001] This application claims priority to Chinese patent application No. 202311672278.9 filed on December 7, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a video processing method, apparatus, device, computer-readable storage medium, and product. Background Art

[0003] With the continuous advancement of audio and video technology, older videos may often suffer from scratches, frame drops, discoloration, and other issues when played back using current playback tools, affecting the visual quality of the videos. Therefore, how to repair defects in older videos and achieve higher-quality visual effects has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present disclosure provide a video processing method, apparatus, device, computer-readable storage medium, and product for solving the technical problems of scratches, frame loss, and discoloration in old videos.

[0005] In a first aspect, an embodiment of the present disclosure provides a video processing method, including:

[0006] Acquire a video to be processed, and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed;

[0007] determining motion information corresponding to the current video frame according to the current video frame and at least one associated video frame corresponding to the current video frame;

[0008] Performing a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame;

[0009] Perform visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame, and obtain a restored target video based on at least one target video frame.

[0010] In a second aspect, an embodiment of the present disclosure provides a video processing device, including:

[0011] An acquisition module, configured to acquire a video to be processed and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed;

[0012] a determination module, configured to determine motion information corresponding to the current video frame based on the current video frame and at least one associated video frame corresponding to the current video frame;

[0013] a repairing module, configured to perform a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information, to obtain a repaired first video frame;

[0014] The processing module is used to perform visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame, and obtain a restored target video based on at least one target video frame.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing method described in the first aspect and various possible designs of the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0019] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video processing method described in the first aspect and various possible designs of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, a brief introduction will be given below to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] FIG1 is a flow chart of a video processing method according to an embodiment of the present disclosure;

[0022] FIG2 is a schematic diagram of video frame restoration provided by an embodiment of the present disclosure;

[0023] FIG3 is a flow chart of a video processing method according to another embodiment of the present disclosure;

[0024] FIG4 is a flow chart of a video processing method provided by another embodiment of the present disclosure;

[0025] FIG5 is a flow chart of a video processing method provided by another embodiment of the present disclosure;

[0026] FIG6 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure; and

[0027] FIG7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0029] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0030] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0031] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0032] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0033] In order to solve the technical problems of scratches, frame loss, and discoloration in old videos, the present disclosure provides a video processing method, device, equipment, computer-readable storage medium, and product.

[0034] It should be noted that the video processing method, apparatus, device, computer-readable storage medium, and product provided by the present disclosure can be applied in any video restoration scenario.

[0035] Due to their age, many classic films suffer from scratches, poor detail, discoloration, frame loss, and other issues when displayed on current media players, impacting the visual experience. Current film restoration methods typically focus solely on repairing scratches. However, since classic films often suffer from discoloration and frame loss, simply repairing scratches on them can be ineffective, impacting the viewing experience.

[0036] In the process of solving the above-mentioned technical problems, the inventors discovered through research that, to ensure the restoration effect of old movies, scratches and noise corresponding to each video frame can be repaired based on the optical flow information corresponding to that video frame. Furthermore, the color and resolution of each video frame can be restored to improve the visual quality of the video frame. To improve the smoothness of the restored target video, frame insertion can also be performed on the video sections with lower frame rates. This allows for comprehensive restoration of old movies, improving the video quality of the restored target video.

[0037] The system architecture underlying this disclosure includes at least a terminal device and a server. The terminal device and the server are communicatively connected, enabling the server to receive a repair request sent by the terminal device and, based on the repair request, retrieve a video to be processed. The server then repairs the defects, color, resolution, and frame rate of the video to be processed using at least one pre-defined repair algorithm, thereby improving the quality of the repaired target video.

[0038] FIG1 is a flow chart of a video processing method provided by an embodiment of the present disclosure. As shown in FIG1 , the method includes:

[0039] Step 101: Obtain a video to be processed, and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed.

[0040] The execution subject of this embodiment is a video processing device. The video processing device can be coupled to a server. The server is capable of communicating with a terminal device. The server can thereby receive a repair request sent by the terminal device and, based on the repair request, obtain a video to be processed. Repair operations are performed on the defects, color, resolution, and frame rate of the video to be processed to improve the quality of the repaired target video.

[0041] Alternatively, the video processing device can be coupled to a terminal device, which can be pre-installed with multiple restoration algorithms. This allows the device to obtain a video to be processed in response to a video restoration request initiated by a user on the terminal device. Repair operations are performed on the video to improve the quality of the restored target video, such as defects, color, resolution, and frame rate.

[0042] In practice, there may be a large number of videos with poor quality that need to be restored. For example, these include old movies or videos shot in poor lighting conditions. To restore and optimize these videos, users can trigger a video restoration request on their devices.

[0043] Accordingly, the video processing device may obtain the video repair request and obtain the video to be processed based on the video repair request. The video to be processed may be obtained by the user from a preset storage path.

[0044] Furthermore, to perform a repair operation on the video to be processed, each video frame in the video to be processed may be sequentially determined as the current video frame, and at least one defect to be repaired corresponding to the current video frame may be identified. The defect to be repaired includes, but is not limited to, scratch defects, noise defects, and the like.

[0045] Step 102: Determine motion information corresponding to the current video frame according to the current video frame and at least one associated video frame corresponding to the current video frame.

[0046] In this embodiment, after identifying at least one defect to be repaired, motion information corresponding to the current video frame can be determined based on the current video frame and at least one associated video frame corresponding to the current video frame. This motion information includes, but is not limited to, any information capable of characterizing the motion of the content displayed in the current video frame, such as optical flow information.

[0047] In practical applications, if the current video frame has scratches, an associated video frame adjacent to it and displaying similar content may not have scratches. Therefore, the current video frame can be repaired based on the associated video frame.

[0048] By identifying the motion information corresponding to the current video frame, the position in the associated video frame that matches the defect of the current video frame can be determined based on the motion information, and then the scratch repair operation can be performed on the current video frame based on the display content in the associated video frame, thereby improving the accuracy of video repair.

[0049] Step 103: Perform a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame.

[0050] In this embodiment, after determining the motion information corresponding to the current video frame, a repair operation can be performed on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame. A defect repair model can be pre-set to implement the repair operation on the current video frame.

[0051] Step 104: Perform visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame, and obtain a restored target video based on at least one target video frame.

[0052] In this embodiment, after repairing defects such as scratches and noise in the video to be processed, in order to improve the visual effect of the video to be processed, the color and resolution corresponding to the video to be processed may also be repaired.

[0053] Optionally, after obtaining the defect-free first video frame, visual effect restoration can be performed on the first video frame to obtain a target video frame, wherein the visual effect restoration includes but is not limited to color restoration, resolution restoration, and other operations.

[0054] Furthermore, after obtaining at least one target video frame, the at least one target video frame may be combined according to the timestamp information of the at least one target video frame to obtain a restored target video.

[0055] Figure 2 is a schematic diagram of video frame restoration according to an embodiment of the present disclosure. As shown in Figure 2 , at least one scratch 22 is present in a video frame 21. After acquiring video frame 21, a restoration operation can be performed on the at least one scratch 22 based on the motion information corresponding to the video frame 21, resulting in a restored first video frame 23. Furthermore, the color and resolution of the first video frame 23 can be adjusted to obtain a target video frame 24.

[0056] The video processing method provided in this embodiment, after acquiring the video to be processed, performs defect repair on the target video based on the motion information corresponding to the current video frame, thereby accurately performing defect repair operations based on the motion trend of the main content in the current video frame, thereby improving the accuracy of defect repair. In addition, after completing the defect repair of the current video frame, the visual effects of the first video frame can also be repaired. Therefore, on the basis of achieving defect repair, the repaired target video can present a higher-quality visual effect, thereby improving the video quality of the repaired target video.

[0057] Optionally, based on any of the above embodiments, the defects to be repaired include scratch defects and noise defects. Step 101 includes:

[0058] The current video frame is input into a preset defect recognition model to obtain a recognition result output by the defect recognition model, wherein the recognition result includes position information of a scratch defect and / or a noise defect.

[0059] The identification result is determined as the at least one defect to be repaired.

[0060] In this embodiment, the defects to be repaired include but are not limited to scratch defects and noise defects.

[0061] To identify defects to be repaired in the current video frame, a defect recognition model can be pre-set. This defect recognition model can be obtained through iterative training based on a large amount of training data. This training data can include multiple training data pairs, each of which includes an image with at least one scratch and / or noise, and an image without scratches and / or noise. The pre-set model can be iteratively trained using the training data until it meets pre-set convergence criteria, resulting in the defect recognition model. This defect recognition model can thus accurately identify scratch and / or noise defects.

[0062] Therefore, after acquiring the video to be processed, each video frame can be sequentially identified as the current video frame, and the current video frame can be input into a preset defect recognition model to obtain the recognition result output by the defect recognition model, where the recognition result includes the location information of the scratch defect and / or noise defect. The recognition result is determined as at least one defect to be repaired.

[0063] The video processing method provided in this embodiment pre-sets a defect recognition model, so that after obtaining the current video frame, it can accurately identify scratches and / or noise in the current video frame based on the defect recognition model, and then can accurately perform video repair operations based on the recognition results, thereby improving the accuracy of video repair.

[0064] FIG3 is a flow chart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG3 , step 102 includes:

[0065] Step 301: Determine at least one video frame before the current video frame and at least one video frame after the current video frame as the associated video frames according to timestamp information corresponding to the video frames in the video to be processed.

[0066] Step 302: Calculate optical flow information corresponding to the current video frame based on the current video frame and the at least one associated video frame, and determine the optical flow information as the motion information.

[0067] In this embodiment, in order to implement the repair operation on the video to be processed, for each video frame in the video to be processed, it is necessary to sequentially determine each video frame as the current video frame and determine at least one associated video frame corresponding to the current video frame.

[0068] Optionally, each video frame in the video to be processed has corresponding timestamp information. Therefore, based on the timestamp information, at least one video frame before the current video frame and at least one video frame after the current video frame can be determined as associated video frames. Optical flow information corresponding to the current video frame is calculated based on the current video frame and the at least one associated video frame, and the optical flow information is determined as motion information. Any optical flow algorithm can be used to calculate the optical flow information, and this disclosure does not impose any restrictions on this.

[0069] The video processing method provided by this embodiment calculates optical flow information based on the current video frame, at least one previous video frame, and at least one subsequent video frame. This allows accurate determination of the motion information of the content displayed in the current video frame based on this optical flow information. Furthermore, based on this optical flow information, the location of a defect in an associated video frame that matches the current video frame can be determined. Furthermore, scratch repair can be performed on the current video frame based on the content displayed in the associated video frame, thereby improving the accuracy of video repair.

[0070] Furthermore, based on any of the above embodiments, step 103 includes:

[0071] Performing a screen alignment operation on the current video frame and the at least one associated video frame according to the optical flow information to obtain a plurality of aligned video frames.

[0072] The aligned multiple video frames are input into a preset defect repair model to obtain the first video frame.

[0073] In this embodiment, a defect repair model can be pre-set to implement defect repair operations on the current video frame. The defect repair model can be obtained by iteratively training based on preset training data until the model converges. The training data can include multiple training data pairs, each of which includes multiple aligned video frames with defects and multiple aligned video frames without defects. Through iterative training based on the training data, the defect repair model can accurately implement defect repair operations.

[0074] Due to changes in the content of the video to be processed, there will be slight differences between consecutive video frames (such as object displacement, changes in perspective, etc.). Therefore, after determining the optical flow information corresponding to the current video frame and at least one associated video frame, the current video frame and at least one associated video frame can be aligned based on the optical flow information to obtain multiple aligned video frames, so that the content / objects in the aligned multiple video frames are in the same position.

[0075] Furthermore, after completing the alignment operation on the multiple video frames, the aligned multiple video frames can be input into a preset defect repair model, which can implement a defect repair operation to obtain a first video frame that does not include defects.

[0076] The video processing method provided in this embodiment pre-sets a defect repair model, thereby accurately performing repair operations on multiple aligned video frames based on the defect repair model, thereby improving the efficiency and accuracy of video repair.

[0077] FIG4 is a flow chart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG4 , step 104 includes:

[0078] Step 401: Perform a color restoration operation on the first video frame to obtain a second video frame after color adjustment.

[0079] Step 402: Perform a resolution adjustment operation on the second video frame to obtain the target video frame.

[0080] In this embodiment, after repairing defects such as scratches and noise in the video to be processed, in order to improve the visual effect of the video to be processed, the color and resolution corresponding to the video to be processed may also be repaired.

[0081] Optionally, after obtaining the defect-free first video frame, a color restoration operation can be performed on the first video frame to obtain a color-adjusted second video frame. This second video frame can thus present a higher-quality, more aesthetically pleasing color effect. Furthermore, the resolution of the second video frame can be adjusted to obtain a clear, color-pleasing target video frame.

[0082] The video processing method provided in this embodiment performs a color restoration operation on the first video frame and a resolution adjustment operation on the second video frame after color restoration, thereby obtaining a target video frame with beautiful colors and high resolution, thereby improving the visual effect of the target video.

[0083] Further, based on any of the above embodiments, step 401 includes:

[0084] If the video to be processed is a color video, color parameters corresponding to the first video frame are identified, and a color adjustment operation is performed on the first video frame according to the color parameters and a preset color adjustment algorithm to obtain the second video frame.

[0085] If the video to be processed is a black and white video, the repaired video frame is input into a preset coloring model to obtain a color video frame, the color parameters corresponding to the color video frame are identified, and the color adjustment operation is performed on the color video frame according to the color parameters and a preset color adjustment algorithm to obtain the second video frame.

[0086] In this embodiment, the video to be processed can be a color video or a black and white video. According to different video types, different color restoration methods can be used to perform restoration operations on the video to be processed.

[0087] Optionally, when the video to be processed is a color video, it may fade over time, resulting in poor visual quality. Therefore, color parameters corresponding to the first video frame can be identified, where these color parameters include but are not limited to color, saturation, contrast, brightness, and light sensitivity. After determining these color parameters, a color adjustment operation can be performed on the first video frame based on the color parameters and a preset color adjustment algorithm to obtain a second video frame. The color adjustment algorithm can be a color adjustment model pre-trained based on a large amount of training data.

[0088] As an implementable method, multiple sets of more beautiful color matching information can be pre-set. After identifying the color information corresponding to the first video frame, the target color matching information closest to the color information can be determined in the preset color matching information, and the color adjustment operation can be performed on the first video frame based on the target color matching information. The present disclosure does not impose any restrictions on this.

[0089] Optionally, when the video to be processed is a black and white video, the black and white video to be processed can be first colorized. Then, a color restoration operation can be performed on the color video to be processed. The restored video frame can be input into a preset colorization model to obtain a color video frame, and the color parameters corresponding to the color video frame can be identified, wherein the color parameters include but are not limited to color, saturation, contrast, brightness, light sensitivity and other parameters. A color adjustment operation is performed on the color video frame according to the color parameters and a preset color adjustment algorithm to obtain a second video frame. The color adjustment scheme for the color video frame is the same as the color adjustment scheme for the first video frame, and will not be repeated here.

[0090] The video processing method provided in this embodiment performs color adjustment on a first video frame for color video, colorizes the first video frame for black-and-white video, and then performs color adjustment on the colored color video frame, thereby making the color effect of the color video more beautiful. Furthermore, the color effect of the color video can be further improved by adjusting the black-and-white video to color video.

[0091] Further, based on any of the above embodiments, step 402 includes:

[0092] The second video frame is super-resolved using a preset super-resolution algorithm to obtain a third video frame.

[0093] A target area in the third video frame is identified, and a restoration operation is performed on the target area using a preset image restoration algorithm to obtain the target video frame.

[0094] In this embodiment, the older video to be processed generally has a lower resolution, resulting in a poor viewing experience. Therefore, in order to further improve the restoration effect of the video to be processed, after obtaining the color-restored second video frame, the second video frame can be super-resolved using a preset super-resolution algorithm to obtain a third video frame.

[0095] Furthermore, the third video frame may include a target area, which may be a facial area, a body area, a text area, etc. In order to improve the display effect of the third video frame and reduce the computational complexity during the restoration process, a targeted restoration operation may be performed on the target area.

[0096] Optionally, a target area in the third video frame may be identified. Different recognition algorithms may be selected for different target areas. For example, when the target area is a facial area, a preset facial recognition model may be used to implement the recognition operation on the target area.

[0097] Furthermore, a preset image restoration algorithm can be used to restore the target region to obtain a target video frame. Continuing with the above example, after the facial region is identified, a preset facial restoration model can be used to restore the facial region.

[0098] The video processing method provided in this embodiment can further improve the content quality of the target video by performing super-resolution processing on the second video frame and performing target area optimization operation on the third video frame.

[0099] FIG5 is a flow chart of a video processing method according to another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG5 , step 104 includes:

[0100] Step 501: Combine the at least one target video frame according to timestamp information corresponding to the at least one target video frame to obtain a video to be processed.

[0101] Step 502: Determine the frame rate information corresponding to the video to be processed.

[0102] Step 503: If the frame rate information is lower than a preset frame rate threshold, and any two target video frames meet a preset condition, perform a frame insertion operation on the video to be processed to obtain the target video.

[0103] Frame rate is the frequency (rate) at which bitmap images, in units of frames, appear continuously on the display. A high frame rate can produce smoother and more realistic visual effects.

[0104] Taking the video to be processed as an old movie as an example, old movies generally have a low frame rate, resulting in a poor user experience during playback.

[0105] Therefore, after completing defect repair and visual effect repair of the current video frame, in order to further improve the video quality of the target video, a frame insertion operation can be performed to increase the frame rate of the target video.

[0106] Optionally, after completing the repair operation on each video frame corresponding to the to-be-processed video, at least one target video frame may be combined according to the timestamp information corresponding to the at least one target video frame to obtain the to-be-processed video having the same frame rate as the to-be-processed video.

[0107] In order to implement the repair operation on the video to be processed, the frame rate information corresponding to the video to be processed can be determined. Among them, the frame rate information of the video to be processed can be determined by any frame rate determination method, and this disclosure does not limit this.

[0108] Furthermore, if the frame rate information is lower than a preset frame rate threshold and any two target video frames meet a preset condition, a frame insertion operation is performed on the processed video to obtain the target video, where the preset condition may be that the inter-frame variation is less than a preset variation threshold.

[0109] Optionally, a pre-set frame insertion model can be used to automatically calculate optical flow information based on the two preceding and succeeding frames of the video to be processed, and then insert video frames between the two frames based on the optical flow information to implement the frame insertion operation. Alternatively, the frame insertion operation of the video to be processed can be implemented using any other frame insertion method, which is not limited by this disclosure.

[0110] The video processing method provided in this embodiment can improve the fluency of the target video by performing a frame insertion operation on the video to be processed, thereby further improving the content quality of the target video.

[0111] Figure 6 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure. As shown in Figure 6, the device includes: an acquisition module 61, a determination module 62, a repair module 63, and a processing module 64. Among them, the acquisition module 61 is used to acquire a video to be processed and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed. The determination module 62 is used to determine the motion information corresponding to the current video frame based on the current video frame and at least one associated video frame corresponding to the current video frame. The repair module 63 is used to perform a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame. The processing module 64 is used to perform visual effect repair on the first video frame according to a preset repair algorithm to obtain a target video frame, and obtain a repaired target video based on at least one target video frame.

[0112] Furthermore, based on any of the above embodiments, the defects to be repaired include scratch defects and noise defects. The acquisition module is configured to: input the current video frame into a preset defect recognition model, obtain a recognition result output by the defect recognition model, wherein the recognition result includes location information of the scratch defect and / or noise defect, and determine the recognition result as the at least one defect to be repaired.

[0113] Furthermore, based on any of the above embodiments, the determining module is configured to: determine, based on timestamp information corresponding to video frames in the to-be-processed video, at least one video frame before the current video frame and at least one video frame after the current video frame as the associated video frames; calculate optical flow information corresponding to the current video frame based on the current video frame and the at least one associated video frame, and determine the optical flow information as the motion information.

[0114] Furthermore, based on any of the above embodiments, the repair module is configured to: perform an image alignment operation on the current video frame and the at least one associated video frame based on the optical flow information to obtain a plurality of aligned video frames, and input the plurality of aligned video frames into a preset defect repair model to obtain the first video frame.

[0115] Furthermore, based on any of the above embodiments, the processing module is configured to: perform a color restoration operation on the first video frame to obtain a second video frame after color adjustment; and perform a resolution adjustment operation on the second video frame to obtain the target video frame.

[0116] Furthermore, based on any of the above embodiments, the processing module is configured to: if the video to be processed is a color video, identify color parameters corresponding to the first video frame, perform color adjustment on the first video frame based on the color parameters and a preset color adjustment algorithm to obtain the second video frame. If the video to be processed is a black and white video, input the restored video frame into a preset colorization model to obtain a color video frame, identify color parameters corresponding to the color video frame, perform color adjustment on the color video frame based on the color parameters and a preset color adjustment algorithm to obtain the second video frame.

[0117] Furthermore, based on any of the above embodiments, the processing module is configured to: perform super-resolution processing on the second video frame using a preset super-resolution algorithm to obtain a third video frame; identify a target area in the third video frame, and perform a restoration operation on the target area using a preset image restoration algorithm to obtain the target video frame.

[0118] Furthermore, based on any of the above embodiments, the processing module is configured to: combine the at least one target video frame according to the timestamp information corresponding to the at least one target video frame to obtain a video to be processed; determine frame rate information corresponding to the video to be processed; and if the frame rate information is lower than a preset frame rate threshold and any two target video frames meet a preset condition, perform a frame interpolation operation on the video to be processed to obtain the target video.

[0119] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0120] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video processing method described in any of the above embodiments is implemented.

[0121] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the video processing method as described in any of the above embodiments when executed by a processor.

[0122] In order to implement the above embodiment, the present disclosure further provides an electronic device, including: a processor and a memory;

[0123] The memory stores computer-executable instructions;

[0124] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method as described in any of the above embodiments.

[0125] FIG7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device 700 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG7 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0126] As shown in FIG7 , the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0127] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 7 shows an electronic device 700 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0128] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0129] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0130] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0131] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0132] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0135] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0136] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0138] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0139] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, comprising: Acquire a video to be processed, and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed; Determine motion information corresponding to the current video frame according to the current video frame and at least one associated video frame corresponding to the current video frame; Performing a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame; Perform visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame, and obtain a restored target video based on at least one target video frame.

2. The method according to claim 1, wherein: The defects to be repaired include scratch defects and noise defects; The identifying at least one defect to be repaired corresponding to the current video frame in the video to be processed includes: Inputting the current video frame into a preset defect recognition model to obtain a recognition result output by the defect recognition model, wherein the recognition result includes location information of a scratch defect and / or a noise defect; The identification result is determined as the at least one defect to be repaired.

3. The method according to claim 1 or 2, wherein: The determining, according to the current video frame and at least one associated video frame corresponding to the current video frame, motion information corresponding to the current video frame comprises: Determine, according to timestamp information corresponding to video frames in the video to be processed, at least one video frame before the current video frame and at least one video frame after the current video frame as the associated video frame; Optical flow information corresponding to the current video frame is calculated based on the current video frame and the at least one associated video frame, and the optical flow information is determined as the motion information.

4. The method according to claim 3, wherein: The performing a repair operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame includes: Performing a screen alignment operation on the current video frame and the at least one associated video frame according to the optical flow information to obtain a plurality of aligned video frames; The aligned multiple video frames are input into a preset defect repair model to obtain the first video frame.

5. The method according to any one of claims 1 to 4, wherein: The performing visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame includes: Performing a color restoration operation on the first video frame to obtain a second video frame after color adjustment; A resolution adjustment operation is performed on the second video frame to obtain the target video frame.

6. The method according to claim 5, wherein: The performing a color restoration operation on the restored video frame to obtain a second video frame after color adjustment includes: If the video to be processed is a color video, identifying color parameters corresponding to the first video frame, performing a color adjustment operation on the first video frame according to the color parameters and a preset color adjustment algorithm, and obtaining the second video frame; If the video to be processed is a black and white video, the repaired video frame is input into a preset coloring model to obtain a color video frame, the color parameters corresponding to the color video frame are identified, and the color adjustment operation is performed on the color video frame according to the color parameters and a preset color adjustment algorithm to obtain the second video frame.

7. The method according to claim 5 or 6, wherein: The performing a resolution adjustment operation on the second video frame to obtain the target video frame includes: Performing super-resolution processing on the second video frame by using a preset super-resolution algorithm to obtain a third video frame; A target area in the third video frame is identified, and a restoration operation is performed on the target area using a preset image restoration algorithm to obtain the target video frame.

8. The method according to any one of claims 1 to 7, wherein: The step of obtaining a restored target video based on at least one target video frame comprises: Combining the at least one target video frame according to the timestamp information corresponding to the at least one target video frame to obtain a video to be processed; Determine the frame rate information corresponding to the video to be processed; If the frame rate information is lower than a preset frame rate threshold, and any two target video frames meet a preset condition, a frame insertion operation is performed on the video to be processed to obtain the target video.

9. A video processing device, comprising: An acquisition module is configured to acquire a video to be processed and identify at least one defect to be repaired corresponding to a current video frame in the video to be processed; a determination module, configured to determine motion information corresponding to the current video frame according to the current video frame and at least one associated video frame corresponding to the current video frame; a repairing module configured to perform a repairing operation on at least one defect to be repaired corresponding to the current video frame based on the motion information to obtain a repaired first video frame; The processing module is configured to perform visual effect restoration on the first video frame according to a preset restoration algorithm to obtain a target video frame, and obtain a restored target video based on at least one target video frame.

10. An electronic device comprising a processor and a memory, wherein: The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions, wherein: When the processor executes the computer-executable instruction, the video processing method according to any one of claims 1 to 8 is implemented.

Images