Object repair method, repair evaluation method, electronic device and storage medium

The object repair method addresses inefficiencies in traditional and AI-based video restoration by pre-repairing and fusing objects with similarity and inter-frame correlation weights, enhancing restoration quality and efficiency.

CN114913468BActive Publication Date: 2024-12-03ALIBABA (CHINA) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210681242.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-16
Publication Date
2024-12-03
Estimated Expiration
2042-06-16

AI Technical Summary

Technical Problem

Traditional manual video restoration methods are inefficient and costly, while existing automated methods using artificial intelligence do not achieve ideal restoration effects in video frames.

Method used

An object repair method involving pre-repairing candidate objects, selecting objects based on similarity thresholds, and fusing them with pre-repaired objects using fusion weights and inter-frame correlation weights to improve restoration quality.

Benefits of technology

The method enhances video frame restoration by avoiding distortion and improving efficiency through precise object selection and fusion techniques, resulting in higher quality restored video frames.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114913468B_ABST
    Figure CN114913468B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide an object repair method, a repair evaluation method, an electronic device, and a storage medium. The object repair method includes: pre-repairing the candidate objects to be repaired in the video frame to obtain the pre-repaired objects of the candidate objects to be repaired; selecting the object to be repaired from the candidate objects to be repaired; fusing the object to be repaired and its pre-repaired object to obtain the repaired object. In the scheme of the embodiments of the present invention, the repaired object is obtained by fusing the object to be repaired and its pre-repaired object, which avoids the video frame distortion caused by some pre-repaired objects and improves the video frame repair effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to an object repair method, a repair evaluation method, an electronic device, and a storage medium. Background Art

[0002] As cloud computing technology continues to develop towards cloud native, cloud service providers not only provide computing, storage, network, security and other basic services (Infrastructure as a Service, IaaS), but also provide database, big data processing, video cloud and other platform services (Platform as a Service, PaaS). Video restoration is one of the video processing functions provided by video cloud. For film and television works, through video restoration, high-quality video data can be obtained, which not only ensures the user's viewing experience, but also saves the huge cost of remaking film and television works.

[0003] In the process of video restoration, determining the object to be restored in the video frame is an important step. In traditional restoration methods, manual restoration is usually performed by professionals, which has low restoration efficiency and high cost.

[0004] With the development of artificial intelligence technology represented by deep learning, automated restoration methods based on artificial intelligence technology have achieved lower restoration costs and higher processing efficiency. However, the video frame restoration effect is still not ideal. Summary of the invention

[0005] In view of this, embodiments of the present invention provide an object repair method, a repair evaluation method, an electronic device, and a storage medium to at least partially solve the above problems.

[0006] According to a first aspect of an embodiment of the present invention, there is provided an object repair method, comprising: pre-repairing candidate objects to be repaired in a video frame to obtain a pre-repaired object of the candidate objects to be repaired; selecting an object to be repaired from the candidate objects to be repaired; and fusing the object to be repaired and its pre-repaired object to obtain a repaired object.

[0007] In another implementation of the present invention, the method of obtaining the repaired object by fusing the object to be repaired and the pre-repaired object thereof comprises: fusing the object to be repaired and the pre-repaired object of the video frame based on a fusion weight to obtain the repaired object, wherein the fusion weight is determined based on an initial fusion weight and a smoothing weight.

[0008] In another implementation of the present invention, the method further comprises: determining the smoothing weight based on at least one of an inter-frame correlation weight between a previous video frame of the video frame and the video frame, and an inter-frame distortion smoothing weight.

[0009] In another implementation of the present invention, the method further includes: determining an inter-frame distortion smoothing weight of the video frame based on a similarity between the object to be repaired in the video frame and its pre-repaired object.

[0010] In another implementation of the present invention, the inter-frame distortion smoothing weight of the video frame is determined based on the similarity between the object to be repaired in the video frame and the pre-repaired object, including: when the similarity between the object to be repaired in the video frame and the pre-repaired object is greater than a first similarity, the inter-frame distortion smoothing weight of the video frame is determined as a first inter-frame distortion smoothing weight threshold; when the similarity between the object to be repaired in the video frame and the pre-repaired object is between the first similarity and a second similarity, the inter-frame distortion smoothing weight of the video frame is determined to be positively correlated with the similarity; when the similarity between the object to be repaired in the video frame and the pre-repaired object is less than the second similarity, the inter-frame distortion smoothing weight of the video frame is determined as a second inter-frame distortion smoothing weight threshold, and the value range of the similarity is between the first inter-frame distortion smoothing weight threshold and the second inter-frame distortion smoothing weight threshold.

[0011] In another implementation of the present invention, the method further includes: determining the inter-frame correlation weight based on the weight of the candidate object to be repaired determined as the object to be repaired in each previous video frame of the video frame, and the each previous video frame is stored in a buffer corresponding to the video frame.

[0012] In another implementation of the present invention, the method further includes: determining each initial fusion weight between the object to be repaired and the pre-repaired object of each previous video frame based on the discrete degree of the inter-frame correlation weight.

[0013] According to a second aspect of an embodiment of the present invention, a repair evaluation method is provided, comprising: pre-repairing a candidate object to be repaired in a video frame to obtain a pre-repaired object of the candidate object to be repaired; determining the similarity between the candidate object to be repaired and its pre-repaired object; and determining the candidate object to be repaired whose similarity meets a preset similarity threshold as the object to be repaired in the video frame.

[0014] In another implementation of the present invention, the method further includes: determining an inter-frame distortion smoothing weight corresponding to the video frame, a larger inter-frame distortion smoothing weight indicates a larger inter-frame difference between the video frame and an adjacent video frame; and determining a preset similarity threshold that is positively correlated with the inter-frame distortion smoothing weight.

[0015] In another implementation of the present invention, the inter-frame distortion smoothing weight represents the degree of discreteness of pixel channel values ​​between the video frame and an adjacent video frame. The determining of the similarity between the candidate object to be repaired and the pre-repaired object comprises: determining the degree of discreteness between the pixel channel value of the candidate object to be repaired and the pixel channel value of the pre-repaired object as the similarity.

[0016] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect or the second aspect.

[0017] According to a fourth aspect of an embodiment of the present invention, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0018] In the object restoration method of the embodiment of the present invention, the restoration object is obtained by fusing the object to be restored and its pre-restoration object, thereby avoiding video frame distortion caused by some pre-restoration objects and improving the video frame restoration effect.

[0019] In the restoration evaluation method of the embodiment of the present invention, the candidate objects to be restored whose similarity meets the preset similarity threshold are determined as the objects to be restored, so that the degree of distortion of the restoration result of the objects to be restored is low. In other words, the candidate objects to be restored that do not meet the preset similarity threshold are not restored, thereby improving the video frame restoration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 is a schematic block diagram of a video restoration process according to an example.

[0022] Figure 2 The figure is a flowchart of the steps of an object repair method according to an embodiment of the present invention.

[0023] Figure 3 The figure is a flowchart of the steps of a repair assessment method according to an embodiment of the present invention.

[0024] Figure 4 For combination Figure 2 and Figure 3 A schematic block diagram of a video repair process of an embodiment.

[0025] Figure 5 FIG. 4 is a schematic structural diagram of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0027] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0028] Figure 1 1 is a schematic block diagram of a video repair process according to an example. In this example, the video repair process is processed based on the video frame extraction module 110, the video frame repair module 120 and the video synthesis module 130 in sequence. Specifically, the video data can be first input into the video frame extraction module 110, and each video frame data can be extracted based on the time series. Then, each video frame is input into the video frame repair module 120 for video frame repair to obtain a repaired video frame, and then the repaired video frame is input into the video synthesis module 130, and based on the inter-frame relationship of each video frame, each repaired video frame is synthesized to obtain the repaired video data.

[0029] For example, the video frame restoration module 120 may be based on a video frame restoration model such as a convolutional neural network (CNN). The video frame restoration model is obtained through supervised learning or unsupervised learning, which is beneficial to improving the efficiency of video frame restoration.

[0030] Figure 2 The solution of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. For example, the electronic device can be configured as Figure 1 The video frame restoration module 120 is configured to perform object restoration and perform video restoration based on the object restoration.

[0031] The object repair method of this embodiment includes:

[0032] S210: Pre-repairing the candidate object to be repaired in the video frame to obtain a pre-repaired object of the candidate object to be repaired.

[0033] For example, the candidate objects to be repaired of the video frame may be objects in the video frame, such as characters, objects, scenes, backgrounds, etc. Specifically, in video repair of film and television works, the candidate objects to be repaired include at least facial features of characters. For another example, the objects to be repaired may be edge pixels of an object in a video frame, that is, pixels at the boundary between object pixels and background pixels.

[0034] For example, the video frame may include at least one candidate object to be repaired, or may not include any candidate object to be repaired. In addition, the pre-repair may be performed using a pre-trained video frame repair model. The video frame repair model may be trained based on video frame samples and labels, which is beneficial for the video frame repair model to learn context information.

[0035] S220: Selecting an object to be repaired from among the candidate objects to be repaired.

[0036] For example, at least part of the candidate objects to be repaired are objects to be repaired, for example, the candidate objects to be repaired include the object to be repaired and other objects. The object to be repaired can be selected from the candidate objects to be repaired based on a preset similarity threshold. In one example, the similarity between the object to be repaired and its pre-repaired object is greater than the preset similarity threshold, and the similarity between the other objects and their pre-repaired objects is less than the preset similarity threshold.

[0037] S230: Fusing the to-be-repaired object and the pre-repaired object to obtain a repaired object.

[0038] For example, the object to be repaired and its pre-repaired object may be fused based on the fusion weight. In the fusion process, the weight of the object to be repaired is inversely correlated with the similarity (i.e., the similarity between the object to be repaired and its pre-repaired object), and the weight of the pre-repaired object is positively correlated with the similarity (i.e., the similarity between the object to be repaired and its pre-repaired object).

[0039] In the object restoration method of the embodiment of the present invention, the restoration object is obtained by fusing the object to be restored and its pre-restoration object, thereby avoiding video frame distortion caused by some pre-restoration objects and improving the video frame restoration effect.

[0040] In other examples, the restoration object is obtained by fusing the object to be restored and the pre-restoration object thereof, including: based on a fusion weight, the object to be restored and the pre-restoration object of the video frame are fused to obtain the restoration object, wherein the fusion weight is determined by weighting the initial fusion weight and the smoothing weight. The fusion weight takes into account the initial fusion weight and the smoothing weight, so that the smoothing weight can be separated out for performing inter-frame smoothing, and the initial fusion weight is separated out to ensure the restoration effect of the object to be restored within the video frame, thereby improving the efficiency of video restoration.

[0041] In some other examples, the video restoration method further includes determining a smoothing weight based on at least one of an inter-frame correlation weight between a previous video frame of the video frame and the video frame, and an inter-frame distortion smoothing weight.

[0042] For example, the smoothing weight is determined based on the inter-frame correlation weight between the previous video frame and the video frame. More specifically, W his =W prior ⅹW scale .W his represents the fusion weight considering the inter-frame correlation weight, W prior represents the initial fusion weight, W scale represents the smoothing weight.

[0043] For another example, based on both the inter-frame distortion and the smoothing weight, the smoothing weight is determined. More specifically, W sim =W prior ⅹW scale .W sim represents the fusion weight considering the inter-frame distortion smoothing weight, W prior represents the initial fusion weight, W scale represents the smoothing weight.

[0044] More generally, the smoothing weight is determined based on both the inter-frame correlation weight between the previous video frame and the video frame, and the inter-frame distortion smoothing weight. Specifically, the smoothing weight W=W his ⅹW sim .

[0045] The inter-frame correlation weight reflects the pixel position correlation of the object to be repaired between different video frames with a time series relationship, and can reliably reflect the inter-frame smoothness. The object to be repaired itself has low distortion and high smoothness in different video frames. The inter-frame distortion smoothness weight indirectly reflects the distortion and smoothness of the repaired object through the degree of repair of the object to be repaired.

[0046] In some other examples, the video restoration method further includes: determining an inter-frame distortion smoothing weight of the video frame based on a similarity between an object to be restored in the video frame and a pre-restoration object thereof.

[0047] In other words, the inter-frame distortion smoothing weight indirectly reflects the degree of distortion and smoothness of the repaired object through the degree of repair of the object to be repaired. For example, the inter-frame distortion smoothing weight can be determined between a first inter-frame distortion smoothing weight threshold and a second inter-frame distortion smoothing weight threshold through multiple similarity intervals (e.g., an interval greater than the first similarity, an interval between the first similarity and the second similarity, and an interval less than the second similarity), thereby efficiently and reliably obtaining the inter-frame distortion smoothing weight.

[0048] Furthermore, in order to determine the inter-frame distortion smoothing weight of the video frame based on the similarity between the object to be repaired in the video frame and the pre-repaired object, when the similarity between the object to be repaired in the video frame and the pre-repaired object is greater than a first similarity, the inter-frame distortion smoothing weight of the video frame is determined as a first inter-frame distortion smoothing weight threshold; when the similarity between the object to be repaired in the video frame and the pre-repaired object is between the first similarity and the second similarity, the inter-frame distortion smoothing weight of the video frame is determined to be positively correlated with the similarity; when the similarity between the object to be repaired in the video frame and the pre-repaired object is less than the second similarity, the inter-frame distortion smoothing weight of the video frame is determined as a second inter-frame distortion smoothing weight threshold, and the value range of the similarity is between the first inter-frame distortion smoothing weight threshold and the second inter-frame distortion smoothing weight threshold.

[0049] More specifically, the inter-frame distortion smoothing weight scale(s) represents a function of the similarity s:

[0050]

[0051] In other examples, the video restoration method also includes: determining the inter-frame correlation weights based on the degree of discreteness between each initial fusion weight between the object to be restored and the pre-repaired object in each previous video frame. This example takes into account the degree of discreteness between each initial fusion weight to determine the inter-frame correlation weights, thereby avoiding excessive discreteness of each initial fusion weight and improving the reliability of inter-frame smoothing.

[0052] Alternatively, the inter-frame correlation weight is determined based on the weight of the candidate to be repaired object determined as the object to be repaired in each previous video frame of the video frame, and each previous video frame is stored in a buffer corresponding to the video frame. The reference value of the previous video frame that is far away from the video frame in time is lower than the reference value of the previous video frame that is closer, and the previous video frames stored in the buffer improve the calculation efficiency. In addition, based on the weight of the candidate to be repaired object determined as the object to be repaired in each previous video frame of the video frame, the inter-frame correlation weight is determined, which improves the calculation efficiency. For example, for a previous video frame, if the candidate to be repaired object is determined as the object to be repaired, the mark value of the previous video frame can be 1, and if the candidate to be repaired object is not determined as the object to be repaired, the mark value of the previous video frame is 0. More generally, the mark value when the candidate to be repaired object is determined as the object to be repaired is greater than the mark value when the candidate to be repaired object is not determined as the object to be repaired. Correspondingly, the above weight can be the average of the mark values ​​of each previous video frame. For example, the mark values ​​of the previous 5 video frames are 1, 0, 1, 1 and 1 respectively, then the above weight = (1+0+1+1+1) / 5 = 0.8, that is, the inter-frame correlation weight of the previous 5 video frames and the video frame (current video frame) is 0.8.

[0053] In other examples, the video restoration method also includes: based on the discrete degree of inter-frame correlation weights, determining the initial fusion weights between the object to be restored and the pre-repaired object of each previous video frame. This example takes into account the discrete degree of inter-frame correlation weights to determine the initial fusion weights, thereby avoiding excessive discreteness of the initial fusion weights and improving the reliability of inter-frame smoothing.

[0054] In other examples, the discreteness between the initial fusion weights is smaller than the discreteness of the inter-frame correlation weights. Compared with the initial fusion weights, the inter-frame correlation weights can better ensure the inter-frame smoothing effect, avoiding the intra-frame repair effect at the expense of too many inter-frame effects.

[0055] Figure 3 The solution of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. For example, the electronic device can be configured as Figure 1 The video frame restoration module 120 is used to perform restoration evaluation and perform object restoration based on the evaluated object to be restored. Further, the restoration evaluation method of the embodiment of the present embodiment can be used as an example of selecting an object to be restored from candidate objects to be restored (S220).

[0056] The repair assessment method of this embodiment includes:

[0057] S310: Pre-repairing the candidate object to be repaired in the video frame to obtain a pre-repaired object of the candidate object to be repaired.

[0058] For example, the candidate objects to be repaired of the video frame may be objects in the video frame, such as characters, objects, scenes, backgrounds, etc. Specifically, in video repair of film and television works, the candidate objects to be repaired include at least facial features of characters.

[0059] The video frame may include at least one candidate object to be repaired, or may not include any candidate object to be repaired. In addition, the pre-repair may be performed using a pre-trained video frame repair model. The video frame repair model may be trained based on video frame samples and labels, which is beneficial for the video frame repair model to learn context information.

[0060] S320: Determine the similarity between the candidate to-be-repaired object and the pre-repaired object.

[0061] For example, the similarity here may be the cosine similarity, the Pierce coefficient, etc. between the candidate object to be repaired and the pre-repaired object. The similarity may be represented by the distance between the candidate object to be repaired and the pre-repaired object in the object vector space.

[0062] As an example, when determining the similarity between an alternative object to be repaired and its pre-repaired object, it can also be performed based on a pre-trained similarity prediction model. The similarity prediction model can be trained based on video frame object samples and their labels. For example, the video frame object samples can be partial video frames. In other words, the video frame object samples are located in the area where the object is located in the video frame. Therefore, compared with the video frame repair model, the similarity prediction model includes less contextual information, which is conducive to improving the reliability of the prediction.

[0063] As another example, the video frame restoration model and the similarity prediction model may both use video frame samples or video frame object samples, except that the label confidence of the video frame samples or video frame object samples in the case of training the video frame restoration model is greater than the label confidence of the video frame samples or video frame object samples in the case of training the similarity prediction model.

[0064] S330: Determine candidate objects to be repaired whose similarity meets a preset similarity condition as the objects to be repaired in the video frame.

[0065] For example, the preset similarity condition may be a similarity threshold, and the candidate objects to be repaired having a similarity greater than the similarity threshold are used as the objects to be repaired in the video frame.

[0066] In the restoration evaluation method of the embodiment of the present invention, the candidate objects to be restored whose similarity meets the preset similarity threshold are determined as the objects to be restored, so that the degree of distortion of the restoration result of the objects to be restored is low. In other words, the candidate objects to be restored that do not meet the preset similarity threshold are not restored, thereby improving the evaluation accuracy and further improving the video frame restoration effect.

[0067] In other examples, the restoration evaluation method also includes: determining an inter-frame distortion smoothing weight corresponding to a video frame, wherein a larger inter-frame distortion smoothing weight indicates a larger inter-frame difference between the video frame and adjacent video frames; and determining a preset similarity threshold that is positively correlated with the inter-frame distortion smoothing weight.

[0068] The larger the inter-frame distortion smoothing weight is, the greater the inter-frame difference between the video frame and the adjacent video frame is, which is conducive to determining the accuracy of the preset similarity threshold and further improving the evaluation accuracy of the object to be repaired. In addition, there is no need to determine the preset similarity threshold based on prior data between the candidate object to be repaired and the pre-repaired object.

[0069] Further,

[0070] In some other examples, the inter-frame distortion smoothing weight represents the discrete degree of pixel channel values ​​between a video frame and an adjacent video frame. Determining the similarity between the candidate to-be-repaired object and its pre-repaired object includes: determining the discrete degree between the pixel channel values ​​of the candidate to-be-repaired object and the pixel channel values ​​of the pre-repaired object as the similarity.

[0071] The preset similarity threshold is positively correlated with the inter-frame distortion smoothing weight, and the discrete degree between the pixel channel value of the candidate object to be repaired and the pixel channel value of the pre-repaired object is used as the similarity, which further improves the evaluation accuracy of the object to be repaired.

[0072] Further,

[0073] Figure 4 For combination Figure 2 and Figure 3 A schematic block diagram of a video repair process of an embodiment. Figure 4 In the video restoration process shown, processing is performed based on a video frame extraction module 410 , a pre-restoration module 421 , a determination module 422 , a restoration module 423 and a video synthesis module 430 .

[0074] First, the video data is input into the video frame extraction module 410, and each video frame data is extracted based on the time series.

[0075] Then, each video frame data is input to the pre-repair module 421 to perform pre-repair on the candidate to-be-repaired object in each video frame to obtain a pre-repaired object.

[0076] Then, each candidate object to be repaired and the pre-repaired object is input into the determination module 422 to perform similarity calculation to determine the object to be repaired.

[0077] Then, the video frame is input to the repair module 423, and the object to be repaired is repaired to obtain a repaired video frame.

[0078] Then, the restored video frames are input into the video synthesis module 430, and based on the inter-frame relationship of each video frame, each restored video frame is synthesized to obtain restored video data.

[0079] The following will be combined Figure 5 describe Figure 2 and Figure 3 The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0080] like Figure 5 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 storing a program 510 , and a communication bus 508 .

[0081] The processor, the communication interface, and the memory communicate with each other via a communication bus.

[0082] Communication interface, used to communicate with other electronic devices or servers.

[0083] The processor is used to execute the program, and specifically can execute the relevant steps in the above method embodiment.

[0084] Specifically, the program may include program codes including computer operation instructions.

[0085] The processor may be a CPU, or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0086] The memory is used to store programs. The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0087] The program can be specifically used to enable the processor to perform the following operations: pre-repair the candidate objects to be repaired in the video frame to obtain the pre-repaired objects of the candidate objects to be repaired; select the object to be repaired from the candidate objects to be repaired; and fuse the object to be repaired and its pre-repaired object to obtain the repaired object.

[0088] Alternatively, the program can be specifically used to enable the processor to perform the following operations: pre-repair the alternative object to be repaired in the video frame to obtain a pre-repaired object of the alternative object to be repaired; determine the similarity between the alternative object to be repaired and its pre-repaired object; determine the alternative object to be repaired whose similarity meets a preset similarity threshold as the object to be repaired in the video frame.

[0089] In addition, the specific implementation of each step in the program can refer to the corresponding description of the corresponding steps and units in the above method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0090] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0091] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0092] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.

[0093] The above implementation methods are only used to illustrate the embodiments of the present invention, and are not limitations of the embodiments of the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A method for repairing an object, comprising: Pre-repairing the candidate object to be repaired in the video frame to obtain a pre-repaired object of the candidate object to be repaired; Selecting an object to be repaired from among the candidate objects to be repaired; Based on the fusion weight, the object to be repaired and the pre-repaired object of the video frame are fused to obtain the repaired object, and the fusion weight is determined based on the initial fusion weight and the smoothing weight.

2. The method according to claim 1, wherein: The method further comprises: The smoothing weight is determined based on at least one of an inter-frame correlation weight between a previous video frame of the video frame and the video frame, and an inter-frame distortion smoothing weight.

3. The method according to claim 2, wherein: The method further comprises: Based on the similarity between the to-be-repaired object of the video frame and the pre-repaired object, an inter-frame distortion smoothing weight of the video frame is determined.

4. The method according to claim 3, wherein: The determining of the inter-frame distortion smoothing weight of the video frame based on the similarity between the object to be repaired and the pre-repaired object of the video frame comprises: When the similarity between the to-be-repaired object and the pre-repaired object of the video frame is greater than a first similarity, determining the inter-frame distortion smoothing weight of the video frame as a first inter-frame distortion smoothing weight threshold; When the similarity between the to-be-repaired object and the pre-repaired object of the video frame is between the first similarity and the second similarity, determining the inter-frame distortion smoothing weight of the video frame to be positively correlated with the similarity; When the similarity between the object to be repaired and the pre-repaired object of the video frame is less than the second similarity, the inter-frame distortion smoothing weight of the video frame is determined as the second inter-frame distortion smoothing weight threshold, and the value range of the similarity is between the first inter-frame distortion smoothing weight threshold and the second inter-frame distortion smoothing weight threshold.

5. The method according to claim 2, wherein: The method further comprises: The inter-frame correlation weight is determined based on the weight of the candidate to-be-repaired object determined as the to-be-repaired object in each previous video frame of the video frame, and the each previous video frame is stored in a buffer corresponding to the video frame.

6. The method according to claim 2, wherein: The method further comprises: Based on the discrete degree of the inter-frame correlation weights, each initial fusion weight between the object to be restored and the pre-restored object of each previous video frame is determined.

7. A restoration assessment method comprising: Pre-repairing the candidate object to be repaired in the video frame to obtain a pre-repaired object of the candidate object to be repaired; Determining the similarity between the candidate to-be-repaired object and the pre-repaired object; Determine a candidate to-be-repaired object whose similarity meets a preset similarity threshold as the to-be-repaired object in the video frame; The method further comprises: Determine an inter-frame distortion smoothing weight corresponding to the video frame, wherein a larger inter-frame distortion smoothing weight indicates a larger inter-frame difference between the video frame and an adjacent video frame; A preset similarity threshold value positively correlated with the inter-frame distortion smoothing weight is determined.

8. The method according to claim 7, wherein: The inter-frame distortion smoothing weight represents the discrete degree of pixel channel values ​​between the video frame and adjacent video frames; The determining of the similarity between the candidate to-be-repaired object and the pre-repaired object includes: The degree of discreteness between the pixel channel values ​​of the candidate to-be-repaired object and the pixel channel values ​​of the pre-repaired object is determined as the similarity.

9. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 8.

10. A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.