An image inpainting method adaptive to blur type

Abstract

The application discloses a kind of fuzzy type self-adapting image restoration methods, belong to image processing and computer vision field, comprising: frame extraction processing is carried out to input video stream, and the image frame extracted is preprocessed;Based on preprocessed image, through multiple feature fuzzy evaluation mechanism, integrated fuzzy score is calculated;According to integrated fuzzy score, whether image frame is fuzzy frame is judged;If image frame is fuzzy frame, then according to multiple evaluation scores in multiple feature fuzzy evaluation mechanism, the fuzzy type of image frame is discriminated by preset rule;From multiple preset repair strategies, a corresponding repair strategy is selected to repair image frame, and a repaired image is generated;The quality of the repaired image is evaluated, and if the evaluation result does not meet the preset condition, the repair strategy is adjusted and re-repaired;The repaired image that meets the preset condition is re-inserted into the input video stream, and the repaired video is output.The application realizes the efficient identification and accurate repair of fuzzy image frame.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing and computer vision, and particularly relates to an image inpainting method adaptive to blur types. BACKGROUND

[0002] In the field of image processing and computer vision, image deblurring and image enhancement have always been the focus of research. Existing technologies have proposed various methods for improving image quality, such as blind deconvolution-based restoration methods, frequency-domain enhancement processing algorithms, and end-to-end blur image reconstruction techniques using generative adversarial networks (GAN). These methods can improve the visual quality of images or enhance their structural information to some extent, and are widely used in the processing of static images.

[0003] However, existing technologies still have obvious limitations in dealing with application requirements in actual video monitoring and industrial vision systems. On the one hand, most methods rely on a single type of image feature (such as edge sharpness or frequency energy) for blur discrimination, resulting in evaluation results that are easily disturbed by image content and poor stability. On the other hand, there is a lack of automatic identification and differentiation of blur types (such as motion blur, out-of-focus blur, and mixed blur), making it difficult to select a matching repair strategy according to the specific blur cause. In addition, traditional methods mostly use fixed repair models that cannot be dynamically adjusted according to image quality feedback, and also do not fully consider the temporal consistency and batch processing requirements between video frames, limiting their practicality and expandability in real-time video systems. SUMMARY

[0004] To solve the above technical problems, the present application proposes an image inpainting method adaptive to blur types to solve the problems existing in the prior art.

[0005] To achieve the above purpose, in a first aspect, the present application provides an image inpainting method adaptive to blur types, comprising:

[0006] frame extraction processing is performed on the input video stream, and the extracted image frames are preprocessed to obtain grayscale images;

[0007] Based on the grayscale images, a comprehensive blur score is calculated through a multi-feature blur evaluation mechanism; whether the image frame is a blurred frame is determined according to the comprehensive blur score;

[0008] If the image frame is a blurred frame, the blur type of the image frame is determined through a preset rule according to the multiple evaluation scores in the multi-feature blur evaluation mechanism;

[0009] According to the determined blur type, a corresponding repair strategy is selected from multiple preset repair strategies to repair the image frame and generate a repaired image;

[0010] performing quality evaluation on the repaired image, and adjusting a repair strategy and repairing again if the evaluation result does not satisfy a preset condition;

[0011] reinserting the repaired image satisfying the preset condition into the input video stream, and outputting the repaired video.

[0012] Preferably, the process of performing frame extraction on the input video stream and pre-processing the extracted image frames comprises:

[0013] periodically extracting frames from the input video stream according to a preset frame interval parameter to obtain a set of image frames;

[0014] scaling each image frame in the set of image frames to a fixed size;

[0015] converting the scaled image into a grayscale image.

[0016] Preferably, the process of calculating a comprehensive blur score through a multi-feature blur evaluation mechanism comprises:

[0017] calculating an edge sharpness index of the grayscale image to obtain a first evaluation score;

[0018] calculating a frequency domain entropy value index of the grayscale image to obtain a second evaluation score;

[0019] processing the grayscale image through a lightweight blur recognition network to obtain a third evaluation score;

[0020] performing weighted summation on the first evaluation score, the second evaluation score and the third evaluation score to obtain a comprehensive blur score.

[0021] Preferably, the process of discriminating the blur type of the image frame through a preset rule comprises:

[0022] combining the first evaluation score, the second evaluation score and the third evaluation score into a feature vector;

[0023] matching the feature vector with a preset blur type discrimination rule to discriminate the blur type of the image frame;

[0024] The blur type includes defocus blur, motion blur and mixed blur.

[0025] Preferably, the plurality of preset repair strategies include a first strategy, a second strategy and a third strategy.

[0026] The process of selecting a corresponding repair strategy from the plurality of preset repair strategies according to the discriminated blur type comprises:

[0027] If the blur type is defocus blur, the first strategy is selected, the first strategy being a reconstruction strategy based on a U-Net structure;

[0028] If the blur type is motion blur, the second strategy is selected, the second strategy being a strategy based on maximum a posteriori deconvolution;

[0029] If the blur type is mixed blur, the third strategy is selected, the third strategy being a composite repair strategy combining a frequency domain branch and a spatial domain branch.

[0030] Preferably, the determination process that the evaluation result does not satisfy the preset condition comprises:

[0031] The structural similarity index and the peak signal-to-noise ratio between the repaired image and the original image are calculated;

[0032] If the structural similarity index or the peak signal-to-noise ratio is lower than a preset threshold value, it is determined that the evaluation result does not satisfy the preset condition.

[0033] Preferably, in the frame extraction processing, the original timestamp information of each extracted image frame in the input video stream is recorded;

[0034] The repaired image is re-inserted into the input video stream in sequence according to the original timestamp information.

[0035] In a second aspect, the application further provides an image repair system with adaptive blur type, for implementing the method of the first aspect, comprising:

[0036] A video frame extraction and image preprocessing module is configured to perform frame extraction processing on an input video stream and to perform preprocessing on the extracted image frames;

[0037] A blur detection module is configured to calculate a comprehensive blur score based on the preprocessed image frames through a multi-feature blur evaluation mechanism, and to determine whether the image frames are blurred frames according to the comprehensive blur score;

[0038] A blur type discrimination module is configured to, if the image frames are blurred frames, discriminate the blur type of the image frames according to a plurality of evaluation scores in the multi-feature blur evaluation mechanism through a preset rule;

[0039] A blur repair path selection module is configured to select a corresponding repair strategy from a plurality of preset repair strategies according to the discriminated blur type to repair the image frames, thereby generating a repaired image;

[0040] A repair evaluation and feedback module is configured to perform quality evaluation on the repaired image, and to adjust the repair strategy and re-repair if the evaluation result does not satisfy a preset condition.

[0041] The clear video output module is used for reinserting the repaired image meeting the preset condition into the input video stream and outputting the repaired video.

[0042] In a third aspect, the application further discloses a computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the method in the first aspect.

[0043] In a fourth aspect, the application further discloses a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method in the first aspect.

[0044] Compared with the prior art, the application has the following advantages and technical effects:

[0045] The application provides a fuzzy type adaptive image repairing method, which comprises the following steps: firstly, frame extraction is performed on an input video stream, and the extracted image frames are preprocessed to obtain gray images; a comprehensive fuzzy score is calculated based on the gray images through a multi-feature fuzzy evaluation mechanism; whether the image frames are fuzzy frames is determined according to the comprehensive fuzzy score; if the image frames are fuzzy frames, the fuzzy types of the image frames are determined through a preset rule according to a plurality of evaluation scores in the multi-feature fuzzy evaluation mechanism; secondly, a corresponding repairing strategy is selected from a plurality of preset repairing strategies to repair the image frames according to the determined fuzzy types, and a repaired image is generated; then, quality evaluation is performed on the repaired image, and if the evaluation result does not meet a preset condition, the repairing strategy is adjusted and the image is repaired again; finally, the repaired image meeting the preset condition is reinserted into the input video stream, and a repaired video is output.

[0046] The application introduces the technical scheme of the multi-feature fuzzy evaluation mechanism and calculates the comprehensive fuzzy score, overcomes the defects of poor stability and easy interference caused by relying on a single feature for determination, significantly reduces the probability of misjudgment and omission, and improves the robustness and accuracy of fuzzy determination capability.

[0047] The application determines the fuzzy types according to a plurality of evaluation scores, can automatically identify specific causes (such as defocus, motion or mixed blur) of image blur, lays a solid foundation for subsequent accurate repairing, and solves the problem that the prior art cannot automatically distinguish fuzzy types.

[0048] The application selects a corresponding repairing strategy according to the fuzzy types, changes the extensive mode of using a single fixed model to process all fuzzy images in the prior art, matches the repairing strategy with the fuzzy causes, significantly improves the pertinence and effectiveness of repairing, realizes accurate and adaptive selection of the repairing strategy, and avoids over-repairing or insufficient repairing.

[0049] The application introduces a closed-loop feedback and optimization mechanism for repairing quality, has the ability of self-verification and iterative optimization through the technical content of "quality evaluation" and "adjusting the repair strategy when the condition is not met", and ensures the reliability of the final output result, solving the problem of lack of feedback mechanism and inability to dynamically adjust in the prior art.

[0050] The application considers the processing requirements of the video stream at the beginning of the design by "frame extraction" and finally "reinserting the repaired image into the video stream", ensures that the processed frame can be seamlessly restored to the video sequence, maintains the time sequence consistency and overall smoothness of the video, and makes up for the short board of the existing static image processing method which is difficult to apply to the video system.

[0051] The application is a complete processing system with high integration, strong adaptability and quality feedback closed loop, thereby comprehensively improving the accuracy, reliability and practicability of intelligent repair of complex and blurred images in the video stream. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their description together with the drawings serve to explain the application. In the drawings:

[0053] Figure 1 The image repair method flowchart of the embodiment of the application;

[0054] Figure 2 The image repair system schematic diagram of the embodiment of the application. DETAILED DESCRIPTION

[0055] It should be noted that the embodiments and features in the embodiments in the application can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0056] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0057] Embodiment one

[0058] As shown in Figure 1 The embodiment provides a blur type adaptive image repair method, which comprises:

[0059] Step S1, frame extraction processing is performed on the input video stream, and the extracted image frame is preprocessed;

[0060] Further, the process of frame extraction on the input video stream and pre-processing on the extracted image frames includes:

[0061] Periodically extracting frames from the input video stream according to the set frame interval parameter to obtain a set of image frames;

[0062] Scaling each image frame in the set of image frames to a fixed size;

[0063] Converting the scaled image into a grayscale image.

[0064] Specifically, the original video is usually a continuous frame stream, and directly processing the full frame will result in waste of computing resources and limited positioning accuracy of the blurred frame. Therefore, the embodiment periodically extracts frames from the video stream by setting the frame interval parameter: set the frame interval to N, that is, extract a frame every N frames; use the OpenCV tool library to realize the frame extraction operation, and the extracted frame set is denoted as {I1, I2, …, IT}, where T is the total number of extracted frames, and It represents the t-th single frame image in the extracted frame sequence set, t represents the index of the frame sequence, and t = 1, 2, …, T. T t Record the original timestamp information of each frame during the frame extraction process to facilitate the restoration of the complete video sequence after the repair is completed.

[0065] To unify the image input format, all extracted frames are scaled to a fixed size while keeping the original aspect ratio unchanged through mirroring or zero padding. At the same time, to adapt to various blur detection algorithms, the image is further converted into a grayscale image, denoted as wherein represents the pixel value of the grayscale image at coordinates (x, y) to reduce the computational dimension and enhance the expression of edge and structure features.

[0066] Step S2, based on the pre-processed image frames, a comprehensive blur score is calculated through a multi-feature blur evaluation mechanism; whether the image frame is a blurred frame is determined according to the comprehensive blur score;

[0067] Further, the process of calculating a comprehensive blur score through a multi-feature blur evaluation mechanism includes:

[0068] Calculating the edge sharpness index of the grayscale image to obtain a first evaluation score; calculating the frequency domain entropy value index of the grayscale image to obtain a second evaluation score;

[0069] Processing the grayscale image through a lightweight blur recognition network to obtain a third evaluation score;

[0070] ​performing a weighted summation on the first evaluation score, the second evaluation score and the third evaluation score to obtain a comprehensive blur score.

[0071] In step S3, if the image frame is a blur frame, a blur type of the image frame is determined according to a plurality of evaluation scores in the multi-feature blur evaluation mechanism through a preset rule.

[0072] Further, the process of determining the blur type of the image frame through the preset rule includes:

[0073] combining the first evaluation score, the second evaluation score and the third evaluation score into a feature vector;

[0074] matching the feature vector with a preset blur type determination rule to determine the blur type of the image frame; wherein the blur type includes defocus blur, motion blur and mixed blur.

[0075] Specifically, the multi-feature blur evaluation mechanism includes an edge sharpness index, a frequency domain entropy value index and a lightweight blur recognition network.

[0076] 1) Edge sharpness index:

[0077] The Laplacian operator is a second-order operator, which is obtained by operating twice in the horizontal direction and twice in the vertical direction, and the two results are superimposed to replace the gray value of the center point. The second-order edge response of the image is directly extracted by the Laplacian operator:

[0078] ;

[0079] wherein G(x,y) is a gradient amplitude graph, and I(x,y) represents a pixel value of a gray image at coordinates (x,y).

[0080] The proportion of high-frequency edge pixels is counted by counting the gradient amplitude graph G(x,y), and the gradient threshold value is set, and the proportion of pixels with edge strength greater than the threshold value is counted:

[0081] ;

[0082] wherein W is the width of the image, H is the height of the image, 1{⋅} is an indicator function, is the set gradient threshold value.

[0083] The embodiment defines an image edge sharpness index S1 for measuring the retention degree of high-frequency edge pixels in the image. First, the proportion of edge pixels greater than the threshold value is counted based on the gradient amplitude graph G(x,y) to obtain the edge intensity proportion Then, combined with the upper and lower limits of the edge proportion of historical samples, the normalized mapping formula is:

[0084] ;

[0085] wherein, is the maximum value of the high-frequency edge proportion, is the minimum value of the high-frequency edge proportion.

[0086] In the initialization stage, the high-frequency edge proportion of multiple clear images and blurred images is calculated respectively, and the maximum value and the minimum value are taken as the normalization parameters , .

[0087] 2) Frequency domain entropy value index:

[0088] The two-dimensional Fourier transform is performed on the gray image I(x, y):

[0089] ;

[0090] wherein, x and y are the horizontal and vertical coordinates in the spatial domain, u and v are the frequency coordinates in the frequency domain, denotes the Fourier transform symbol, is the complex spectrum value in the frequency domain, j is the imaginary unit, and satisfies .

[0091] Then, the frequency spectrum is taken:

[0092] ;

[0093] wherein, denotes the amplitude spectrum in the frequency domain after Fourier transform, which is used to measure the intensity of different frequency components of the image.

[0094] In order to enhance the contrast of low-frequency and high-frequency components, the amplitude spectrum is usually taken logarithm and centered:

[0095] ;

[0096] wherein, denotes the frequency spectrum after logarithmic compression of the amplitude spectrum.

[0097] Map to the probability density distribution:

[0098] ;

[0099] wherein, is the probability density distribution in the frequency domain.

[0100] According to the definition of Shannon entropy, the frequency domain entropy is calculated:

[0101] ;

[0102] wherein, is the image frequency complexity.

[0103] The image frequency complexity is calculated by using the frequency domain analysis method based on Shannon entropy , and the value reflects the richness of high-frequency texture and detail structure in the image. In order to unify the expression interval of different blur features, the is normalized and mapped into the blur score index S2, and the specific mapping method is:

[0104] ;

[0105] wherein, is the maximum value of the frequency domain entropy in the clear image; is the minimum value of the frequency domain entropy in the extreme blur image.

[0106] 3) Lightweight blur recognition network:

[0107] This part aims to realize the automatic recognition and classification of blurred images. A lightweight architecture is constructed based on deep convolutional neural network (CNN), which takes into account the recognition accuracy and running efficiency. Specifically, a lightweight backbone network structure based on ShuffleNe is adopted, and the feature extraction capability is improved through depthwise separable convolution and channel attention mechanism. After the input image passes through multiple layers of convolution, the network automatically learns the spatial features and texture degradation patterns related to blur, and outputs the blur probability P C .

[0108] The probability value P C ∈[0,1], of the current image being a blurred image is output. This probability is directly defined as the blur score index S3, that is:

[0109] S3=P C ;

[0110] This embodiment introduces a blur discrimination fusion mechanism based on weighted summation, which comprehensively evaluates the edge sharpness, frequency domain entropy features and deep learning recognition results of the image. Let the normalized blur scores output by the three blur discrimination methods be S1, S2 and S3, respectively, corresponding to the edge sharpness method, frequency domain entropy analysis method and lightweight blur recognition network prediction result. The final comprehensive blur score S final is calculated by the following weighted formula:

[0111] ;

[0112] wherein the weight coefficients α1, α2, α3 ∈ [0, 1], and α1+ α2+ α3= 1, which can be set by experience or obtained by tuning the validation set according to the task scenario. The higher the score, the more blurred the image is. In this embodiment, a threshold θ is set, and when >θ, the frame is determined to be a blurred frame, and subsequent repair is required.

[0113] This embodiment introduces a rule matching-based blur type discrimination method. This method is based on three normalized blur evaluation score indicators: edge sharpness score S1, frequency domain entropy feature score S2, and depth model output confidence score S3, to form a three-dimensional feature vector [S1, S2, S3]. According to the typical distribution characteristics of different types of blur in the above three dimensions, the following discrimination rule table is designed:

[0114] Defocus blur: S1< and S2> and S3> ;

[0115] Motion blur: S1< and S2< and S3> ;

[0116] Mixed blur: S1, S2, and S3 are in the middle value segment, or there is a conflict between the indicators;

[0117] Clear image: S1> and S2> and S3< ;

[0118] wherein, , , , , , , , are the type discrimination thresholds preset or adaptively adjusted in this embodiment.

[0119] Step S4, according to the blur type discriminated, selecting a corresponding repair strategy from a plurality of preset repair strategies to repair the image frame and generate a repaired image;

[0120] Further, the plurality of preset repair strategies includes a first strategy, a second strategy, and a third strategy.

[0121] The selecting a corresponding repair strategy from a plurality of preset repair strategies according to the blur type discriminated includes:

[0122] if the blur type is a defocus blur, a first strategy is selected, the first strategy being a reconstruction strategy based on a U-Net structure;

[0123] if the blur type is a motion blur, a second strategy is selected, the second strategy being a strategy based on maximum a posteriori probability deconvolution;

[0124] if the blur type is a mixed blur, a third strategy is selected, the third strategy being a composite restoration strategy combining a frequency domain branch and a spatial domain branch.

[0125] Specifically, the selection of the restoration strategy comprises:

[0126] For defocus blurred image frames in which image edges are generally blurred and lack high-frequency details, a U-Net structure is used to perform symmetrical encoding-decoding reconstruction on the image, focusing on detail recovery in the blurred area.

[0127] For motion blurred image frames with obvious directional trailing or overall image shift characteristics, a traditional maximum a posteriori probability (MAP) method is used to estimate the degradation kernel and perform deconvolution.

[0128] If the image frame contains multiple blur causes or there are classified blur crossover areas, the system routes the image to a composite restoration path, and through a hybrid structure joint processing, the frequency domain branch extracts image high-frequency information and enhances edge contours; the spatial domain branch uses convolution to reconstruct details and textures; the two branches are integrated through a feature fusion module to form the final output, improving overall perceptual consistency.

[0129] In step S5, the restored image is quality evaluated, and if the evaluation result does not satisfy a preset condition, the restoration strategy is adjusted and the restoration is re-performed;

[0130] Further, the determination process that the evaluation result does not satisfy the preset condition comprises:

[0131] The structural similarity index and the peak signal-to-noise ratio between the restored image and the original image are calculated;

[0132] If the structural similarity index or the peak signal-to-noise ratio is lower than a preset threshold value, it is determined that the evaluation result does not satisfy the preset condition.

[0133] Specifically, the embodiment further performs image detail enhancement and quality optimization processing according to the preliminary reconstruction result returned by the selected repair channel, to ensure the consistency of the repair result in subjective perception and objective evaluation. The multi-scale differences between the repair image and the original blurred image are modeled through the image pyramid fusion mechanism, to compensate for phenomena such as texture degradation, structure blurring, and oversmoothing that may be introduced in the blind deconvolution process. The image pyramid fusion structure can extract multi-level detail information from the bottom up and fuse it with the repair image output, effectively restoring the fine texture and edge details of the image.

[0134] To realize the automatic evaluation and screening of the repair result, the embodiment introduces the classic image quality indicators of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) for quantitative comparison between the repair image and the original image. If the evaluation indicators do not meet the preset threshold value, the embodiment will trigger an automatic backflow repair mechanism to dynamically adjust the current repair strategy, specifically including: switching to an alternative repair model (such as switching from blind deconvolution to U-Net reconstruction) for repair reinforcement.

[0135] Step S6: Reinserting the repair image that meets the preset condition into the input video stream and outputting the repaired video.

[0136] In the embodiment, in the frame extraction processing in step S1, the original timestamp information of each extracted image frame in the input video stream is recorded; and in step S6, the repair image is reinserted into the input video stream in sequence according to the original timestamp information.

[0137] Specifically, the embodiment ensures that the final output image has the characteristics of clear structure, rich details, and high subjective realism through the residual compensation and quality feedback mechanism, and has the ability to optimize the adaptive repair effect, which is suitable for fine enhancement and reliable output in multiple blurred image scenarios. When the repair image passes the quality evaluation, the system marks it as a "high-quality clear frame", and according to the timestamp and sequence recorded during the previous video frame extraction, the clear image frame is reinserted into the original video sequence, and finally a video result with completed image quality repair is output, ensuring the dual reliability of the video in subjective visibility and technical accuracy.

[0138] Embodiment Two

[0139] As shown in Figure 2 Based on the same inventive concept, the embodiment also provides an image repair system adaptive to the blur type, for realizing the method of embodiment one, comprising:

[0140] The video frame extraction and image preprocessing module is configured to perform frame extraction on the input video stream and perform preprocessing on the extracted image frames.

[0141] The blur detection module is configured to calculate a comprehensive blur score based on the preprocessed image frames by using a multi-feature blur evaluation mechanism, and determine whether the image frames are blurred frames according to the comprehensive blur score.

[0142] The blur type discrimination module is configured to, if the image frames are blurred frames, discriminate the blur type of the image frames according to the evaluation scores in the multi-feature blur evaluation mechanism by using a preset rule.

[0143] The blur repair path selection module is configured to select a corresponding repair strategy from a plurality of preset repair strategies according to the discriminated blur type to repair the image frames, and generate repaired images.

[0144] The repair evaluation and feedback module is configured to evaluate the quality of the repaired images, and adjust the repair strategy and re-repair if the evaluation result does not satisfy a preset condition.

[0145] The clear video output module is configured to insert the repaired images satisfying the preset condition into the input video stream, and output the repaired video.

[0146] The blur type adaptive image repair system provided in the embodiment has all the advantages of the blur type adaptive image repair method provided in the first embodiment.

[0147] Embodiment Three

[0148] The embodiment also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method in the first embodiment.

[0149] Embodiment Four

[0150] The embodiment also discloses a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method in the first embodiment.

[0151] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements within the technical range disclosed in the present application can be easily thought by those skilled in the art, and should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of image inpainting with blur type adaptation, characterized in that, The method comprises the following steps: frame extraction processing is performed on an input video stream, and the extracted image frames are preprocessed to obtain grayscale images; a comprehensive fuzzy score is calculated based on the grayscale images through a multi-feature fuzzy evaluation mechanism; it is determined whether the image frames are fuzzy frames according to the comprehensive fuzzy score; the process of calculating the comprehensive fuzzy score through the multi-feature fuzzy evaluation mechanism comprises: an edge definition index of the grayscale image is calculated to obtain a first evaluation score, and a frequency domain entropy value index of the grayscale image is calculated to obtain a second evaluation score; the grayscale image is processed through a lightweight blur recognition network to obtain a third evaluation score; the first evaluation score, the second evaluation score and the third evaluation score are weighted and summed to obtain a comprehensive fuzzy score; if the image frames are fuzzy frames, the fuzzy type of the image frames is determined according to a plurality of evaluation scores in the multi-feature fuzzy evaluation mechanism through a preset rule; the process of determining the fuzzy type of the image frames through the preset rule comprises: the first evaluation score, the second evaluation score and the third evaluation score are combined into a feature vector; the feature vector is matched with a preset fuzzy type determination rule to determine the fuzzy type of the image frames; wherein the fuzzy type comprises defocus blur, motion blur and mixed blur; a corresponding repair strategy is selected from a plurality of preset repair strategies according to the determined fuzzy type to repair the image frames to generate a repaired image; the repaired image is quality evaluated, and if the evaluation result does not satisfy a preset condition, the repair strategy is adjusted and repaired again; the repaired image satisfying the preset condition is reinserted into the input video stream to output a repaired video.

2. The method of claim 1, wherein, The process of frame extraction processing on an input video stream and preprocessing the extracted image frames comprises: periodically frame extraction is performed on the input video stream according to a set frame interval parameter to obtain a set of image frames; each image in the set of image frames is scaled to a fixed size; the scaled image is converted into a grayscale image.

3. The method of claim 1, wherein, The plurality of preset repair strategies comprises a first strategy, a second strategy and a third strategy; the process of selecting a corresponding repair strategy from the plurality of preset repair strategies according to the determined fuzzy type comprises: if the fuzzy type is defocus blur, the first strategy is selected, and the first strategy is a reconstruction strategy based on a U-Net structure; if the fuzzy type is motion blur, the second strategy is selected, and the second strategy is a strategy based on maximum a posteriori probability deconvolution; if the fuzzy type is mixed blur, the third strategy is selected, and the third strategy is a composite repair strategy combining a frequency domain branch and a spatial domain branch.

4. The method of claim 1, wherein, The determination process that the evaluation result does not satisfy the preset condition comprises: a structural similarity index and a peak signal-to-noise ratio between the repaired image and an original image are calculated; if the structural similarity index or the peak signal-to-noise ratio is lower than a preset threshold value, it is determined that the evaluation result does not satisfy the preset condition.

5. The method of claim 1, wherein In the frame extraction process, original timestamp information of each extracted image frame in the input video stream is recorded; The repaired image is re-inserted into the input video stream according to the original timestamp information.

6. An image inpainting system with blur type adaptation, characterized in that, A method for implementing any one of claims 1-5, comprising: a video frame extraction and image preprocessing module for performing frame extraction on the input video stream and preprocessing the extracted image frames; a blur detection module for calculating a comprehensive blur score based on the preprocessed image frames through a multi-feature blur evaluation mechanism, and determining whether the image frame is a blurred frame according to the comprehensive blur score; a blur type identification module for identifying the blur type of the image frame according to the multiple evaluation scores in the multi-feature blur evaluation mechanism through a preset rule if the image frame is a blurred frame; a blur repair path selection module for selecting a corresponding repair strategy from multiple preset repair strategies to repair the image frame according to the identified blur type, and generating a repaired image; a repair evaluation and feedback module for quality evaluation of the repaired image, and adjusting the repair strategy and re-repairing if the evaluation result does not meet the preset condition; a clear video output module for re-inserting the repaired image that meets the preset condition into the input video stream and outputting the repaired video.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by a processor to implement the steps of the method of any one of claims 1-5.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-5.

Images