Image Restoration Method, System and Storage Medium Based on Semantic Segmentation and Wiener Filtering

Abstract

The present invention discloses an image restoration method, system and storage medium based on semantic segmentation and Wiener filtering. The image restoration method includes: S1. Performing semantic segmentation on an image based on a lightweight model to obtain a semantic segmentation image; S2. Evaluating the degree of image blurring according to an image evaluation index; when the image evaluation index is less than or equal to an index threshold, it is determined as a motion-blurred image, and based on the semantic segmentation image, the motion-blurred image is divided into a foreground region and a background region; S3. Obtaining the PSF parameters of the foreground region and the background region; S4. Performing Wiener filtering processing on the foreground region and the background region according to the PSF parameters of the foreground region and the background region; S5. Evaluating the degree of image blurring of the image after Wiener filtering processing according to the image evaluation index; when the image evaluation index is less than or equal to the index threshold, it is determined that the image restoration is not completed, and after adjusting the PSF parameters, return to step S4 until the image restoration is completed. The present invention can improve the accuracy and speed of image restoration.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an image restoration method, system and storage medium based on semantic segmentation and Wiener filtering. Background Art

[0002] Images are of great significance in various fields and have received extensive attention and applications in aerospace, military technology, security monitoring, autonomous driving, etc. Among them, in most scenario applications, the camera will acquire or save data in the form of video image sequences. However, during the shooting process, image data is often interfered by the external environment. Among them, the image blur caused by the relative motion between the camera and the target object is called motion blur. Due to the universality and severity of motion blur, the restoration processing of motion-blurred images has become a research hotspot.

[0003] The restoration processing for the above-mentioned motion blur can be divided into two processing methods: software and hardware. Compared with hardware processing, software processing has the advantages of high flexibility, low cost, high accuracy, etc. Among them, as a classic software image restoration processing method, the Wiener filtering algorithm has significant superiority compared with other methods. The Wiener filtering algorithm not only ensures the quality and accuracy of image restoration but also avoids a large amount of computation. The Wiener filtering algorithm is based on the principle of minimum mean square error, comprehensively considering two aspects: the degradation function and the noise statistical characteristics, and can obtain corresponding prior knowledge for the input image to ensure that the mean square error between the original image and the restored image is minimized. Among them, according to the basic principle of motion blur, the point spread function (PSF) is proposed as the prior knowledge for the blurred image. The PSF describes the motion blur parameters: blur length and blur angle. By using an appropriate PSF and the image signal-to-noise ratio SNR, Wiener filtering can obtain a high-quality restored image, thus meeting the requirements of various applications.

[0004] For the image restoration processing of video image sequences, it is often difficult to meet the requirements of real-time and high accuracy only using existing technologies. The main reasons are as follows:

[0005] First, not every frame of the video image sequence is affected by motion blur. However, the Wiener filtering image restoration algorithm for a single image does not consider this problem. If the Wiener filtering algorithm is implemented for image restoration for all input image sequences, the real-time performance will be greatly reduced.

[0006] Second, the Wiener filtering algorithm can calculate the blur length and blur direction in the PSF through the Radon transform, but the calculated PSF parameters usually do not meet the requirements of high precision. Therefore, it is difficult to meet the requirements of high-quality image restoration if only the calculated PSF parameters are simply used. Secondly, there is strong continuity and correlation between adjacent frames of the video image sequence, and calculating the Radon transform will consume a large amount of computing resources. If the Radon transform is calculated for each frame of the video image sequence, it will be difficult to meet the real-time requirements.

[0007] Finally, in practical applications, the moving objects in the image and the movement of the camera are often different. Therefore, the corresponding PSF is also different. If the PSF is calculated for the whole image, it will lead to inaccurate calculation of the PSF, thus causing a decrease in the accuracy of image restoration.

[0008] Therefore, in view of the above technical problems, it is necessary to provide an image restoration method, system and storage medium based on semantic segmentation and Wiener filtering. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide an image restoration method, system and storage medium based on semantic segmentation and Wiener filtering to improve the accuracy and speed of image restoration.

[0010] To achieve the above purpose, the technical solution provided by an embodiment of the present invention is as follows:

[0011] An image restoration method based on semantic segmentation and Wiener filtering, the image restoration method includes the following steps:

[0012] S1. Perform semantic segmentation on the image based on a lightweight model to obtain a semantic segmentation image;

[0013] S2. Evaluate the image blur degree according to the image evaluation index;

[0014] When the image evaluation index is greater than the index threshold, determine that the image is a non-blurred image and directly output the image;

[0015] When the image evaluation index is less than or equal to the index threshold, determine it as a motion-blurred image. Based on the semantic segmentation image, divide the motion-blurred image into a foreground area and a background area, where the foreground area is the motion-blurred part generated by the dynamic object, and the background area is the motion-blurred part generated by the camera movement;

[0016] S3. Obtain the PSF parameters of the foreground area and the background area, and the PSF parameters include the blur angle and the blur length;

[0017] S4. Perform Wiener filtering processing on the foreground area and the background area according to the PSF parameters of the foreground area and the background area;

[0018] S5. Evaluate the blurriness of the image after Wiener filtering according to the image evaluation index;

[0019] When the image evaluation index is greater than the index threshold, it is determined that the image restoration is completed, and the restored image is output;

[0020] When the image evaluation index is less than or equal to the index threshold, it is determined that the image restoration is not completed. After adjusting the PSF parameters, return to step S4 until the image restoration is completed.

[0021] In one embodiment, before the step S1, it further includes:

[0022] Convert the original image into a grayscale image.

[0023] In one embodiment, the image evaluation index in the step S1 is the Laplace operator, and its calculated value L is:

[0024]

[0025] where Src is the input image, x and y respectively represent the horizontal axis direction and the vertical axis direction in the Cartesian coordinate system of the input image, and Δ 2 is a second-order linear differential operation.

[0026] In one embodiment, the blur angle and the blur length in the step S3 are obtained by Radon transform, and the step S3 is specifically:

[0027] Perform a Hamming window on the image f(x, y);

[0028] Perform a fast Fourier transform on the image after performing the Hamming window, and take the logarithm of the obtained FFT result to obtain the logarithmic spectrum F(u, v);

[0029] Calculate the Radon transform matrix in the range of θ = 0 to 180°:

[0030]

[0031] where θ is the angle between the horizontal direction and the perpendicular line from the origin to the straight line, δ represents the distance from the origin to any point (x, y) in the input image, that is, ρ = xcosθ + ysinθ; δ represents the impulse function,

[0032] Obtain the peak value in the Radon transform matrix, which is the blur angle;

[0033] Obtain all the local minimum values in the Radon transform matrix, and calculate the average distance d between the local minimum values. The blur length is the ratio N / d of the image size N×N to the average distance d.

[0034] In one embodiment, step S4 includes:

[0035] Perform noise reduction processing on the input image using Gaussian filtering and median filtering to obtain an ideal image, and obtain the SNR of the input image and the ideal image:

[0036]

[0037] where f i (x, y) is the ideal image after noise reduction, f(x, y) is the input image, and N and M respectively correspond to the width and length of the input image;

[0038] Establish a motion blur mathematical model of the image:

[0039] f(x, y) = g(x, y) * h(x, y) + n(x, y),

[0040] where g(x, y) is the noise-free image; h(x, y) is the motion blur function, and n(x, y) is the additive noise;

[0041] Perform a fast Fourier transform on the motion blur mathematical model:

[0042] F(u, v) = G(u, v)H(u, v) + N(u, v)

[0043] where F(u, v), G(u, v), H(u, v), and N(u, v) are the frequency domain representations of the corresponding functions, and H(u, v) is the PSF parameter;

[0044] According to the minimum mean square error E = {[g(x, y, ) - f(x, y , )] 2}, perform Wiener filtering on the motion blur image:

[0045]

[0046] where k is 1 / SNR.

[0047] In one embodiment, step S5 includes:

[0048] Obtain the calculated value of the Laplace operator of the current image;

[0049] Obtain the SNR of the current image and the previous image;

[0050] Adaptively adjust the PSF parameter according to the calculated value of the Laplace operator and the SNR.

[0051] In one embodiment, step S5 includes:

[0052] Adjust the blur angle and / or blur length of the foreground area and the background area at a preset step size.

[0053] In one embodiment, before the step S1, it further includes:

[0054] Convert the video into a picture sequence including multiple frames of images;

[0055] After the step S5, it further includes:

[0056] Convert the restored multiple frames of images into a video.

[0057] The technical solution provided by another embodiment of the present invention is as follows:

[0058] An image restoration system based on semantic segmentation and Wiener filtering, the image restoration system includes:

[0059] A semantic segmentation module, configured to perform semantic segmentation on an image based on a lightweight model to obtain a semantic segmentation image;

[0060] A blur degree evaluation module, configured to evaluate the blur degree of an image according to an image evaluation index; when the image evaluation index is greater than the index threshold, determine that the image is a non-blurred image; when the image evaluation index is less than or equal to the index threshold, determine that it is a motion-blurred image;

[0061] An image division module, configured to divide a motion-blurred image into a foreground area and a background area based on the semantic segmentation image, where the foreground area is the motion-blurred part generated by a dynamic object, and the background area is the motion-blurred part generated by camera movement;

[0062] A PSF parameter acquisition module, configured to acquire the PSF parameters of the foreground area and the background area, where the PSF parameters include a blur angle and a blur length;

[0063] A Wiener filtering module, configured to perform Wiener filtering processing on the foreground area and the background area according to the PSF parameters of the foreground area and the background area;

[0064] A PSF parameter adjustment module, configured to adjust the PSF parameters when the image restoration is not completed.

[0065] The technical solution provided by another embodiment of the present invention is as follows:

[0066] A machine-readable storage medium storing executable instructions that, when executed, cause the machine to execute the above image restoration method.

[0067] The present invention has the following beneficial effects:

[0068] In the image restoration method based on semantic segmentation and Wiener filtering of the present invention, for different motion states of objects in the image, a semantic segmentation algorithm is introduced to divide the image into multiple regions to achieve targeted image restoration processing. For the original Wiener filtering algorithm, an adaptive algorithm based on Laplace is introduced, which improves the robustness of the algorithm to different scenarios, thereby greatly improving the accuracy and speed of image restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0070] Figure 1 It is a schematic flowchart of the image restoration method based on semantic segmentation and Wiener filtering in a specific embodiment of the present invention;

[0071] Figure 2 It is a schematic diagram of the modules of the image restoration system based on semantic segmentation and Wiener filtering in a specific embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0073] In order to facilitate the understanding of the embodiments of the present invention, several elements introduced in the description of the embodiments of the present invention will be introduced here first.

[0074] PSF: point spread function.

[0075] SNR: signal noise ratio.

[0076] Laplace: Laplace operator.

[0077] Radon transform: Radon transform, which projects the digital image matrix in the direction of a specified angle ray.

[0078] Semantic segmentation: Classification of image object categories at the pixel level.

[0079] Fourier transform: In the field of image processing, it is expressed as transforming the gray distribution function of an image into the frequency distribution function of the image.

[0080] Parameter Figure 1 As shown, in a specific embodiment of the present invention, an image restoration method based on semantic segmentation and Wiener filtering includes the following steps:

[0081] S1. Perform semantic segmentation on the image based on a lightweight model to obtain a semantic segmentation image;

[0082] S2. Evaluate the image blurring degree according to the image evaluation index;

[0083] When the image evaluation index is greater than the index threshold, determine that the image is a non-blurred image and directly output the image;

[0084] When the image evaluation index is less than or equal to the index threshold, determine it as a motion-blurred image. Based on the semantic segmentation image, divide the motion-blurred image into a foreground region and a background region. Among them, the foreground region is the motion-blurred part generated by the dynamic object, and the background region is the motion-blurred part generated by the camera movement;

[0085] S3. Obtain the PSF parameters of the foreground region and the background region, where the PSF parameters include the blur angle and the blur length;

[0086] S4. Perform Wiener filtering processing on the foreground region and the background region according to the PSF parameters of the foreground region and the background region;

[0087] S5. Evaluate the image blurring degree of the image after Wiener filtering processing according to the image evaluation index;

[0088] When the image evaluation index is greater than the index threshold, determine that the image restoration is completed and output the restored image;

[0089] When the image evaluation index is less than or equal to the index threshold, determine that the image restoration is not completed, adjust the PSF parameters, and return to step S4 until the image restoration is completed.

[0090] The following elaborates on this embodiment in detail in combination with each step in the image restoration method.

[0091] In this embodiment, for the restoration of a video image sequence, first convert the video into a picture sequence including multiple frames. The method of converting a video into a picture sequence is a prior art and will not be elaborated here.

[0092] Furthermore, convert the original image in the image sequence into a grayscale image to reduce the subsequent computational requirements.

[0093] Subsequently, the grayscale image is delivered to the semantic segmentation thread and the image motion blur degree evaluation thread in a multi-threaded manner for image restoration.

[0094] The specific steps of image restoration are as follows:

[0095] 1. In the semantic segmentation thread, the image is semantically segmented based on a lightweight model to obtain a semantic segmentation image.

[0096] Semantic segmentation is a fundamental task in the field of computer vision, and its goal is to label the semantic tags to which each pixel in the image belongs. Compared with the semantic segmentation models implemented based on traditional machine learning algorithms and the Transformer architecture, the existing semantic segmentation models based on convolutional neural networks show more excellent accuracy and speed, which better meet the requirements of the present invention.

[0097] Therefore, the present invention adopts a lightweight semantic segmentation model implemented based on a convolutional neural network. Existing lightweight semantic segmentation models proposed based on convolutional neural networks, such as: DeepLab series models, SegNet models, PP-LiteSeg models, etc. can all be applied to the present invention.

[0098] 2. In the image motion blur degree evaluation thread, the image blur degree is evaluated according to the image evaluation index.

[0099] The Laplace operator is a second-order linear differential operator and has strong image edge detection ability compared with other image evaluation indexes. Therefore, in this embodiment, it is used as the image evaluation index, and its calculated value L is:

[0100]

[0101] where Src is the input image, x and y respectively represent the horizontal axis direction and the vertical axis direction in the Cartesian coordinate system of the input image, and Δ 2 is the second-order linear differential operation.

[0102] According to the index L calculated by the Laplace operator and the set index threshold, it is judged whether the input image has motion blur. The set index threshold is different for different scenarios and applications. In this embodiment, the index threshold is set to 11.

[0103] When the image evaluation index L is greater than the index threshold, it is determined that the image is a non-blurred image, and the image is directly output;

[0104] When the image evaluation index L is less than or equal to the index threshold, it is determined as a motion-blurred image. Based on the semantic segmentation image generated by the semantic segmentation thread, the motion-blurred image is divided into a foreground region and a background region. Among them, the foreground region is the motion-blurred part generated by dynamic objects, and the background region is the motion-blurred part generated by camera movement. Here, the foreground and background can be divided and the number of foreground dynamic object categories can be adjusted according to the actual application scenario.

[0105] 3. Obtain the PSF parameters of the foreground region and the background region. The PSF parameters include the blur angle and the blur length.

[0106] In this embodiment, the blur angle and the blur length are calculated by the Radon transform, specifically:

[0107] Since the image f(x, y) will be Fourier-transformed later, first perform a Hamming window on the image f(x, y) to reduce image artifacts (high-frequency components);

[0108] Perform a fast Fourier transform (FFT) on the image after performing the Hamming window, and take the logarithm of the obtained FFT result to obtain the logarithmic spectrum F(u, v);

[0109] Next, calculate the Radon transform matrix R in the range of θ = 0 to 180°:

[0110]

[0111] where θ is the angle between the horizontal direction and the perpendicular line from the origin to the straight line, δ represents the distance from the origin to any point (x, y) in the input image, that is, ρ = xcosθ + ysinθ; δ represents the impulse function,

[0112] Obtain the peak value in the Radon transform matrix R, which is the blur angle;

[0113] Obtain all the local minimum values in the Radon transform matrix R, and calculate the average distance d between the local minimum values. The blur length is the ratio N / d of the image size N×N to the average distance d.

[0114] 4. According to the PSF parameters of the foreground region and the background region, use a multi-threaded method to perform Wiener filtering processing on the foreground region and the background region respectively to improve real-time performance.

[0115] The specific steps of Wiener filtering processing in this embodiment are as follows:

[0116] Perform noise reduction processing on the input image using Gaussian filtering and median filtering to obtain an ideal image, and obtain the SNR of the input image and the ideal image:

[0117]

[0118] where f i (x, y) is the ideal image after noise reduction, f(x, y) is the input image, and N and M respectively correspond to the width and length of the input image;

[0119] Establish a motion blur mathematical model of the image:

[0120] f(x, y) = g(x, y) * h(x, y) + n(x, y),

[0121] where g(x, y) is the noise-free image; h(x, y) is the motion blur function, and n(x, y) is the additive noise;

[0122] Perform a fast Fourier transform on the motion blur mathematical model:

[0123] F(u, y) = G(u, v)H(u, v) + N(u, v),

[0124] where F(u, v), G(u, v), H(u, v), and N(u, v) are the frequency domain representations of the corresponding functions respectively, and H(u, v) is the PSF parameter;

[0125] According to the minimum mean square error E = {[g(x, y) - f(x, y)] 2}}, perform Wiener filtering on the motion blur image:

[0126]

[0127] where k is 1 / SNR.

[0128] 5. Evaluate the blurriness of the image after Wiener filtering according to the image evaluation index L;

[0129] When the image evaluation index L is greater than the index threshold, it is determined that the image restoration is completed, and the restored image is output;

[0130] When the image evaluation index L is less than or equal to the index threshold, it is determined that the image restoration is not completed. After adjusting the PSF parameter, return to step 4 until the image restoration is completed.

[0131] After Wiener filtering, the Laplace operator will calculate the blurriness of the processed image again. If it exceeds the set index threshold, it is considered that the image restoration is completed, and the restored result is output; if it is less than the set index threshold, it is considered that the restored image does not meet the requirements. After adjusting the PSF parameter, continue to perform Wiener filtering until the condition is met.

[0132] Among them, the adjustment of the PSF parameter is specifically:

[0133] Obtain the calculated value of the Laplace operator for the current image;

[0134] Obtain the SNR between the current image and the previous image;

[0135] Adaptively adjust the blur length and blur angle in the PSF parameters according to the calculated value of the Laplace operator and the SNR.

[0136] Through the adaptive adjustment of the PSF parameters, the image restoration result can be further optimized. Preferably, in this embodiment, the blur angle and blur length in the PSF are finely adjusted in a small step size to obtain a higher-quality restored image.

[0137] Finally, the high-quality restored image that meets the application requirements is output to the downstream task.

[0138] In this embodiment, for the restoration of video image sequences, after obtaining the restored image sequences, all the image sequences need to be converted into a video. The method of converting image sequences into a video is a prior art and will not be elaborated here.

[0139] The image restoration method based on semantic segmentation and Wiener filtering proposed by the present invention improves the accuracy and speed of image restoration.

[0140] Since the original image restoration method does not consider the internal relationships (continuity and correlation) of video image sequences, inaccurate calculation of PSF parameters, and different motion states within the image, the real-time performance of image motion blur restoration is low and the accuracy is not high.

[0141] Therefore, the present invention first proposes a targeted framework for the internal relationships of video image sequences, which can efficiently and highly real-timely process the motion blur restoration of video image sequences; furthermore, the input image is subjected to semantic segmentation processing to ensure the accuracy of calculating the PSF when multiple regions have different motion states; finally, on this basis, an adaptive image restoration algorithm based on Laplace is introduced to further optimize the PSF. Ultimately, the motion blur restoration processing of video image sequences with faster speed and higher accuracy is realized.

[0142] See Figure 2 As shown, in a specific embodiment of the present invention, an image restoration system based on semantic segmentation and Wiener filtering includes:

[0143] A semantic segmentation module, configured to perform semantic segmentation on an image based on a lightweight model to obtain a semantic segmentation image;

[0144] A blur degree evaluation module, which is used to evaluate the blur degree of an image according to an image evaluation index; when the image evaluation index is greater than the index threshold, it is determined that the image is a non-blurred image; when the image evaluation index is less than or equal to the index threshold, it is determined to be a motion-blurred image;

[0145] An image division module, which is used to divide a motion-blurred image into a foreground area and a background area based on a semantic segmentation image, where the foreground area is the motion-blurred part generated by a dynamic object, and the background area is the motion-blurred part generated by camera movement;

[0146] A PSF parameter acquisition module, which is used to acquire the PSF parameters of the foreground area and the background area, and the PSF parameters include a blur angle and a blur length;

[0147] A Wiener filtering module, which is used to perform Wiener filtering processing on the foreground area and the background area according to the PSF parameters of the foreground area and the background area;

[0148] A PSF parameter adjustment module, which is used to adjust the PSF parameters when the image restoration is not completed.

[0149] The present invention also discloses a machine-readable storage medium, which stores executable instructions, and when the instructions are executed, the machine executes the above image restoration method.

[0150] Specifically, a system or device equipped with a readable storage medium can be provided, and software program codes for implementing the functions of any one of the above embodiments are stored on the readable storage medium, and the computer or processor of the system or device is made to read and execute the instructions stored in the readable storage medium.

[0151] In this case, the program code read from the readable medium itself can implement the functions of any one of the above embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of this specification.

[0152] Examples of the readable storage medium include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer or a cloud via a communication network.

[0153] Those skilled in the art should understand that various modifications and changes can be made to the above-disclosed embodiments without departing from the essence of the invention. Therefore, the protection scope of this specification should be defined by the appended claims.

[0154] It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined according to requirements. The device structures described in the above embodiments can be physical structures or logical structures. That is, some units may be implemented by the same physical entity, or some units may be implemented separately by multiple physical entities, or some components in multiple independent devices may jointly implement them.

[0155] In the above embodiments, the hardware units or modules can be implemented mechanically or electrically. For example, a hardware unit, module, or processor can include permanent dedicated circuits or logic (such as a dedicated processor, FPGA, or ASIC) to perform corresponding operations. The hardware unit or processor can also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which can be temporarily set by software to perform corresponding operations. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0156] The specific embodiments described above in conjunction with the accompanying drawings describe exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of protection of the claims. The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration", and does not mean "preferred" or "advantageous" compared to other embodiments. For the purpose of providing an understanding of the described technology, the specific embodiments include specific details. However, these technologies can be implemented without these specific details. In some instances, well-known structures and devices are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0157] The above description of the present disclosure is provided to enable any ordinary person skilled in the art to implement or use the present disclosure. Various modifications to the present disclosure are obvious to those of ordinary skill in the art, and the general principles corresponding herein can also be applied to other variations without departing from the scope of protection of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but is consistent with the broadest scope that conforms to the principles and novel features disclosed herein.

Claims

1. An image restoration method based on semantic segmentation and Wiener filtering, characterized in that, the image restoration method includes the following steps: S1. Perform semantic segmentation on the image based on a lightweight model to obtain a semantic segmentation image; S2. Evaluate the image blur degree according to an image evaluation index; When the image evaluation index is greater than the index threshold, determine that the image is a non-blurred image and directly output the image; When the image evaluation index is less than or equal to the index threshold, determine it as a motion-blurred image. Based on the semantic segmentation image, divide the motion-blurred image into a foreground region and a background region. Among them, the foreground region is the motion-blurred part generated by the dynamic object, and the background region is the motion-blurred part generated by the camera movement; S3. Obtain the PSF parameters of the foreground region and the background region, and the PSF parameters include a blur angle and a blur length; S4. Perform Wiener filtering processing on the foreground region and the background region according to the PSF parameters of the foreground region and the background region; S5. Evaluate the image blur degree of the image after Wiener filtering processing according to the image evaluation index; When the image evaluation index is greater than the index threshold, determine that the image restoration is completed and output the restored image; When the image evaluation index is less than or equal to the index threshold, determine that the image restoration is not completed, adjust the PSF parameters and return to step S4 until the image restoration is completed; The image evaluation index in the step S2 is the Laplace operator, and its calculated value L is as follows: ； Among them, Src is the input image, x and y respectively represent the horizontal axis direction and the vertical axis direction in the Cartesian coordinate system of the input image, is a second-order linear differential operation; The step S5 includes: Obtain the calculated value of the Laplace operator of the current image; Obtain the SNR between the current image and the previous image; Adaptive adjust the PSF parameters according to the calculated value of the Laplace operator and the SNR.

2. The image restoration method based on semantic segmentation and Wiener filtering according to claim 1, characterized in that, before the step S1, it further includes: Convert the original image into a grayscale image.

3. The image restoration method based on semantic segmentation and Wiener filtering according to claim 1, characterized in that, the blur angle and the blur length in the step S3 are obtained by Radon transform, and the step S3 is specifically: Perform Hamming window on the image Execute the Hamming window; Perform a fast Fourier transform on the image after applying the Hamming window, and take the logarithm of the resulting FFT result to obtain the logarithmic spectrum ; Calculate the Radon transform matrix within the range: ， Among them, is the angle between the horizontal direction and the perpendicular line from the origin to the straight line, represents the distance from the origin to any point (x, y) in the input image, that is ; represents the impulse function, ; Obtain the peak value in the Radon transform matrix, which is the blur angle; Obtain all local minima in the Radon transform matrix and calculate the average distance between the local minima d , and the blurring length is the image size N×N and the average distance d ratio N / d .

4. The image restoration method based on semantic segmentation and Wiener filtering according to claim 1, characterized in that, the step S4 includes: Perform noise reduction processing on the input image using Gaussian filtering and median filtering to obtain an ideal image, and obtain the SNR between the input image and the ideal image: ， Among them, f i (x,y) is the ideal image after noise reduction, f(x,y) is the input image, N, M respectively correspond to the width and length of the input image; Establish a motion blur mathematical model of the image: ， wherein, g(x,y) is the noise-free image; h(x,y) is the motion blur function, n(x,y) is the additive noise; Perform fast Fourier transform on the motion blur mathematical model: ， wherein, are the frequency domain representations of the corresponding functions, which are the PSF parameters; According to the minimum mean square error , perform Wiener filtering on the motion-blurred image: ， Among them, k is 1 / SNR .

5. The image restoration method based on semantic segmentation and Wiener filtering according to claim 4, characterized in that, the step S5 includes: Adjust the blur angle and / or the blur length of the foreground region and the background region with a preset step size.

6. The image restoration method based on semantic segmentation and Wiener filtering according to claim 1, characterized in that, before the step S1, it further includes: Convert the video into a picture sequence including multiple frames of images; after the step S5, it further includes: Convert the restored multiple frames of images into a video.

7. An image restoration system based on semantic segmentation and Wiener filtering, characterized in that, the image restoration system includes: a semantic segmentation module for performing semantic segmentation on an image based on a lightweight model to obtain a semantic segmentation image; a blur degree evaluation module for evaluating the blur degree of an image according to an image evaluation index; when the image evaluation index is greater than the index threshold, determining that the image is a non-blurred image; when the image evaluation index is less than or equal to the index threshold, determining that it is a motion-blurred image; an image division module for dividing the motion-blurred image into a foreground region and a background region based on the semantic segmentation image, where the foreground region is the motion-blurred part generated by a dynamic object, and the background region is the motion-blurred part generated by camera motion; a PSF parameter acquisition module for acquiring the PSF parameters of the foreground region and the background region, where the PSF parameters include a blur angle and a blur length; a Wiener filtering module for performing Wiener filtering processing on the foreground region and the background region according to the PSF parameters of the foreground region and the background region; a PSF parameter adjustment module for adjusting the PSF parameters when the image restoration is not completed; The image evaluation index is the Laplace operator, and its calculated value L is as follows: ； Among them, Src is the input image, x and y respectively represent the horizontal axis direction and the vertical axis direction in the Cartesian coordinate system of the input image, is the second-order linear differential operation the PSF parameter adjustment module is further configured to obtain the calculated value of the Laplace operator of the current image, obtain the SNR between the current image and the previous image, and adaptively adjust the PSF parameters according to the calculated value of the Laplace operator and the SNR.

8. A machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the image restoration method according to any one of claims 1 to 6.

Images