Image restoration method and device, storage medium and electronic equipment

Abstract

The invention discloses an image restoration method and device, a storage medium and electronic equipment, and relates to the technical field of image processing. The image restoration method comprises the following steps: acquiring a to-be-restored image and a degradation function corresponding to the to-be-restored image; based on the to-be-restored image, calculating to obtain a signal-to-noise ratio of the to-be-restored image; and based on the degradation function and the signal-to-noise ratio of the to-be-restored image, performing image restoration on the to-be-restored image by adopting a preset approximate Wiener filtering model to obtain a restored image. The method does not need a power spectrum of an undegraded image and noise any more, is high in operability, can guarantee correct recovery of the to-be-recovered image, and improves the reliability of image recovery.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image restoration method, an image restoration device, a machine-readable storage medium and an electronic device. Background Art

[0002] During the imaging process, the image quality often deteriorates due to the relative motion between the imaging device and the object being photographed during the exposure time, resulting in motion-blurred images. For example, when an excavator is equipped with a surround-view camera, the camera rotates with the excavator body, which causes motion blur noise in the image. This not only affects the image quality, but also loses important details. This causes the operator to be unable to observe the environment around the excavator well, especially during remote control, when the operator can only watch the video captured by the camera. Therefore, it is necessary to restore the motion-blurred images.

[0003] The prior art proposes to use Wiener filtering for image restoration. This method requires the power spectrum of the undegraded image and noise. However, in actual use, it is difficult to accurately obtain the power spectrum of the undegraded image and noise, so the operability of this method is not strong. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image restoration method, an image restoration device, a machine-readable storage medium and an electronic device, so as to solve the problem in the prior art that it is difficult to accurately obtain the power spectrum of the undegraded image and the noise when using Wiener filtering for image restoration, and the operability is not strong.

[0005] In order to achieve the above-mentioned object, the present application provides an image restoration method in a first aspect, comprising: Acquire an image to be restored and a degradation function corresponding to the image to be restored; Based on the image to be restored, calculating a signal-to-noise ratio of the image to be restored; Based on the degradation function and the signal-to-noise ratio of the image to be restored, a preset approximate Wiener filter model is used to perform image restoration on the image to be restored to obtain a restored image.

[0006] In an embodiment of the present application, obtaining a degradation function corresponding to the image to be restored includes: Acquiring motion information corresponding to the image to be restored, where the motion information corresponding to the image to be restored includes at least one type of motion speed; Acquire an image acquisition position of the image to be restored; Based on the type of the motion speed, determining a motion state; Based on the motion state, the image acquisition position and the motion speed, the motion parameter is determined according to a preset parameter determination rule; Substituting the motion parameters into a preset degradation function model, a degradation function is obtained.

[0007] In the embodiment of the present application, the preset degradation function model is: , in, , is the motion parameter, For time, is the exposure time, is the degradation function, is a frequency domain variable.

[0008] In the embodiment of the present application, the preset approximate Wiener filter model is: , in, , is the degradation function, is the signal-to-noise ratio of the image to be restored, is the frequency domain representation of the image to be restored, To restore the image.

[0009] In the embodiment of the present application, the signal-to-noise ratio of the image to be restored is calculated based on the image to be restored: Calculate the square of the average grayscale value of all pixels in the image to be restored to obtain signal power; Calculating the maximum value of the local variance of all pixel grayscales in the image to be restored to obtain noise power; A signal-to-noise ratio of the image to be restored is obtained based on the signal power and the noise power.

[0010] In an embodiment of the present application, the image to be restored is restored based on the degradation function and the signal-to-noise ratio of the image to be restored using a preset approximate Wiener filter model to obtain a restored image, including: Determining whether the signal-to-noise ratio of the image to be restored is greater than a preset threshold; When it is confirmed that the signal-to-noise ratio of the image to be restored is greater than a preset threshold, the image to be restored is restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

[0011] In the embodiment of the present application, it also includes: When it is confirmed that the signal-to-noise ratio of the image to be restored is not greater than a preset threshold, a preset image restoration filter is used to perform image restoration on the image to be restored to obtain a restored image.

[0012] A second aspect of the present application provides an image restoration device, comprising: An acquisition module, used for acquiring an image to be restored and a degradation function corresponding to the image to be restored; A calculation module, used for calculating the signal-to-noise ratio of the image to be restored based on the image to be restored; The restoration module is used to restore the image to be restored by using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

[0013] A third aspect of the present application provides an electronic device, the electronic device comprising: at least one processor; a memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the above-mentioned image restoration method by executing the instructions stored in the memory.

[0014] A fourth aspect of the present application provides a machine-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned image restoration method.

[0015] Through the above technical solution, by obtaining the image to be restored and the degradation function corresponding to the image to be restored, based on the image to be restored, the signal-to-noise ratio of the image to be restored is calculated, and based on the degradation function and the signal-to-noise ratio of the image to be restored, the image to be restored is restored using a preset approximate Wiener filter model to obtain a restored image. The approximate Wiener filter model can restore the image to be restored based on the degradation function and the signal-to-noise ratio of the image to be restored, and no longer requires the power spectrum of the non-degraded image and the noise. It has strong operability, can ensure the correct restoration of the image to be restored, and improves the reliability of image restoration.

[0016] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings: Figure 1A schematic diagram of a process of an image restoration method according to an embodiment of the present application is schematically shown; Figure 2 The schematic diagram of the principle of restoring the original image according to the embodiment of the present application is schematically shown; Figure 3 The following schematically shows a process diagram of image restoration for an excavator operation according to an embodiment of the present application; Figure 4 A schematic diagram of the structure of an image restoration device according to an embodiment of the present application is shown; Figure 5 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown.

[0018] Description of Reference Numerals 410 - acquisition module; 420 - calculation module; 430 - recovery module; A01 - processor; A02 - network interface; A03 - internal memory; A04 - display screen; A05 - input device; A06 - non-volatile storage medium; B01 - operating system; B02 - computer program. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0021] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] Figure 1 The following schematically shows a flow chart of an image restoration method according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides an image restoration method, which can restore the image to be restored by using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored, and no longer requires the power spectrum of the non-degraded image and the noise. This method can restore image blur caused by motion, such as images captured by a surround-view camera when an excavator is operating; images captured by cameras installed on both sides of the vehicle body when the vehicle is driving. In order to facilitate the description of the scheme, this embodiment mainly describes the image restoration of images captured by a surround-view camera when an excavator is operating. It should be noted that the image restoration mentioned in this embodiment The direction is horizontal. The direction is vertical.

[0024] An image restoration method according to an embodiment of the present application comprises the following steps: Step 210: obtaining an image to be restored and a degradation function corresponding to the image to be restored; In this embodiment, the image to be restored may refer to an image captured by an imaging device, and the imaging device may be a camera, etc. For example, a surround view camera is installed on an excavator, and when the excavator is operating, the operating image captured by the surround view camera is the image to be restored.

[0025] Please see Figure 2 , Figure 2 The schematic diagram of the principle of restoring the original image according to the embodiment of the present application is shown schematically. For a grayscale image, when it is still, we can use a The numerical matrix representation of , Represents the image after degradation. The degradation process can be modeled as a degradation function and an additive noise , based on the degradation function and additive noise , for an input image After processing, a degraded image is produced , the degraded image can be processed by constructing a restoration function to restore the original image.

[0026] For example, when a camera mounted on an excavator acquires an image, the image and the camera undergo linear motion. Assuming that the time taken for the shutter to open and close is very short, the linear motion can be approximated as uniform motion. and Respectively represent the displacement in and The component that changes with time in the direction, the exposure time T, the total exposure amount at any point of the recording medium is obtained by integrating the instantaneous exposure in the time interval, then: , in is the degraded image, Perform Fourier transform on the above formula, that is:

[0027] Changing the order of integration, the above formula can be expressed as: , The curly brackets contain the displacement function The Fourier transform of is the frequency domain representation of the degraded image, is a frequency domain variable, then:

[0028] Then, the degradation function .

[0029] Assume that the image is Direction at a given speed To move in a straight line at a uniform speed; Direction at a given speed Perform uniform linear motion. Then the degradation function becomes: , That is, the preset degradation function model is: , in, , is the motion parameter, For time, is the exposure time, is the degradation function, is a frequency domain variable.

[0030] In some embodiments, obtaining the degradation function corresponding to the image to be restored includes: First, obtaining motion information corresponding to the image to be restored, wherein the motion information corresponding to the image to be restored includes at least one type of motion speed; In this embodiment, the motion information corresponding to the image to be restored refers to the motion data when the image to be restored is collected, and the motion data can be one or more types of motion speed. The types of motion speed include rotation speed, forward speed, etc. For example, for a moving vehicle, the camera on the side of the vehicle collects the image to be restored, and at the same time obtains the current forward speed of the vehicle as motion information. For example, when the excavator is working, the surround view camera collects the working image, the excavator track is equipped with an encoder to collect the chassis speed, and the body rotation is equipped with an encoder to collect the rotation angular velocity. Each encoder is connected to the controller of the excavator, and the image host can be connected to the controller through a bus. The controller will transmit the chassis speed, rotation angular velocity and other parameters to the image host as the motion information corresponding to the image to be restored.

[0031] Then, obtaining the image acquisition position of the image to be restored; In this embodiment, the image acquisition position refers to the position where the image to be restored is taken by the imaging device installed, and can be obtained when the image to be restored is acquired. For example, a camera is installed on the body of an excavator. When the excavator is working, the working image collected by the camera on the body is the image to be restored. After obtaining the image to be restored, the image host can determine that the image acquisition position is the body.

[0032] Then, based on the type of the motion speed, a motion state is determined; In this embodiment, the above determination is obtained according to the type of motion speed. For example, when only the forward speed is obtained, the motion state can be determined to be forward. If only the rotational speed is obtained, the motion state can be determined to be rotational.

[0033] Then, based on the motion state, the image acquisition position and the motion speed, the motion parameters are determined according to a preset parameter determination rule; In this embodiment, parameter determination rules under different motion states can be preset, for example: when the motion state is a rotation state, there is a corresponding parameter determination rule 1; when the motion state is a forward state, there is a corresponding parameter determination rule 2. The specific parameter determination rules can be determined according to actual conditions. Among them, the motion parameters include horizontal direction parameters and vertical direction parameters, the horizontal direction parameters are related to the horizontal direction speed, and the vertical direction parameters are related to the vertical direction speed.

[0034] For example, cameras are installed on the front, back, left and right sides of the excavator body. When the excavator is rotating, the cameras around the excavator are The direction of movement is very fast. The speed of movement in the direction is very small, almost no; when the chassis moves forward and backward, the cameras on both sides are The direction of movement is faster, The movement speed in the direction is small, and the front and rear cameras are Direction and The speed of movement in the direction is relatively small, such as some bumps, which will cause the camera to vibrate slightly. The parameter determination rule can set multiple values ​​according to different movement speeds and different image acquisition positions. For example, the parameter determination rule is set to: when the movement state is the rotation state, , ,in, It is the rotation speed in r / min. Usually the maximum rotation speed of the excavator is 10r / min.

[0035] When the movement state is forward and backward, , ,in, It is the forward and reverse speed in km / h. Usually the maximum speed of an excavator is 5km / h.

[0036] In the excavator, the four cameras are divided into two situations: the body rotation and the chassis forward and backward movement. The values ​​are divided into sections according to the rotation speed. According to the minimum value, that is, if the rotation speed is 10 r / min, the above parameter determination rules can be used to obtain is 0.6, is 0.1; the chassis moves forward and backward, for the left and right cameras, According to the speed segment value, According to the minimum value, that is, if the forward and backward speed is 4 km / h, the above parameter determination rules can be used to obtain is 0.5, is 0.1. The chassis moves forward and backward. For the front and rear cameras, and According to the minimum value, that is is 0.1, is 0.1.

[0037] It should be noted that, in the specific implementation, there may be multiple movements at the same time, that is, compound movement. For compound movement, the obtained compound speed can be first decomposed into the image movement direction, and then the corresponding speed can be matched according to the speed of the image movement direction. and The value of Add together to get the final and , the direction needs to be considered during the superposition process. If the directions are the same, they are added, and if the directions are opposite, they are subtracted.

[0038] For example, when the excavator body is in the upright position, it rotates to the right with an angular velocity of 6r / min while moving forward at a speed of 2 km / h. According to the above parameter determination rules, for the left camera, , ; For the right camera, , ; For front and rear cameras, , When the excavator body and chassis form a certain angle, it is necessary to further consider speed decomposition. First, the forward speed of the chassis needs to be decomposed into the image movement direction in the image coordinate system, and then the corresponding speed is matched according to the speed of the image movement direction. and , and then the corresponding Compared with the state of the excavator body being straightened, the forward speed after decomposition will become smaller. Correspondingly, for the left camera and the front camera The value will also decrease for the right camera and the rear camera. The value will increase.

[0039] Finally, the motion parameters are substituted into a preset degradation function model to obtain a degradation function.

[0040] In this embodiment, after the motion parameters are obtained, the values ​​of the motion parameters are substituted into a preset degradation function model to obtain a degradation function.

[0041] By determining the image acquisition position based on the image to be restored; determining the motion state based on the type of motion speed in the motion information corresponding to the image to be restored; determining the motion parameters according to the preset parameter determination rules based on the motion state, the image acquisition position and the motion speed; substituting the motion parameters into the preset degradation function model to obtain the degradation function. For the image to be restored acquired by the imaging device at different positions, different degradation functions correspond to different motion states, which helps to restore the image to be restored more accurately.

[0042] Step 220: Based on the image to be restored, calculate the signal-to-noise ratio of the image to be restored; In this embodiment, the image signal-to-noise ratio (SNR) is the ratio of the signal power to the noise power in the image. The signal power reflects the strength of the useful information in the image, while the noise power represents the strength of the random factors that interfere with the image quality.

[0043] The image signal-to-noise ratio is the ratio of the power spectrum of the signal to the power spectrum of the noise, but the power spectrum is rarely known and difficult to estimate. Therefore, in some embodiments, the signal-to-noise ratio of the image to be restored is calculated based on the image to be restored, including: First, the square of the average grayscale value of all pixels in the image to be restored is calculated to obtain the signal power; In this embodiment, the signal power can be calculated by The square of the average grayscale value of all pixels can be expressed as ,in is an image The average grayscale value of all pixels. The grayscales of all pixels can be obtained after obtaining the image to be restored. The above calculation process can be obtained by using the existing calculation method, which will not be described in detail here.

[0044] Then, the maximum value of the local variance of all pixel grayscales in the image to be restored is calculated to obtain the noise power; In this embodiment, the noise power can be calculated by The local variance maximum of all pixel grayscales is obtained, which can be expressed as For an image, local variance refers to the degree of discreteness of pixel values ​​in a local area. Specifically, for a pixel point in an image, its local area can be defined as a neighborhood centered on the pixel point, and the local variance is the average of the squares of the difference between the pixel value in this neighborhood and the average value of the neighborhood pixels. The calculation of the above local variance can be obtained using existing calculation methods, which will not be repeated here.

[0045] Finally, based on the signal power and the noise power, a signal-to-noise ratio of the image to be restored is obtained.

[0046] In this embodiment, the signal-to-noise ratio of the image to be restored can be expressed as .

[0047] The signal power is obtained by calculating the square of the average grayscale value of all pixels in the image to be restored, and the noise power is obtained by calculating the maximum value of the local variance of the grayscale value of all pixels in the image to be restored. Then, an approximate method is used to solve the signal-to-noise ratio of the image to be restored. This method does not use the power spectrum, and can quickly determine the signal-to-noise ratio, thereby further improving the operability of image restoration.

[0048] Step 230: Based on the degradation function and the signal-to-noise ratio of the image to be restored, a preset approximate Wiener filter model is used to perform image restoration on the image to be restored to obtain a restored image.

[0049] In this embodiment, the Wiener filter is used to find the original image An estimate of , so that the mean square error between them is minimized. This error metric is given by the following formula:

[0050] in, is the mean square error, is the expectation of the parameters. Based on this condition, the expression of the Wiener filter after filtering can be obtained as follows: , , for conjugation of; , for conjugation of; , for conjugation of; in, is the degradation function, is the power spectrum of the noise, is the power spectrum of the undegraded image, is the frequency domain representation of the noise, is the frequency domain representation of the undegraded image, To restore the image.

[0051] However, the power spectrum of the noise and the power spectrum of the undegraded image cannot be known, so the expression after the Wiener filter filtering can be approximately expressed, that is, the preset approximate Wiener filter model is: , in, , is the degradation function, is the signal-to-noise ratio of the image to be restored, is the frequency domain representation of the undegraded image, that is, the frequency domain representation of the image to be restored, To restore the image.

[0052] It can be seen from the above expression that the variables in the formula are only the degradation function and the signal-to-noise ratio of the image to be restored. Therefore, after obtaining the degradation function and the signal-to-noise ratio of the image to be restored, substituting the degradation function and the signal-to-noise ratio of the image to be restored into the above approximate Wiener filter model can obtain the restored image to achieve image restoration.

[0053] In some embodiments, the image restoration of the image to be restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image includes: Determining whether the signal-to-noise ratio of the image to be restored is greater than a preset threshold; In this embodiment, considering that the approximate Wiener filter model performs better in low-noise conditions, a threshold can be set in advance, and the threshold can be set according to actual needs. Then, the signal-to-noise ratio of the image to be restored is compared with the threshold. If the signal-to-noise ratio is greater than the threshold, it means that the image to be restored has less noise. Otherwise, it means that the image to be restored is in a high-noise situation.

[0054] When it is confirmed that the signal-to-noise ratio of the image to be restored is greater than a preset threshold, the image to be restored is restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

[0055] In this embodiment, if the image to be restored is low-noise, a preset approximate Wiener filter model is used to perform image restoration on the image to be restored.

[0056] When it is confirmed that the signal-to-noise ratio of the image to be restored is not greater than a preset threshold, a preset image restoration filter is used to perform image restoration on the image to be restored to obtain a restored image.

[0057] In this embodiment, if the image to be restored has a large noise, that is, it belongs to the high noise and medium noise conditions, a preset image restoration filter can be used to restore the image to be restored. The above preset image restoration filter can be set according to actual needs. The preset image restoration filter can be a filter with a good filtering effect in the case of high noise and medium noise, such as a filter designed by a constrained least squares filtering method, a filter designed by a Kalman filtering method, etc.

[0058] Taking the constrained least squares filtering method as an example, in order to reduce the noise sensitivity problem, it is based on the optimal restoration of smoothing measures, such as calculating the second-order derivative of the image (Laplace transform operator), but the restoration must be constrained by the relevant parameters. Therefore, a minimum criterion function can be defined as follows: , The constraints are: , in is the Euclidean vector norm.

[0059] The frequency domain solution to this optimization problem is given by the following expression: , in is a parameter that must satisfy the constraints. is the Fourier transform of the Laplace transform operator.

[0060] To get the best optimization, you need to adjust the parameters To satisfy the constraints. You can first define a residual vector: , because yes So yes function, it can be proved that: , is about A monotonically increasing function, adjusting So that: ,in is a precision factor. Can be Make an estimate: .

[0061] The adjustment steps are as follows: 1. Designate The initial value of 2. According to calculate ,calculate , in order to calculate , we can first calculate : ,right Perform inverse Fourier transform to get , .

[0062] 3. If , end; otherwise, if , increase Otherwise, if , reduce ; Return to step 2.

[0063] For high noise and medium noise conditions, the preset image restoration filter has a better image restoration effect; the approximate Wiener filtering model performs better in low noise conditions. Combining the two filtering methods, when the signal-to-noise ratio of the image to be restored is not greater than a certain threshold, the preset image restoration filter is selected, otherwise the approximate Wiener filtering model is used for image restoration. Therefore, different image restoration methods can be used for different noises, thereby improving the image restoration accuracy.

[0064] The following is a specific example to illustrate the solution. Figure 3 , Figure 3 The following is a schematic diagram of the excavator operation image restoration process according to an embodiment of the present application.

[0065] The excavator is equipped with a surround view camera. When the excavator is working, the camera will rotate with the excavator body. First, the excavator status and the image collected by the camera are obtained, and then a degradation function is set for each camera. Then the signal-to-noise ratio of the collected image is approximately calculated. Different restoration filters are selected according to the signal-to-noise ratio, and then the image is restored and filtered using the selected restoration filter. Combining the advantages of the approximate Wiener filter model and the least squares filter, it is possible to restore the image motion blur caused by the surrounding cameras during the operation of the excavator.

[0066] In the above implementation process, by obtaining the image to be restored and the degradation function corresponding to the image to be restored, the signal-to-noise ratio of the image to be restored is calculated based on the image to be restored, and based on the degradation function and the signal-to-noise ratio of the image to be restored, the image to be restored is restored using a preset approximate Wiener filter model to obtain a restored image. The approximate Wiener filter model can restore the image to be restored based on the degradation function and the signal-to-noise ratio of the image to be restored, and no longer requires the power spectrum of the non-degraded image and the noise. It has strong operability, can ensure the correct restoration of the image to be restored, and improves the reliability of image restoration.

[0067] Figure 1 FIG. 1 is a flow chart of the image restoration method in the embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0068] Please see Figure 4 , Figure 4 The schematic diagram of the structure of an image restoration device according to an embodiment of the present application is shown schematically. This embodiment provides an image restoration device, including an acquisition module 410, a calculation module 420 and a restoration module 430, wherein: An acquisition module 410 is used to acquire an image to be restored and a degradation function corresponding to the image to be restored; A calculation module 420, configured to calculate a signal-to-noise ratio of the image to be restored based on the image to be restored; The restoration module 430 is used to restore the image to be restored by using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored, so as to obtain a restored image.

[0069] The image restoration method device includes a processor and a memory. The acquisition module 410, the calculation module 420 and the restoration module 430 are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.

[0070] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and image restoration can be achieved by adjusting kernel parameters.

[0071] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0072] An embodiment of the present invention provides a machine-readable storage medium on which a program is stored. When the program is executed by a processor, the image restoration method is implemented.

[0073] An embodiment of the present invention provides a processor, which is used to run a program, wherein the image restoration method is executed when the program is run.

[0074] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, an image restoration method is implemented. The display screen A04 of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0075] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0076] In one embodiment, the image restoration method and apparatus provided in the present application can be implemented in the form of a computer program. The computer program can be Figure 5 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the image restoration method device, such as: Figure 4 The acquisition module 410, the calculation module 420 and the restoration module 430 are shown. The computer program composed of various program modules enables the processor to execute the steps of the image restoration method of each embodiment of the present application described in this specification.

[0077] Figure 5 The computer device shown can be Figure 4 The acquisition module 410 in the image restoration method and apparatus shown executes step 210. The computer device may execute step 220 through the calculation module 420. The computer device may execute step 230 through the restoration module 430.

[0078] An embodiment of the present application provides an electronic device, the electronic device comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, the at least one processor implements the above-mentioned image restoration method by executing the instructions stored in the memory, and the processor implements the following steps when executing the instructions: Acquire an image to be restored and a degradation function corresponding to the image to be restored; Based on the image to be restored, calculating a signal-to-noise ratio of the image to be restored; Based on the degradation function and the signal-to-noise ratio of the image to be restored, a preset approximate Wiener filter model is used to perform image restoration on the image to be restored to obtain a restored image.

[0079] In one embodiment, obtaining the degradation function corresponding to the image to be restored includes: Acquiring motion information corresponding to the image to be restored, where the motion information corresponding to the image to be restored includes at least one type of motion speed; Acquire an image acquisition position of the image to be restored; Based on the type of the motion speed, determining a motion state; Based on the motion state, the image acquisition position and the motion speed, the motion parameter is determined according to a preset parameter determination rule; Substituting the motion parameters into a preset degradation function model, a degradation function is obtained.

[0080] In one embodiment, the preset degradation function model is: , in, , is the motion parameter, For time, is the exposure time, is the degradation function, is a frequency domain variable.

[0081] In one embodiment, the preset approximate Wiener filter model is: , in, , is the degradation function, is the signal-to-noise ratio of the image to be restored, is the frequency domain representation of the image to be restored, To restore the image.

[0082] In one embodiment, the signal-to-noise ratio of the image to be restored is calculated based on the image to be restored: Calculate the square of the average grayscale value of all pixels in the image to be restored to obtain signal power; Calculating the maximum value of the local variance of all pixel grayscales in the image to be restored to obtain noise power; A signal-to-noise ratio of the image to be restored is obtained based on the signal power and the noise power.

[0083] In one embodiment, the image restoration to be restored is performed on the image to be restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image, including: Determining whether the signal-to-noise ratio of the image to be restored is greater than a preset threshold; When it is confirmed that the signal-to-noise ratio of the image to be restored is greater than a preset threshold, the image to be restored is restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

[0084] In one embodiment, it further includes: When it is confirmed that the signal-to-noise ratio of the image to be restored is not greater than a preset threshold, a preset image restoration filter is used to perform image restoration on the image to be restored to obtain a restored image.

[0085] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0086] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0089] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0090] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0091] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0092] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0093] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image restoration method, characterized in that: include: Acquire an image to be restored and a degradation function corresponding to the image to be restored; Based on the image to be restored, calculating a signal-to-noise ratio of the image to be restored; Based on the degradation function and the signal-to-noise ratio of the image to be restored, a preset approximate Wiener filter model is used to perform image restoration on the image to be restored to obtain a restored image.

2. The method according to claim 1, characterized in that Obtaining a degradation function corresponding to the image to be restored, including: Acquiring motion information corresponding to the image to be restored, where the motion information corresponding to the image to be restored includes at least one type of motion speed; Acquire an image acquisition position of the image to be restored; Based on the type of the motion speed, determining a motion state; Based on the motion state, the image acquisition position and the motion speed, the motion parameter is determined according to a preset parameter determination rule; Substituting the motion parameters into a preset degradation function model, a degradation function is obtained.

3. The method according to claim 2, characterized in that The preset degradation function model is: ， in, , is the motion parameter, For time, is the exposure time, is the degradation function, is a frequency domain variable.

4. The method according to claim 1, characterized in that The preset approximate Wiener filter model is: ， in, , is the degradation function, is the signal-to-noise ratio of the image to be restored, is the frequency domain representation of the image to be restored, To restore the image.

5. The method according to claim 1, characterized in that The signal-to-noise ratio of the image to be restored is calculated based on the image to be restored: Calculate the square of the average grayscale value of all pixels in the image to be restored to obtain signal power; Calculating the maximum value of the local variance of all pixel grayscales in the image to be restored to obtain noise power; A signal-to-noise ratio of the image to be restored is obtained based on the signal power and the noise power.

6. The method according to claim 1, characterized in that The method of restoring the image to be restored by using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image includes: Determining whether the signal-to-noise ratio of the image to be restored is greater than a preset threshold; When it is confirmed that the signal-to-noise ratio of the image to be restored is greater than a preset threshold, the image to be restored is restored using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

7. The method according to claim 6, characterized in that Also includes: When it is confirmed that the signal-to-noise ratio of the image to be restored is not greater than a preset threshold, a preset image restoration filter is used to perform image restoration on the image to be restored to obtain a restored image.

8. An image restoration device, characterized in that: include: An acquisition module, used for acquiring an image to be restored and a degradation function corresponding to the image to be restored; A calculation module, used for calculating the signal-to-noise ratio of the image to be restored based on the image to be restored; The restoration module is used to restore the image to be restored by using a preset approximate Wiener filter model based on the degradation function and the signal-to-noise ratio of the image to be restored to obtain a restored image.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; a memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the image restoration method according to any one of claims 1 to 7 by executing the instructions stored in the memory.

10. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to perform the image restoration method according to any one of claims 1 to 7.

Images