Noise Adaptive Non-Blind Image Deblurring

Abstract

Systems and methods for performing noise adaptive non-blind deblurring on an input image that includes blur and noise include applying a first neural network to the input image to obtain one or more parameters and performing regularized deconvolution to obtain a deblurred image from the input image. The regularized deconvolution uses the one or more parameters to control noise in the deblurred image. One method includes applying a second neural network to remove artifacts from the deblurred image and provide an output image.

Description

Technical Field

[0001] The present subject matter disclosure generally relates to image deblurring, and more particularly, to noise adaptive non-blind image deblurring. Background Art

[0002] Vehicles (e.g., cars, trucks, agricultural equipment, construction equipment, automated factory equipment) can include many sensors that provide information about the vehicle and its environment. An exemplary sensor is a camera. For example, images obtained by one or more cameras of a vehicle can be used to perform semi-autonomous or autonomous operations. Images obtained by a camera may be blurred for various reasons, including movement or vibration of the camera. In vehicle applications, based on the known movement of the vehicle or calibration performed on the camera, the source of the blur may be well known. This facilitates non-blind image deblurring. However, blurred images typically include noise and blur. Therefore, it is desirable to provide noise adaptive non-blind image deblurring. Summary of the Invention

[0003] In one exemplary embodiment, a method for performing noise adaptive non-blind deblurring on an input image including blur and noise includes implementing a first neural network on the input image to obtain one or more parameters, and performing regularized deconvolution to obtain a deblurred image from the input image. The regularized deconvolution uses one or more parameters to control noise in the deblurred image. The method further includes implementing a second neural network to remove artifacts from the deblurred image and provide an output image.

[0004] In addition to one or more features described herein, implementing the first neural network results in one parameter that is a regularization parameter.

[0005] In addition to one or more features described herein, implementing the first neural network results in two or more parameters that are weights corresponding to a set of predefined regularization parameters.

[0006] In addition to one or more features described herein, the method further includes training the first neural network and the second neural network alone or together in an end-to-end manner.

[0007] In addition to one or more features described herein, the method further includes obtaining, by a processing circuit, a point spread function that defines the blur in the input image.

[0008] In addition to one or more features described herein, the input image is obtained by a camera in a vehicle, and the point spread function is obtained from one or more sensors of the vehicle or from the camera based on calibration.

[0009] In addition to one or more features described herein, implementing the first neural network includes obtaining a one-dimensional vector of singular values from an input image and implementing a one-dimensional residual convolutional neural network.

[0010] In another exemplary embodiment, a non-transitory computer-readable storage medium stores instructions that, when processed by a processing circuit, cause the processing circuit to implement a method for performing noise-adaptive non-blind deblurring on an input image that includes blur and noise. The method includes implementing a first neural network on the input image to obtain one or more parameters, and performing regularized deconvolution to obtain a deblurred image from the input image. The regularized deconvolution uses one or more parameters to control noise in the deblurred image. The method further includes implementing a second neural network to remove artifacts from the deblurred image and provide an output image.

[0011] In addition to one or more features described herein, implementing the first neural network results in one parameter that is a regularization parameter.

[0012] In addition to one or more features described herein, implementing the first neural network results in two or more parameters that are weights corresponding to a set of predefined regularization parameters.

[0013] In addition to one or more features described herein, the method further includes training the first neural network and the second neural network either alone or together in an end-to-end manner.

[0014] In addition to one or more features described herein, the method further includes obtaining, by the processing circuit, a point spread function that defines the blur in the input image.

[0015] In addition to one or more features described herein, the input image is obtained by a camera in a vehicle, and the point spread function is obtained based on calibration from one or more sensors of the vehicle or from the camera.

[0016] In addition to one or more features described herein, implementing the first neural network includes obtaining a one-dimensional vector of singular values from an input image and implementing a one-dimensional residual convolutional neural network.

[0017] In yet another exemplary embodiment, a vehicle includes a camera to obtain an input image that includes blur and noise. The vehicle further includes a processing circuit for implementing a first neural network on the input image to obtain one or more parameters, and performing regularized deconvolution to obtain a deblurred image from the input image. The regularized deconvolution uses one or more parameters to control noise in the deblurred image. The processing circuit further implements a second neural network to remove artifacts from the deblurred image and provide an output image.

[0018] In addition to one or more features described herein, the processing circuit implements a first neural network and obtains one parameter as a regularization parameter, or obtains two or more parameters as weights corresponding to a set of predefined regularization parameters.

[0019] In addition to one or more features described herein, the processing circuit trains the first neural network and the second neural network separately or together in an end-to-end manner.

[0020] In addition to one or more features described herein, the processing circuit obtains a point spread function that defines blurring in the input image.

[0021] In addition to one or more features described herein, the processing circuit obtains the point spread function from one or more vehicle sensors that measure vehicle motion or from the calibration of a camera.

[0022] In addition to one or more features described herein, the first neural network obtains a one-dimensional vector of singular values from the input image and implements a one-dimensional residual convolutional neural network (CNN).

[0023] When combined with the accompanying drawings, the above and other features and advantages of the present disclosure will become apparent from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, advantages, and details appear only by way of example in the following detailed description, which refers to the accompanying drawings, in which:

[0025] Figure 1 is a block diagram of a vehicle that performs noise adaptive non-blind image deblurring according to one or more embodiments;

[0026] Figure 2 shows an exemplary image that illustrates the process of noise adaptive non-blind image deblurring according to one or more embodiments;

[0027] Figure 3 shows components of a training process of a system that performs noise adaptive non-blind image deblurring according to one or more embodiments;

[0028] Figure 4 shows the architecture of a first neural network for performing noise adaptive non-blind image deblurring according to one or more embodiments;

[0029] Figure 5 shows a process flow for training a first neural network for performing noise adaptive non-blind image deblurring according to one or more embodiments;

[0030] Figure 6illustrates a processing flow for training a first neural network for performing noise - adaptive non - blind image de - blurring according to one or more embodiments;

[0031] Figure 7 illustrates an additional process required to generate a ground truth for training the first neural network when the blur is two - dimensional;

[0032] Figure 8 illustrates an exemplary process flow for end - to - end training of a neural network for performing noise - adaptive non - blind image de - blurring according to one or more embodiments;

[0033] Figure 9 illustrates an exemplary process flow for end - to - end training of a neural network for performing noise - adaptive non - blind image de - blurring according to one or more embodiments; and

[0034] Figure 10 is a block diagram of a system for performing noise - adaptive non - blind image de - blurring according to one or more embodiments. DETAILED DESCRIPTION

[0035] The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application, or uses. It should be understood that in all the figures, corresponding reference numerals represent like or corresponding parts and features.

[0036] As previously mentioned, images obtained by a camera may be blurred. In vehicle applications, camera movement or vibration may cause the images obtained by the camera to be blurred. Non - blind de - blurring of a blurred image refers to the situation where the blur source and the blur model are known. Even when the function or model of the blur is known, non - blind de - blurring is an unstable problem, and boundary conditions must be imposed to solve artifacts. That is, existing de - blurring processes may introduce artifacts. Additionally, if the de - blurring process is not regularized, noise may be amplified. Existing methods help to solve known or fixed noise in the non - blind de - blurring process. Specifically, a joint training process is employed to determine the parameters of regularized de - convolution and the weights of a convolutional neural network.

[0037] Embodiments of the systems and methods detailed herein relate to noise adaptive non-blind image deblurring. A first neural network (e.g., a deep neural network) infers noise-related regularization parameters to be used in a regularized deconvolution process to produce a deblurred image with artifacts. According to an exemplary embodiment, the first neural network provides a regularization parameter value λ. According to another exemplary embodiment, the first neural network provides weights associated with each value in a predefined array of regularization parameter values λ. Using the correct regularization parameter value λ during regularized deconvolution ensures that the noise in the input (blurred) image is not amplified in an uncontrolled manner in the deblurred image. A second neural network (e.g., a CNN) then removes the artifacts from the deblurred image. The correct value of the regularization parameter λ provided by the first neural network ensures that the value is not too small to be useful (i.e., the output image is too noisy), but not so large that the output image remains blurred. According to existing methods, the separate first neural network is not used.

[0038] According to an exemplary embodiment, Figure 1 is a block diagram of a vehicle 100 that performs noise adaptive non-blind image deblurring. Figure 1 The exemplary vehicle 100 shown is an automobile 101. Two exemplary cameras 110 are shown to acquire images from the front of the vehicle 100. Each camera 110 can be a color camera or a grayscale camera or any other imaging device operating in the visible or infrared spectrum. The images acquired with the cameras 110 can be one-dimensional, two-dimensional, or three-dimensional images ( Figure 2 ) that are used as the blurred input image 210.

[0039] The vehicle 100 is also shown to include a controller 120 and additional sensors 130, 140. The additional sensor 130 (e.g., an inertial measurement unit, a wheel speed sensor, a gyroscope, an accelerometer) acquires information about the vehicle 100, while the additional sensor 140 (e.g., a lidar system, a radar system) acquires information about its surrounding environment. The controller 120 can use the information from one or more of the sensors 130, 140, and the cameras 110 to perform semi-autonomous or autonomous operation of the vehicle 100.

[0040] According to one or more embodiments, the controller 120 performs noise-adaptive non-blind image deblurring on the blurred input image 210 obtained by one or more cameras 110. Alternatively, the camera 110 may include a controller to perform the processing. In either case, noise-adaptive non-blind image deblurring requires knowledge of the blur source. The blur source may be the motion of the vehicle 100, indicated by parameters obtained by the sensors 130 of the vehicle 100, or may be inherent to the camera 110, as determined by the calibration of the camera 110. The non-blind aspect of the deblurring process is known and will not be described in detail here. The controller 120 and any controller of the camera 110 may include processing circuitry, which may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or grouped), and a memory that executes one or more software or firmware programs, combinatorial logic circuitry, and / or other suitable components that provide the described functionality.

[0041] Figure 2 An exemplary image is shown that illustrates the process of noise-adaptive non-blind image deblurring according to one or more embodiments. The input image 210 is a blurred image with noise. Based on regularized deconvolution (at block 360 ( Figure 3 )) using a regularization parameter λ obtained using a first neural network 355 (implemented at block 350 ( Figure 3 )) a deblurred image 220 is obtained. This deblurred image 220 may include artifacts 225. Then a second neural network 375 (implemented at block 370 ( Figure 3 )) is implemented to obtain an output image 230 having the artifacts 225 removed from the deblurred image 220.

[0042] Figure 3 The components of a training process 300 of a system 301 that performs noise-adaptive non-blind image deblurring according to one or more embodiments are shown. As detailed and noted with reference to Figure 2 the first neural network 355 (implemented at block 350) helps to obtain a noise-related regularization parameter. According to an alternative embodiment, a single value of the regularization parameter λ or weights corresponding to a predefined array of values of the regularization parameter λ are provided by the first neural network 355. In either form, the regularization parameter λ is used to control the noise in the regularized deconvolution (at block 360), which provides the deblurred image 220. The second neural network 375 (implemented at block 370) helps to remove artifacts from the deblurred image 220 to obtain the output image 230.

[0043] At block 310, obtaining a clear image 315 (denoted as Im) refers to obtaining an image, whether in real time or from a database, without blur or noise. For clarity, the training process 300 uses a large set of clear images Im 315 over multiple iterations. The clear images Im 315 represent the ground truth used in the training process 300. That is, ideally, the output image 230 will be very close to this clear image Im 315.

[0044] At block 320, performing corruption refers to generating noise N and a point spread function (PSF), both of which are applied to the clear image Im 315 to generate the input image 210 (denoted as I B ) of the system 301. Each neural network can be trained individually or together in a process called end-to-end training. Refer to Figures 5 - 9 an exemplary training process discussing the end-to-end training of the first neural network 355 or the entire system 301. The PSF output at block 320 represents a potential source of image blur obtained by the camera 110 in the vehicle 100. The PSF can be based on motion parameters obtained by the sensors 130 of the vehicle 100 or, when the blur is inherent to the camera 110, can be measured based on the calibration of the camera 110.

[0045] At block 330, the PSF is used to determine the blur (i.e., generate the blur kernel matrix K B ). As discussed in reference Figure 3 , the system 301 obtains a blurred image from the camera 110 (at block 340) and determines the blur based on information from the sensors 130 or the camera 110 (at block 330). Since the blur is determined by a known PSF (at block 330), the system 301 performs non-blind deblurring. Since the noise N is unknown, the system 301 performs noise-adaptive deblurring. As Figure 3 shown, at block 340, obtaining the blurred input image I B 210 includes applying the blur (determined at block 330) to the clear image Im 315 and adding the noise N. This is an artificial process for creating the input image I B 210. As Figure 3 shown, in training the system 301, the noise and blur are part of the output of the camera 110. The input image IB 210 is given by:

[0046] I B = Im * K B + N [Equation 1]

[0047] At block 350, implementing the first neural network 355 results in determining a regularization parameter λ. According to an alternative embodiment, at block 350, implementing the first neural network 355 can result in an output of a value of the regularization parameter λ, or can result in an output of weights corresponding to a predetermined set of values of the regularization parameter λ. In the latter case, the weights of the set add up to 1. Refer to Figure 4 Discussing the architecture 400 of the first neural network 355, refer to Figures 5 - 7 Discussing the training of the first neural network 355.

[0048] At block 360, according to an alternative embodiment, regularization deconvolution based on the input image I B 210 and the regularization parameter λ can be performed to generate a deblurred image 220. For purposes of explanation, it is assumed that the first neural network 355 (at block 350) provides a value of the regularization parameter λ rather than weights. According to an exemplary embodiment, when the input image I B 210 exhibits one-dimensional blur (e.g., horizontal blur), Tikhonov regularization deconvolution can be used. In this case, the singular value decomposition (SVD) of the blur kernel matrix K B (determined at block 330) is performed to generate a decomposition matrix:

[0049] K B = USV T [Equation 2]

[0050] In Equation 2, T represents the transpose. Then, at block 360, based on the decomposition matrix from Equation 2 and the regularization parameter λ from block 350, the deblurred image 220 denoted as I DB is obtained as follows:

[0051]

[0052]

[0053] At block 360, according to an alternative embodiment, when the input image I B 210 includes two-dimensional blur, Wiener deconvolution can be performed. In this case,

[0054]

[0055]

[0056] The parameters shown in Equation 5 are the result of a fast Fourier transform. That is, since two-dimensional blur rather than one-dimensional blur must be considered, the equation is in the Fourier space, as shown by the vector k, rather than in the real space. For example, the FFT is performed on the input image I B 210 to obtain Deblurred image I DB 220 is obtained as follows:

[0057]

[0058]

[0059] Based on Equation 8, the deblurred image I DB 220 is obtained by performing an inverse FFT (IFFT).

[0060] At block 370, applying a second neural network 375 to the deblurred image I DB 220 results in an output image 230. Removing artifacts from the deblurred image I DB 220 and the image enhancement neural network, which is indicated as the second neural network 375, are well known and will not be described in detail herein. Refer to Figure 8 and 9 for a discussion of end-to-end training, which refers to training the first neural network 355 and the second neural network 375 together. As Figure 3 shown, a mean squared error (MSE) can be obtained between the output image 230 provided by the system 301 and the clear image Im 315 to determine the effectiveness of the noise adaptive non-blind deblurring performed by the system 301.

[0061] Figure 4 An architecture 400 of a first neural network 355 for performing noise adaptive non-blind image deblurring according to one or more embodiments is shown. The first neural network 355 is a one-dimensional residual CNN. The input to the first neural network 355 is an input image I B 210, which is a blurred image with noise, and the output N out can be a value of the regularization parameter λ, or can be a set of weights corresponding to a predefined set of values of the regularization parameter λ. At 401, a singular value decomposition is performed on the input image I B 210 to obtain a one-dimensional vector of the logarithm of the image singular values (SV). The first convolutional layer 405 converts the input into 64 feature vectors. The next four stages 410 are a cascade of five residual blocks. Although five residual blocks are indicated for each stage 410, in alternative embodiments, the exemplary embodiments of the architecture 400 do not limit to other numbers of sub-units.

[0062] As a known operation as part of each cascade 420, such as Figure 4As shown by the exemplary cascade 420 in, it includes "Conv1d", which refers to a filter that slides along the data in one dimension, "BatchNorm", which refers to the type of batch normalization of a layer, and "ReLU", which refers to a rectified linear unit. The number of filters N f can be 64, 128, 256, or 512, as shown for different stages 410. As shown at 430, after each cascade 420, there are a feature number doubling convolutional layer and a max pooling layer. At the output, 1024 feature vectors are fed into the fully connected layer "FC" to produce the output N out .

[0063] Figure 5 and 6 Details the training of a first neural network 355 for performing noise-adaptive non-blind image deblurring according to one or more embodiments. As previously mentioned, end-to-end training refers to training two neural networks according to the arrangement shown in FIG. 3. According to Figure 5 or Figure 6 shown in the alternative embodiment, the first neural network 355 can be trained separately from the second neural network 375. When training the first neural network 355 alone, the ground truth is obtained through the function Q(λ), as described below.

[0064] Figure 5 Shows a process flow 500 for training a first neural network 355 for performing noise-adaptive non-blind image deblurring according to one or more embodiments. Figure 5 The shown process flow 500 is for one-dimensional blur and when the output of the first neural network 355 is the regularization parameter λ value. The process of obtaining the clear image Im 315 and the blurred input image I B 210 described previously in boxes 310 to 340 will no longer be discussed. At box 510, a set of images is obtained as the regularization deconvolution result for a set of values of the regularization parameter λ. At box 520, the function Q(λ) selects the best regularization parameter λ, λopt, from the set of values of the regularization parameter λ. That is, at box 520, the function Q(λ) obtains the mean square distance (MSD) between each of the set of images (generated using a set of values of the regularization parameter λ) and the clear image Im 315, and selects the regularization parameter λ corresponding to the image that results in the minimum MSD as λopt.

[0065] At box 530, the input image I B210 is subjected to singular value decomposition to generate a decomposition matrix, similar to Equation 2. At block 350, according to an exemplary embodiment, implementing the first neural network 355 results in a single regularization parameter λreg. For a more precise comparison, the logarithmic magnitudes of the regularization parameter λreg and the optimal regularization parameter λopt are compared based on the MSE. The process flow 500 can be repeated for a large set of clear images Im 315 to train the first neural network 355.

[0066] Figure 6 A process flow 600 for training a first neural network 355 for performing noise-adaptive non-blind image deblurring according to one or more embodiments is shown. Figure 6 The process flow 600 shown is for one-dimensional blur and when the output of the first neural network 355 is a set of weights corresponding to a set of predefined values of the regularization parameter λ. As in the Figure 5 discussion, the process of obtaining the clear image Im 315 and the blurred input image I in blocks 310 to 340 described previously is not discussed again. At block 610, a set of images is obtained as the result of regularized deconvolution. Each image in the set is generated by a set of specific weights corresponding to a set of predefined regularization parameter λ values. B At block 620, the function Q(λ) selects a set of weights that result in an image with the minimum MSD relative to the clear image Im 315. Then, the SoftMin function readjusts the weights to ensure that they are between 0 and 1. The result is a weighting function g(λ) such that the area under the curve adds up to 1. At block 530, the input image I

[0067] 210 is subjected to SVD to generate a decomposition matrix, similar to Equation 2 (refer to the B discussion). At block 350, according to an exemplary embodiment, implementing the first neural network 355 results in a set of weights represented as a function f(λ). At block 640, the weighted sums of the images are compared according to the functions g(λ) and f(λ). The process flow 600 can be repeated for a large set of clear images Im 315 to train the first neural network 355. Figure 5 discussion). At block 350, according to an exemplary embodiment, implementing the first neural network 355 results in a set of weights represented as a function f(λ). At block 640, the weighted sums of the images are compared according to the functions g(λ) and f(λ). The process flow 600 can be repeated for a large set of clear images Im 315 to train the first neural network 355.

[0068] Figure 7Shows the additional process 710 required to generate the ground truth for training the first neural network 355 when the blur is two-dimensional. The processes discussed previously will not be elaborated further. The additional process 710 includes performing a fast Fourier transform on the clear image Im 315 that is the input to the additional process 710. The additional process 710 also includes an IFFT at the output of the additional process 710. At block 720, a regularized deconvolution result is obtained in the Fourier space. The additional process 710 can be used to train the first neural network 355 for two-dimensional blur, regardless of whether the first neural network 355 provides a single regularization parameter λ or weights for a predefined set of values of the regularization parameter λ.

[0069] Figure 8 Shows an exemplary process flow 800 for end-to-end training of a neural network for performing noise-adaptive non-blind image deblurring according to one or more embodiments. The exemplary process flow 800 is used when the first neural network 355 outputs a regularization parameter λ. The exponentiation at block 810 of the log results provides the regularization parameter λ used for regularized deconvolution at block 360. Bypass 820 is a preprocessing bypass and allows bypassing the second neural network 375 (implemented at block 370). This facilitates comparing the deblurred image I DB 220 with the clear image Im 315 at block 380. When bypass 820 is not used, the output image 230 is compared with the clear image Im 315, such that the results of both the first neural network 355 (implemented at block 350) and the second neural network 375 (implemented at block 370) are verified as part of the overall system 301.

[0070] Figure 9 Shows an exemplary process flow 900 for end-to-end training of a neural network for performing noise-adaptive non-blind image deblurring according to one or more embodiments. The exemplary process flow 900 is used when the first neural network 355 outputs a set of weights corresponding to a predefined set of values of the regularization parameter λ. As Figure 9 shown, implementing the first neural network 355 at block 350 results in weights as a function f(λ) of a predefined set of values of the regularization parameter λ. At block 610, a set of deconvolved images is generated, each deconvolved image being produced by a different one of the predefined set of values of the regularization parameter λ. At block 910, a weighted sum of the deconvolved images (from block 610) is obtained based on the weights obtained from the first neural network 355.

[0071] Similar to Figure 8 the bypass 820 in Figure 9 the preprocessing bypass 920 in DB220 is compared with the sharp image Im 315. When the bypass 920 is not used, the output image 230 is compared with the sharp image Im 315 such that the results of both the first neural network 355 (implemented at block 350) and the second neural network 375 (implemented at block 370) are verified as part of the overall system 301.

[0072] Figure 10 is a block diagram of a system 301 that performs noise adaptive non-blind image deblurring according to one or more embodiments. For example, system 301 may be implemented by the processing circuitry of a controller 120 of a vehicle 100. A camera 110 provides a blurred input image I B 210. The camera 110 itself and / or sensors 130 indicating the motion of the vehicle 100 provide a PSF indicating the cause of the blur and facilitate non-blind deblurring. At block 350, implementing the first neural network 355 provides a regularization parameter λ or weights corresponding to a set of predefined regularization parameter λ values. The first neural network 355 facilitates controlling the noise in the input image I B 210 (i.e., noise adaptive deblurring). At block 360, regularized deconvolution provides a deblurred image I DB 220. At block 370, implementing the second neural network 375 helps remove artifacts from the deblurred image I DB 220 to generate an output image 230. This output image 230 may be displayed in the vehicle 100 or used for object detection and classification.

[0073] While the foregoing disclosure has been described with reference to exemplary embodiments, those skilled in the art will understand that various changes may be made and equivalents may be substituted for its elements without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from its basic scope. Therefore, it is intended that the disclosure not be limited to the particular embodiments disclosed, but will include all embodiments falling within its scope.

Claims

1. A method for performing noise adaptive non-blind deblurring on an input image including blur and noise, the method comprising: Using a processing circuit to apply a first neural network to the input image to obtain one or more parameters, wherein applying the first neural network includes obtaining a one-dimensional vector of singular values from the input image and applying a one-dimensional residual convolutional neural network; Using the processing circuit to perform regularized deconvolution to obtain a deblurred image from the input image, wherein the regularized deconvolution uses the one or more parameters to control noise in the deblurred image; And Using the processing circuit to apply a second neural network to remove artifacts from the deblurred image and provide an output image.

2. The method according to claim 1, wherein applying the first neural network results in one parameter as a regularization parameter.

3. The method according to claim 1, wherein applying the first neural network results in two or more parameters as weights corresponding to a set of predefined regularization parameters.

4. The method according to claim 1, further comprising training the first neural network and the second neural network separately or together in an end-to-end arrangement.

5. The method according to claim 1, further comprising obtaining, by the processing circuit, a point spread function that defines the blur in the input image.

6. The method according to claim 5, wherein the input image is obtained by a camera in a vehicle, and the point spread function is obtained based on calibration from one or more sensors of the vehicle or from the camera.

7. A vehicle, comprising: A camera configured to obtain an input image including blur and noise; And A processing circuit configured to apply a first neural network to the input image to obtain one or more parameters, perform regularized deconvolution to obtain a deblurred image from the input image, wherein the regularized deconvolution uses the one or more parameters to control noise in the deblurred image, and apply a second neural network to remove artifacts from the deblurred image and provide an output image, wherein the first neural network is configured to obtain a one-dimensional vector of singular values from the input image and apply a one-dimensional residual convolutional neural network.

8. The vehicle according to claim 7, wherein the processing circuit is configured to apply the first neural network and obtain one parameter as a regularization parameter, or obtain two or more parameters as weights corresponding to a set of predefined regularization parameters, and the processing circuit is configured to train the first neural network and the second neural network separately or together in an end-to-end arrangement.

9. The vehicle according to claim 7, wherein the processing circuit is configured to obtain a point spread function that defines the blur in the input image from one or more vehicle sensors measuring vehicle motion or from camera calibration.

Images