Vehicle-mounted camera image denoising and enhancing method and system

Abstract

The invention belongs to the technical field of image processing, and particularly discloses a vehicle-mounted camera image denoising and enhancing method and system, an image is decomposed into sub-band images with different frequencies through multi-scale decomposition, so that noise and details are processed on different scales, aiming at the characteristics of different sub-bands, a self-adaptive algorithm is adopted to estimate noise intensity, and the noise intensity is improved. According to the method, a high-frequency sub-band image is estimated, adaptive denoising processing is carried out according to the estimated noise intensity, the denoising effect is ensured, meanwhile, blurring of edges and textures is reduced, the processing quality is improved, detail enhancement operation is carried out on the denoised high-frequency sub-band image, a deep learning model structure and parameters are optimized in the training process, and the robustness of the high-frequency sub-band image is improved. The method can effectively improve the image quality under limited computing resources, the design of the deep learning model reduces the calculation amount, the powerful feature learning capability of the deep learning model can capture the image features which are difficult to process by a traditional method, and the denoising and enhancement effects are further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method and system for denoising and enhancing images of vehicle-mounted cameras. Background Art

[0002] Images collected by vehicle-mounted cameras in complex environments are often accompanied by noise and distortion, resulting in a decrease in the recognition accuracy of target detection. Denoising and enhancement processing can suppress noise, improve contrast and clarity, making the images more suitable for human visual or algorithm processing requirements.

[0003] Currently, the methods for denoising and enhancing vehicle-mounted camera images usually use Gaussian filtering or median filtering for smoothing processing to suppress noise, and then adjust the gray distribution of the images to enhance the contrast. However, while suppressing noise, it may blur the edges and textures, resulting in the loss of key information, thus affecting the quality of image denoising and enhancement processing. Therefore, we need to propose a method and system for denoising and enhancing vehicle-mounted camera images to solve the above existing problems, enabling it to reduce the computational complexity while ensuring performance, so as to improve the quality of image denoising and enhancement processing. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for denoising and enhancing vehicle-mounted camera images, which can reduce the computational complexity while ensuring performance, so as to improve the quality of image denoising and enhancement processing, and solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A method for denoising and enhancing vehicle-mounted camera images includes the following steps:

[0007] S1. Use a multi-scale analysis method to decompose the input vehicle-mounted camera image into sub-band images of different frequencies;

[0008] S2. According to the characteristics of different sub-bands, use an adaptive algorithm to estimate the noise intensity, and perform adaptive denoising processing based on the estimated noise intensity;

[0009] S3. Perform detail enhancement operations on the high-frequency sub-band images after denoising processing, and then reconstruct the enhanced high-frequency sub-bands and low-frequency sub-bands to obtain a preliminary enhanced image;

[0010] S4. Input the preliminary enhanced image into a deep learning model, and use the deep learning model to perform refined processing on the residual noise and blurred areas in the image to obtain the denoised and enhanced image.

[0011] Preferably, in step S1, the process of decomposing the input vehicle-mounted camera image by the multi-scale analysis method is as follows:

[0012] A1. Select Haar wavelet or Daubechies wavelet according to the characteristics and processing requirements of in-vehicle camera images;

[0013] A2. Perform two-dimensional convolution operation and low-pass downsampling operation on the input image and the low-pass filter to obtain the low-frequency component;

[0014] A3. Perform convolution operation and high-pass downsampling operation on the input image and the high-pass filter to obtain the high-frequency component;

[0015] A4. Take the obtained low-frequency component as the new input image, and repeat steps A2 - A3 for recursive multi-scale decomposition. When the decomposition is completed, the low-frequency sub-band image and high-frequency sub-band image at fine scales are obtained.

[0016] Preferably, in step A2, the process of obtaining the low-frequency component is as follows:

[0017] A21. In the horizontal direction, convolve each row of pixels of the input image with the low-pass filter coefficients once;

[0018] A22. Convolve the result of the horizontal convolution with the low-pass filter again in the vertical direction to obtain the preliminarily filtered image;

[0019] A23. Sample the preliminarily filtered image every other pixel to obtain the low-frequency component.

[0020] Preferably, in step A3, the process of obtaining the high-frequency component is as follows:

[0021] A31. Convolve each row of pixels of the input image with the low-pass filter coefficients in the horizontal direction;

[0022] A32. Convolve the result of the horizontal convolution with the high-pass filter again in the vertical direction to obtain the high-pass filtered image;

[0023] A33. Split the high-pass filtered image into two parts in a ratio of 1:2. The smaller part is convolved with the low-pass filter and downsampled, and the larger part is convolved with the high-pass filter and downsampled to obtain the high-frequency component in the horizontal direction, the high-frequency component in the vertical direction, and the high-frequency component in the diagonal direction.

[0024] Preferably, in step S2, when the adaptive algorithm estimates the noise intensity, for the high-frequency sub-band image, divide the high-frequency sub-band image into multiple non-overlapping local windows, and calculate the noise standard deviation through local statistical analysis within each window. For the low-frequency sub-band image, calculate the variance of the entire low-frequency sub-band image as the estimated value of the noise intensity.

[0025] Preferably, the process of adaptive denoising processing is as follows:

[0026] B1. Filter the low-frequency sub-band image using a Gaussian low-pass filter;

[0027] B2. Determine the high-frequency sub-band noise threshold according to the estimated noise standard deviation of the high-frequency sub-band image. The formula for determining the high-frequency sub-band noise threshold is: Y = l1σ1, where Y is the high-frequency sub-band noise threshold, l1 is the estimated noise standard deviation, and l1 is a constant;

[0028] B3. Process the high-frequency sub-band coefficients using a threshold function to obtain the processed high-frequency sub-band coefficients. The formula for the processed high-frequency sub-band coefficients is: C A = sgn(c a )(|c a |-Y),

[0029] where c a is the high-frequency sub-band coefficient to be processed, C A is the processed high-frequency sub-band coefficient, Y is the high-frequency sub-band noise threshold, sgn(c a ) is the sign function. When c a > 0, sgn(c a ) = 1; when c a = 0, sgn(c a ) = 0; when c a < 0, sgn(c a ) = -1;

[0030] B4. Perform an inverse wavelet transform on the processed high-frequency sub-band coefficients and the filtered low-frequency sub-band coefficients to obtain the denoised image. The inverse wavelet transform formula is:

[0031] P(x, y) = ∑ d,e C(d, e)*ψ d,e (x, y)+∑ d,e K(d, e)*φ d,e (x, y), where P(x, y) is the pixel value of the denoised image at the coordinate (x, y), d and e are the scale and translation parameters respectively, C(d, e) is the wavelet coefficient reflecting the detailed information of the image at the scale d and position e, ψ d,e (x, y) is the wavelet function, representing the reconstruction of the high-frequency part at the coordinate (x, y), K(d, e) is the scale coefficient reflecting the detailed information of the image at the scale d and position e, and φ d,e (x, y) is the scale function, representing the reconstruction of the low-frequency part at the coordinate (x, y).

[0032] Preferably, in step S3, the process of the detail enhancement operation is as follows:

[0033] C1. Divide the high-frequency sub-band image into multiple non-overlapping small blocks, and calculate the histogram of each small block;

[0034] C2. Clip the histogram by restricting the bin height of the histogram. The histogram clipping formula is:

[0035] where, w new (k) is the value of the clipped histogram at the k-th bin, w(k) is the value of the original histogram at the k-th bin, and J is the limit value of the histogram bin height;

[0036] C3. Redistribute the clipped histogram to make the cumulative distribution function of the histogram a linear distribution;

[0037] C4. Map the pixel values within the small block according to the redistributed histogram to obtain the enhanced small block. The formula for pixel value mapping is:

[0038] where, I new (x, y) is the pixel value of the mapped image at the position (x, y), O is the total number of image gray levels, I(x, y) is the pixel value of the image to be mapped at the position (x, y), and w new (i) is the pixel value within the i-th small block after clipping;

[0039] C5. Stitch all the enhanced small blocks together to obtain the enhanced high-frequency sub-band image, and then reconstruct the enhanced high-frequency sub-band image and the low-frequency sub-band image to obtain the preliminary enhanced image.

[0040] Preferably, in step S4, the formula expression of the deep learning model is:

[0041] where, is the pixel value of the output feature map of the v-th convolutional layer at the position (i, j), is the pixel value of the input feature of the (v - 1)-th layer at the position (i + m, j + n), is the weight value of the v-th convolutional kernel at the position (m, n), the size of the convolutional kernel is F * X, and b v is the bias value of the v-th layer.

[0042] Preferably, the process of optimizing the residual noise and blurred areas in the image by the deep learning model is as follows:

[0043] D1. Collect a large number of vehicle-mounted image data as the training data set. The vehicle-mounted image data includes noisy and clear image pairs;

[0044] D2. Input the training dataset into the deep learning model for training, and adjust the weights and biases of the model through the backpropagation algorithm, so that the deep learning model learns the feature representations for denoising and enhancement;

[0045] D3. Input the preliminarily processed image into the trained deep learning model. The trained deep learning model performs refined processing on the residual noise and blurred areas in the image through forward propagation calculation, and outputs the finally enhanced image.

[0046] Based on the above-described method for denoising and enhancing in-vehicle camera images, the present invention also provides a system for denoising and enhancing in-vehicle camera images, including:

[0047] An image input module, which is used to receive the original image data collected by the in-vehicle camera and convert the original image data into a format that can be processed by the multi-scale decomposition module;

[0048] A multi-scale decomposition module, which decomposes the input image collected by the image input module into sub-band images of different frequencies, and at the same time provides hierarchical data that is convenient for targeted operations for the noise estimation and filtering module;

[0049] A noise estimation and filtering module, which estimates the noise intensity according to the sub-band images decomposed by the multi-scale decomposition module respectively, and suppresses the noise by using an adaptive filtering algorithm;

[0050] A detail enhancement module, which performs detail enhancement operations on the high-frequency sub-band images after denoising by the noise estimation and filtering module, and then reconstructs the enhanced high-frequency sub-band images and the low-frequency sub-band images to obtain a preliminarily enhanced image;

[0051] A deep learning post-processing module, which uses a deep learning model to optimize the preliminarily enhanced image, removes the residual noise and blurred areas, and obtains the denoised and enhanced image;

[0052] An image output module, which converts the denoised and enhanced image into a format suitable for use in the in-vehicle system for storage or transmission.

[0053] The method and system for denoising and enhancing in-vehicle camera images proposed by the present invention have the following advantages compared with the prior art:

[0054] 1. The present invention decomposes an image into sub-band images of different frequencies through multi-scale decomposition, enabling noise and details to be processed at different scales. For the characteristics of different sub-bands, an adaptive algorithm is used to estimate the noise intensity, and adaptive denoising processing is performed according to the estimated noise intensity. While ensuring the denoising effect, it reduces the blurring of edges and textures, improves the processing quality, performs detail enhancement operations on the high-frequency sub-band images after denoising processing, and then reconstructs the enhanced high-frequency sub-bands and low-frequency sub-bands. By optimizing the deep learning model structure and parameters during the training process, it can effectively improve the image quality with limited computing resources. The design of the deep learning model reduces the computational amount, and at the same time, the powerful feature learning ability of the deep learning model can capture image features that are difficult to handle by traditional methods, further enhancing the denoising and enhancement effects.

[0055] 2. Through the collaborative work of an image input module, a multi-scale decomposition module, a noise estimation and filtering module, a detail enhancement module, a deep learning post-processing module, and an image output module, the present invention optimizes the data input format, reduces the computational amount through hierarchical processing, improves the processing quality with targeted algorithms, optimizes the deep learning model, and adapts the output format, achieving a reduction in computational complexity while ensuring its performance and improving the quality of image denoising and enhancement processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Shows a flowchart of the method according to an embodiment of the present invention;

[0057] Figure 2 Shows a flowchart of decomposing an input vehicle-mounted camera image by the multi-scale analysis method according to an embodiment of the present invention;

[0058] Figure 3 Shows a flowchart of the adaptive denoising processing according to an embodiment of the present invention;

[0059] Figure 4 Shows a flowchart of the detail enhancement operation according to an embodiment of the present invention;

[0060] Figure 5 Shows a system block diagram of the system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] The present invention provides a method for denoising and enhancing on-vehicle camera images as Figures 1-4 shown, including the following steps:

[0063] S1. Decompose the input on-vehicle camera image into sub-band images with different frequencies by using a multi-scale analysis method;

[0064] As Figure 2 shown, the process of decomposing the input on-vehicle camera image by the multi-scale analysis method is as follows:

[0065] A1. Select Haar wavelet or Daubechies wavelet according to the characteristics and processing requirements of the on-vehicle camera image; Haar wavelet is simple and has high computational efficiency, and is suitable for scenarios with limited computing resources and not particularly high requirements for image edge feature extraction; Daubechies wavelet has better smoothness and vanishing moment characteristics, can capture image details more accurately, and is suitable for on-vehicle image analysis with higher requirements for image detail processing;

[0066] A2. Perform a two-dimensional convolution operation and a low-pass downsampling operation on the input image and a low-pass filter to obtain a low-frequency component;

[0067] The process of obtaining the low-frequency component is as follows:

[0068] A21. In the horizontal direction, convolve each row of pixels of the input image with the low-pass filter coefficients once. The formula for one convolution is:

[0069]

[0070] where q i is the i-th convolution output result, p i+k is the (i + k)-th pixel in the input image sequence, h k is the k-th pixel in the low-pass filter convolution kernel, M is the length of the low-pass filter convolution kernel, and N is the length of the input image sequence;

[0071] A22. Convolve the horizontally convolved result with the low-pass filter again in the vertical direction to obtain a preliminarily filtered image. The formula for the second convolution is:

[0072]

[0073] where Q i is the i-th convolution output result, q i+k is the (i + k)-th pixel in the image sequence after the first convolution, h k is the k-th pixel in the low-pass filter convolution kernel, M is the length of the low-pass filter convolution kernel, and N is the length of the input image sequence;

[0074] A23. Sample every other pixel of the preliminarily filtered image to obtain the low-frequency component. The low-frequency component sampling formula is as follows:

[0075]

[0076] where A j+1 (m,n) is the pixel value at the position (m,n) of the low-frequency subband image at scale j + 1, Q(x,y) is the pixel value at the position (x,y) of the input image after preliminary filtering, R is the size R*R of the input image, h(2m - x) is the mapping value of the low-pass filter in the horizontal direction, 2m - x is the weighted operation on the input image in the horizontal direction, h(2n - y) is the mapping value of the low-pass filter in the vertical direction, 2n - y is the weighted operation on the input image in the vertical direction. Through the weighted operations in the horizontal and vertical directions, the extraction of the low-frequency components of the input image at different scales is realized, facilitating the generation of the low-frequency subband image;

[0077] A3. Perform a convolution operation and a high-pass downsampling operation on the input image and the high-pass filter to obtain the high-frequency component;

[0078] The process of obtaining the high-frequency component is as follows:

[0079] A31. Convolve each row of pixels of the input image with the low-pass filter coefficients in the horizontal direction. The convolution formula is:

[0080]

[0081] where r i is the i-th convolution output result, p i+k is the (i + k)-th pixel in the input image sequence, g k is the k-th pixel in the high-pass filter convolution kernel, H is the length of the high-pass filter convolution kernel, and N is the length of the input image sequence;

[0082] A32. Convolve the result of the horizontal convolution with the high-pass filter again in the vertical direction to obtain the high-pass filtered image. The formula for the second convolution is:

[0083]

[0084] where R i is the i-th convolution output result, r i+k is the (i + k)-th pixel in the input image sequence after horizontal high-pass filtering, g k is the k-th pixel in the high-pass filter convolution kernel, H is the length of the high-pass filter convolution kernel, and N is the length of the input image sequence;

[0085] A33. Split the high-pass filtered image into two parts in a ratio of 1:2. The smaller part is convolved with the low-pass filter and downsampled, while the larger part is convolved with the high-pass filter and downsampled to obtain the high-frequency components in the horizontal direction, vertical direction, and diagonal direction.

[0086] The sampling formula for the high-frequency component in the horizontal direction is:

[0087]

[0088] Where V j+1 (m, n) is the pixel value at position (m, n) in the high-frequency subband image in the horizontal direction at scale j + 1, T j (i, k) is the pixel value at (i, k) in the high-pass filtered image of the j-th layer, h(i - 2m) is the coefficient of the low-pass filter at i - 2m in the horizontal direction, g(k - 2n) is the coefficient of the high-pass filter at k - 2n in the vertical direction, and R is the size of the input image.

[0089] The sampling formula for the high-frequency component in the vertical direction is:

[0090]

[0091] Where D j+1 (m, n) is the pixel value at position (m, n) in the high-frequency subband image in the vertical direction at scale j + 1, T j (i, k) is the pixel value at (i, k) in the high-pass filtered image of the j-th layer, g(i - 2m) is the coefficient of the high-pass filter at i - 2m in the horizontal direction, h(k - 2n) is the coefficient of the low-pass filter at k - 2n in the vertical direction, and R is the size of the input image.

[0092] The sampling formula for the high-frequency component in the diagonal direction is:

[0093]

[0094] Where L j+1 (m, n) is the pixel value at position (m, n) in the high-frequency subband image in the diagonal direction at scale j + 1, T j (i, k) is the pixel value at (i, k) in the high-pass filtered image of the j-th layer, g(i - 2m) is the coefficient of the high-pass filter at i - 2m in the horizontal direction, g(k - 2n) is the coefficient of the high-pass filter at k - 2n in the vertical direction, and R is the size of the input image.

[0095] A4. Use the obtained low-frequency component as the new input image, and repeat steps A2 - A3 for recursive multi-scale decomposition. When the decomposition is completed, the low-frequency sub-band image and high-frequency sub-band image at fine scales are obtained; in each recursion, the image is decomposed into finer frequency components, and the sub-band images at different scales respectively reflect the characteristics of the image in different resolution and frequency ranges. For example, in the first-layer decomposition, the low-frequency sub-band may contain the overall structure of the larger objects in the image, while the high-frequency sub-band captures the obvious edges; in the second-layer decomposition, the finer-scale low-frequency sub-band further refines the details of the object structure, and the high-frequency sub-band can detect more subtle texture and edge changes. Through this multi-scale decomposition, in subsequent processing, different algorithms can be used to process noise and details for sub-band images at different scales and frequencies respectively, avoiding excessive influence on key information during global processing.

[0096] Multi-scale decomposition decomposes the image into sub-band images of different frequencies, enabling noise and details to be processed at different scales. In the low-frequency sub-band, the processing is relatively simple with a small amount of computation, mainly focusing on the main structure of the image; in the high-frequency sub-band, targeted processing is carried out for details and noise. Since the processing range is relatively small, the overall computation amount is also reduced. At the same time, the processing at different scales can be performed in parallel to improve the processing efficiency.

[0097] S2. According to the characteristics of different sub-bands, use an adaptive algorithm to estimate the noise intensity and perform adaptive denoising processing based on the estimated noise intensity.

[0098] When the adaptive algorithm estimates the noise intensity, for the high-frequency sub-band image, divide the high-frequency sub-band image into multiple non-overlapping local windows, and calculate the noise standard deviation through local statistical analysis within each window. For the low-frequency sub-band image, calculate the variance of the entire low-frequency sub-band image as the estimated value of the noise intensity.

[0099] As Figure 3 shown, the process of adaptive denoising processing is as follows:

[0100] B1. Use a Gaussian low-pass filter to filter the low-frequency sub-band image. The formula for the filtering process is: where, I low (m,n) is the pixel value of the filtered image at the coordinate (m,n), I low (m - x,n - y) is the pixel value of the low-frequency sub-band image at (m - x,n - y), G(x,y,σ low ) is the Gaussian low-pass filter, x and y are the spatial coordinates of the Gaussian low-pass filter respectively, σ low is the standard deviation of the Gaussian filter that determines the filter bandwidth, and a and b are the coefficients that determine the size of the convolution kernel respectively.

[0101] B2. Determine the high-frequency subband noise threshold according to the estimated noise standard deviation of the high-frequency subband image. The formula for determining the high-frequency subband noise threshold is: Y = l1σ1, where Y is the high-frequency subband noise threshold, l1 is the estimated noise standard deviation, and l1 is a constant;

[0102] B3. Process the high-frequency subband coefficients using a threshold function to obtain the processed high-frequency subband coefficients. The formula for the processed high-frequency subband coefficients is: C A = sgn(c a )(|c a |-Y), where c a is the high-frequency subband coefficient to be processed, C A is the processed high-frequency subband coefficient, Y is the high-frequency subband noise threshold, sgn(c a ) is the sign function. When c a > 0, sgn(c a ) = 1; when c a = 0, sgn(c a ) = 0; when c a < 0, sgn(c a ) = -1;

[0103] B4. Perform an inverse wavelet transform on the processed high-frequency subband coefficients and the filtered low-frequency subband coefficients to obtain the denoised image. The inverse wavelet transform formula is:

[0104] P(x, y) = ∑ d,e C(d, e)*ψ d,e (x, y)+∑ d,e K(d, e)*φ d,e (x, y), where P(x, y) is the pixel value of the denoised image at the coordinate (x, y), d and e are the scale and translation parameters respectively, C(d, e) is the wavelet coefficient reflecting the detail information of the image at the scale d and position e, ψ d,e (x, y) is the wavelet function representing the reconstruction of the high-frequency part at the coordinate (x, y), K(d, e) is the scale coefficient reflecting the detail information of the image at the scale d and position e, and φ d,e (x, y) is the scale function representing the reconstruction of the low-frequency part at the coordinate (x, y).;

[0105] For the characteristics of different subbands, an adaptive algorithm is used to estimate the noise intensity and perform adaptive denoising processing according to the estimated noise intensity, avoiding the use of a unified filtering method for the entire image, reducing unnecessary calculations. Compared with traditional filtering methods, while ensuring the denoising effect, it reduces the blurring of edges and textures and improves the processing quality.

[0106] S3. Perform detail enhancement on the denoised high-frequency subband image, and then reconstruct the enhanced high-frequency subband and the low-frequency subband to obtain a preliminary enhanced image;

[0107] As Figure 4 shown, the process of detail enhancement is as follows:

[0108] C1. Divide the high-frequency subband image into multiple non-overlapping small blocks, and calculate the histogram of each small block;

[0109] C2. Clip the histogram by restricting the bin height of the histogram. The histogram clipping formula is:

[0110] where, w new (k) is the value of the clipped histogram at the k-th bin, w(k) is the value of the original histogram at the k-th bin, and J is the limit value of the histogram bin height;

[0111] C3. Redistribute the clipped histogram to make the cumulative distribution function of the histogram a linear distribution;

[0112] C4. Map the pixel values within the small block according to the redistributed histogram to obtain an enhanced small block. The formula for pixel value mapping is:

[0113] where, I new (x, y) is the pixel value of the mapped image at the position (x, y), O is the total number of image gray levels, I(x, y) is the pixel value of the image to be mapped at the position (x, y), and w new (i) is the pixel value within the i-th small block after clipping;

[0114] C5. Stitch all the enhanced small blocks together to obtain an enhanced high-frequency subband image, and then reconstruct the enhanced high-frequency subband image and the low-frequency subband image to obtain a preliminary enhanced image;

[0115] S4. Input the preliminary enhanced image into the deep learning model, and optimize the remaining noise and blurred areas in the image through the deep learning model to obtain a denoised and enhanced image.

[0116] The formula expression of the deep learning model is:

[0117] where, is the pixel value of the output feature map of the v-th convolutional layer at the position (i, j), is the pixel value of the input feature of the (v - 1)-th layer at the position (i + m, j + n), is the weight value of the v-th convolutional kernel at the position (m, n). The size of the convolutional kernel is F * X, bv is the bias value of the v-th layer;

[0118] The process of the deep learning model optimizing the residual noise and blurred areas in the image is as follows:

[0119] D1. Collect a large number of vehicle-mounted image data as the training data set. The vehicle-mounted image data includes noisy and clear image pairs;

[0120] D2. Input the training data set into the deep learning model for training, and adjust the weights and biases of the model through the backpropagation algorithm, so that the deep learning model learns the feature representations of denoising and enhancement;

[0121] D3. Input the preliminarily processed image into the trained deep learning model. The trained deep learning model performs refined processing on the residual noise and blurred areas in the image through forward propagation calculation, and outputs the finally enhanced image.

[0122] By optimizing the deep learning model structure and parameters during the training process, it can effectively improve the image quality under limited computing resources. The design of the deep learning model reduces the amount of calculation. At the same time, the powerful feature learning ability of the deep learning model can capture image features that are difficult to handle by traditional methods, further improving the denoising and enhancement effects.

[0123] Based on the above-described method for denoising and enhancing vehicle-mounted camera images, the present invention also provides a system for denoising and enhancing vehicle-mounted camera images, as Figure 5 shown, including an image input module, a multi-scale decomposition module, a noise estimation and filtering module, a detail enhancement module, a deep learning post-processing module, and an image output module. The image input module is used to receive the original image data collected by the vehicle-mounted camera and convert the original image data into a format that the multi-scale decomposition module can process, avoiding additional computational overhead caused by data format problems and laying a foundation for efficient processing; the multi-scale decomposition module decomposes the input image collected by the image input module into sub-band images of different frequencies. Different scales can be processed in parallel, greatly shortening the processing time. At the same time, it provides hierarchical data for the noise and estimation module that is convenient for targeted operations; the noise estimation and filtering module estimates the noise intensity respectively according to the sub-band images decomposed by the multi-scale decomposition module, and uses an adaptive filtering algorithm to suppress the noise, avoiding the blurring of key information such as edges and textures, improving the denoising quality, and reducing unnecessary calculations at the same time; the detail enhancement module performs detail enhancement operations on the high-frequency sub-band images after denoising by the noise estimation and filtering module, and then reconstructs the enhanced high-frequency sub-band images and low-frequency sub-band images to obtain a preliminarily enhanced image;

[0124] The deep learning post - processing module uses a deep learning model to optimize the preliminarily enhanced image, remove residual noise and blurred areas, and obtain a denoised and enhanced image, further improving the processing effect; the image output module converts the denoised and enhanced image into a format suitable for use in a vehicle system for storage or transmission, avoiding complex data conversion due to format incompatibility, thereby reducing the overall computational complexity;

[0125] Through the collaborative work of the image input module, multi - scale decomposition module, noise estimation and filtering module, detail enhancement module, deep learning post - processing module, and image output module, from optimizing the data input format, reducing the computational amount through hierarchical processing, improving the processing quality with targeted algorithms, optimizing the deep learning model to adapting the output format, it realizes reducing the computational complexity while ensuring its performance, and improving the quality of image denoising and enhancement processing.

[0126] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for denoising and enhancing in-vehicle camera images, characterized in that: It includes the following steps: S1. Use the multi-scale analysis method to decompose the input vehicle-mounted camera image into sub-band images with different frequencies; S2. According to the characteristics of different sub-bands, use an adaptive algorithm to estimate the noise intensity, and perform adaptive denoising processing according to the estimated noise intensity; S3. Perform detail enhancement operations on the denoised high-frequency sub-band images, and then reconstruct the enhanced high-frequency sub-bands and low-frequency sub-bands to obtain a preliminary enhanced image; S4. Input the preliminary enhanced image into the deep learning model, and use the deep learning model to perform refined processing on the remaining noise and blurred areas in the image to obtain the denoised and enhanced image.

2. The method for denoising and enhancing the on-vehicle camera image according to claim 1, wherein: In step S1, the process of decomposing the input vehicle-mounted camera image by the multi-scale analysis method is as follows: A1. Select Haar wavelet or Daubechies wavelet according to the characteristics and processing requirements of the vehicle-mounted camera image; A2. Perform two-dimensional convolution operation and low-pass downsampling operation on the input image and the low-pass filter to obtain the low-frequency component; A3. Perform convolution operation and high-pass downsampling operation on the input image and the high-pass filter to obtain the high-frequency component; A4. Use the obtained low-frequency component as the new input image, and repeat steps A2 - A3 for recursive multi-scale decomposition. When the decomposition is completed, the fine-scale low-frequency sub-band image and high-frequency sub-band image are obtained.

3. The method for denoising and enhancing the in-vehicle camera image according to claim 2, wherein: In step A2, the process of obtaining the low-frequency component is as follows: A21. In the horizontal direction, convolve each row of pixels of the input image with the low-pass filter coefficients once; A22. Convolve the result of the horizontal convolution with the low-pass filter again in the vertical direction to obtain the preliminarily filtered image; A23. Sample the preliminarily filtered image every other pixel to obtain the low-frequency component.

4. The method for denoising and enhancing vehicle-mounted camera images according to claim 3, characterized in that: In step A3, the process of obtaining the high-frequency component is as follows: A31. Convolve each row of pixels of the input image with the low-pass filter coefficients in the horizontal direction; A32. Convolve the result of the horizontal convolution with the high-pass filter again in the vertical direction to obtain the high-pass filtered image; A33. Split the high-pass filtered image into two parts in a ratio of 1:

2. The smaller part is convolved with the low-pass filter and downsampled, and the larger part is convolved with the high-pass filter and downsampled to obtain the high-frequency component in the horizontal direction, the high-frequency component in the vertical direction, and the high-frequency component in the diagonal direction.

5. The method for denoising and enhancing the in-vehicle camera image according to claim 1, characterized in that: In step S2, when the adaptive algorithm estimates the noise intensity, for the high-frequency sub-band image, the high-frequency sub-band image is divided into multiple non-overlapping local windows, and the noise standard deviation is calculated through local statistical analysis within each window. For the low-frequency sub-band image, the variance of the entire low-frequency sub-band image is calculated as the estimated value of the noise intensity.

6. The method for denoising and enhancing in-vehicle camera images according to claim 5, wherein: The process of adaptive denoising processing is as follows: B1. Use a Gaussian low-pass filter to filter the low-frequency sub-band image; B2. Determine the high-frequency sub-band noise threshold according to the estimated noise standard deviation of the high-frequency sub-band image. The formula for determining the high-frequency sub-band noise threshold is: Y = l1σ1, where Y is the high-frequency sub-band noise threshold, l1 is the estimated noise standard deviation, and l1 is a constant; B3. Use a threshold function to process the high-frequency subband coefficients to obtain the processed high-frequency subband coefficients. The calculation formula for the processed high-frequency subband coefficients is: C A = sgn(c a )(|c a |- Y), Among them, c a is the high-frequency subband coefficient during processing, C A is the high-frequency subband coefficient after processing, Y is the high-frequency subband noise threshold, sgn(c a ) is the sign function. When c a > 0, sgn(c a ) = 1. When c a = 0, sgn(c a ) = 0. When c a < 0, sgn(c a ) = -1; B4. Perform inverse wavelet transform on the processed high-frequency subband coefficients and the filtered low-frequency subband coefficients to obtain the denoised image. The formula for inverse wavelet transform is as follows: P(x,y) = ∑ d,e C(d,e) * ψ d,e (x,y) + ∑ d,e K(d,e) * φ d,e (x,y), where P(x,y) is the pixel value of the denoised image at the coordinate (x,y), d and e are the scale and translation parameters respectively, C(d,e) is the wavelet coefficient reflecting the detailed information of the image at scale d and position e, ψ d,e (x,y) is the wavelet function, representing the reconstruction of the high-frequency part at the coordinate (x,y), K(d,e) is the scale coefficient reflecting the detailed information of the image at scale d and position e, φ d,e (x,y) is the scaling function, representing the reconstruction of the low-frequency part at the coordinate (x,y).

7. The method for denoising and enhancing in-vehicle camera images according to claim 1, characterized in that: In step S3, the process of detail enhancement operation is as follows: C1. Divide the high-frequency subband image into multiple non-overlapping small blocks, and calculate the histogram of each small block; C2. Clip the histogram by restricting the bin height of the histogram. The formula for histogram clipping is as follows: where, w new (k) is the value of the cropped histogram at the k-th bin, w(k) is the value of the original histogram at the k-th bin, and J is the limit value of the histogram bin height; C3. Redistribute the clipped histogram to make the cumulative distribution function of the histogram a linear distribution; C4. Map the pixel values within the small block according to the redistributed histogram to obtain the enhanced small block. The formula for pixel value mapping is as follows: Among them, I new (x, y) is the pixel value of the mapped image at the position (x, y), O is the total number of image gray levels, I(x, y) is the pixel value of the image to be mapped at the position (x, y), and w new (i) is the pixel value within the i-th small block after cropping; C5. Stitch all the enhanced small blocks together to obtain the enhanced high-frequency subband image, and then reconstruct the enhanced high-frequency subband image and the low-frequency subband image to obtain the preliminary enhanced image.

8. The method for denoising and enhancing vehicle-mounted camera images according to claim 1, characterized in that: In step S4, the formula expression of the deep learning model is as follows: Among them, is the pixel value of the output feature map of the v-th convolutional layer at the position (i, j), is the pixel value of the input feature of the (v - 1)-th layer at the position (i + m, j + n), is the weight value of the v-th convolutional kernel at the position (m, n), and the size of the convolutional kernel is F * X, b v is the bias value of the v-th layer.

9. The method for denoising and enhancing vehicle-mounted camera images according to claim 8, characterized in that: The process by which the deep learning model optimizes the residual noise and blurred areas in the image is as follows: D1. Collect a large amount of vehicle-mounted image data as the training data set. The vehicle-mounted image data includes noisy and clear image pairs; D2. Input the training data set into the deep learning model for training, and adjust the weights and biases of the model through the backpropagation algorithm, so that the deep learning model learns the feature representations of denoising and enhancement; D3. Input the preliminarily processed image into the trained deep learning model. The trained deep learning model performs refined processing on the residual noise and blurred areas in the image through forward propagation calculation, and outputs the finally enhanced image.

10. A system for denoising and enhancing in-vehicle camera images, based on the method for denoising and enhancing in-vehicle camera images according to any one of claims 1-9, characterized in that: Including: An image input module, which is used to receive the original image data collected by the vehicle-mounted camera and convert the original image data into a format that the multi-scale decomposition module can process; A multi-scale decomposition module, which decomposes the input image collected by the image input module into subband images of different frequencies, and at the same time provides hierarchical and convenient data for targeted operations for the noise and estimation module; A noise estimation and filtering module, which estimates the noise intensity according to the subband images decomposed by the multi-scale decomposition module respectively, and suppresses the noise by using an adaptive filtering algorithm; A detail enhancement module, which performs detail enhancement operations on the high-frequency subband image after denoising by the noise estimation and filtering module, and then reconstructs the enhanced high-frequency subband image and the low-frequency subband image to obtain a preliminary enhanced image; A deep learning post-processing module, which uses a deep learning model to optimize the preliminary enhanced image, remove the residual noise and blurred areas, and obtain the denoised and enhanced image; An image output module, which converts the denoised and enhanced image into a format suitable for use in the vehicle-mounted system for storage or transmission.

Images