Vehicle image processing method and system for harsh weather environment

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a vehicle-mounted image processing method and system for severe weather environment. The method comprises separating luminance components and chrominance components from an original image; performing highlight region detection on the luminance components, correcting a global atmosphere light value and an initial atmosphere transmittance map based on the highlight region detection result to obtain initial parameters corrected by a physical degradation model; constructing a target function for joint iterative optimization to obtain a preliminary recovered clear scene luminance component; constructing an adaptive filter to perform denoising processing on the preliminary recovered clear scene luminance component to obtain a final enhanced clear luminance component; and fusing the final enhanced clear luminance component with the chrominance components and performing color correction to obtain a final enhanced color vehicle-mounted image. The present application can realize effective enhancement and denoising of vehicle-mounted images in severe weather environment, protect the luminance and structural information of the highlight region, and improve the environmental perception reliability and safety of the vehicle-mounted vision system.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a vehicle-mounted image processing method and system for use in adverse weather conditions. Background Technology

[0002] Existing physical model-based image restoration methods typically rely on atmospheric scattering models, with the core being the use of statistical prior assumptions, such as dark channel priors, to estimate model parameters.

[0003] However, in nighttime, twilight, or urban road environments with complex lighting, images often contain localized artificial light sources with intensity far exceeding ambient light. These bright areas severely violate the fundamental premise of the dark channel prior. Therefore, localized highlight interference causes the prior assumptions of the physical model to fail, leading to the core technical problem of overestimating global atmospheric light values ​​and abnormally low transmittance.

[0004] In subsequent image restoration calculations, this leads to abnormal over-darkening or severe color distortion of highlight areas and adjacent objects, resulting in the loss of crucial texture and contour information. Simultaneously, the intense brightness of highlight areas is incorrectly decomposed and treated as extremely severe noise or interference, disrupting the stability of overall denoising and contrast enhancement, and causing localized information confusion or artifacts in the final enhanced image. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides an in-vehicle image processing method and system for harsh weather environments. It proposes a "recognition-isolation-differential processing" specular interference suppression framework as a pre- and embedded constraint of the physical model, constructs a joint optimization model that integrates special constraints of specular regions, and proposes a structure-adaptive post-processing denoising method linked with the noise estimation of the physical model. This fundamentally overcomes the destructive impact of local strong light interference on model parameter estimation, ensuring that under various extreme and complex real-world harsh weather lighting conditions, it can stably and reliably recover enhanced images with clear details, natural colors, and reasonable preservation of specular regions.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the present invention provides an in-vehicle image processing method for use in adverse weather conditions.

[0007] A vehicle-mounted image processing method for use in adverse weather conditions includes the following steps: Acquire the original image and separate the luminance and chrominance components from it; Highlight regions are detected for the brightness component. Based on the highlight region detection results, the global atmospheric light value and the initial atmospheric transmittance map are corrected to obtain the initial parameters after the physical degradation model is corrected. Based on the initial parameters corrected by the physical degradation model, the scene's inherent brightness and atmospheric transmittance map are used as variables to be optimized. An objective function is constructed and jointly iteratively optimized to obtain the initially restored clear scene brightness components. An adaptive filter is constructed to denoise the initially recovered sharp scene luminance component, resulting in the final enhanced sharp luminance component. The enhanced luminance and chrominance components are then fused and color-corrected to obtain the final enhanced color in-vehicle image.

[0008] As an alternative technical solution, the original image is acquired, and the luminance and chrominance components are separated from the original image, specifically including: Linearly convert the original image from the RGB color space to the YCbCr color space; In the YCbCr color space, the two-dimensional matrix of the Y channel is extracted as the luminance component. Simultaneously, the two-dimensional matrices of the Cb and Cr channels are extracted to form the chromaticity component group. .

[0009] As an alternative technical solution, highlight region detection is performed on the luminance component, and the global atmospheric light value and the initial atmospheric transmittance map are corrected based on the highlight region detection results. Specifically, this includes: Spectrum region detection is performed on the brightness component to generate a spectrum region mask; Based on the highlight region mask, the interference of the highlight region is eliminated when estimating the global atmospheric light value and the initial atmospheric transmittance map, so as to obtain the corrected global atmospheric light value and the initial atmospheric transmittance map. The specific generation process of the highlight area mask includes: A multi-scale Gaussian difference operator is used to perform convolution operations on the luminance components to calculate the luminance gradient magnitude at each pixel location at multiple scales. Set a dynamic brightness threshold; Pixel regions whose brightness values ​​exceed the dynamic brightness threshold and whose sum of gradient magnitudes in the surrounding neighborhood is below the smoothing threshold are marked as candidate highlight regions. A morphological closing operation is performed on the candidate highlight region to generate a binarized highlight region mask, where the pixel value of the highlight region is 1 and the pixel value of the non-highlight region is 0.

[0010] As an alternative technical solution, the process of obtaining the initial parameters after the physical degradation model correction also includes: When calculating the dark channel image, when calculating the minimum value within each local window, ignore the highlight pixels with a mask indicator of 1 in the highlight region. Based on the dark channel image after excluding highlights, select the pixels with the highest brightness values ​​from the previous set percentage; From the positions of the original luminance components corresponding to the previously set percentage of pixels, points in the highlight area are excluded again, and the average luminance of the remaining pixels is calculated as the corrected global atmospheric light value. Based on the corrected global atmospheric light value, the corrected initial atmospheric transmittance map is estimated. The luminance variance is calculated in the non-highlight flat region indicated by the high-light region mask, and the initial estimate of the noise variance is obtained by back-reasoning through the physical model; The initial estimate of noise variance, the corrected global atmospheric light value, and the corrected initial atmospheric transmittance map are used as the initial parameters after the physical degradation model is corrected.

[0011] As an alternative technical solution, the inherent brightness of the scene to be solved is defined as... Atmospheric transmittance The objective function constructed is as follows: ; in: Data fidelity item , Based on noise variance The robust norm function is designed; For the luminance component; This is an initial estimate of the noise variance; This is the corrected global atmospheric light value; Gradient sparsity term yes Image gradient Norm; Transmittance smoothing term yes Image gradient Norm; Specular constraint This is used to constrain the relative ratio of the recovered brightness to the original observed brightness in the highlight region to be close to a preset value. ; Masking for highlight areas; It is a constant; , , These are the preset weighting coefficients.

[0012] As an alternative technical solution, during the joint iterative optimization of the objective function, differentiated constraint strategies are adopted for highlight and non-highlight regions, ultimately yielding the initially recovered clear scene brightness components: An iterative solution is performed using the Alternating Direction Multiplier Method (ADMM) framework. Each iteration is divided into two sub-problems: The first subproblem is to fix the current transmittance. Solving makes Minimize the next round At this time, the specular constraint term Effective, guiding the highlight area The value converges towards a reasonable range; The second subproblem is fixed after the update Solving makes Minimize the next round This process utilizes Provide scene structure information to refine transmittance estimation; After each iteration, update the noise variance based on the current residual. Local estimation; The iteration termination condition is two consecutive iterations. The change norm is less than the threshold or the maximum number of iterations has been reached; The final output is the converged scene inherent brightness as the initially recovered sharp scene brightness component. As an alternative technical solution, the local gradient magnitude map of the initially recovered clear scene brightness component is calculated, the spatial domain kernel parameters are dynamically adjusted based on the local gradient magnitude map, and then an adaptive filter is constructed to denoise the initially recovered clear scene brightness component. Among them, the adaptive filter at the pixel position Spatial domain kernel standard deviation Based on local gradient magnitude Dynamic adjustment, the relationship is as follows: ; in , and These are preset parameters; the standard deviation of the luminance domain kernel is directly linearly related to the noise variance.

[0013] A second aspect of the present invention provides an in-vehicle image processing system for use in adverse weather conditions.

[0014] Vehicle-mounted image processing systems for use in adverse weather conditions include: The original image acquisition module is configured to: acquire the original image and separate the luminance component and chrominance component from the original image; The highlight region suppression module is configured to: detect highlight regions on the brightness component, correct the global atmospheric light value and the initial atmospheric transmittance map based on the highlight region detection results, and obtain the initial parameters after the physical degradation model is corrected. The iterative optimization module is configured to: based on the initial parameters corrected by the physical degradation model, take the scene's inherent brightness and atmospheric transmittance map as variables to be optimized, construct an objective function and perform joint iterative optimization to obtain the initially restored clear scene brightness components; The denoising module is configured to: construct an adaptive filter to denoise the initially recovered sharp scene luminance component to obtain the final enhanced sharp luminance component; The fusion module is configured to fuse and color correct the final enhanced clarity luminance component and chrominance component to obtain the final enhanced color in-vehicle image.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the vehicle-mounted image processing method for adverse weather conditions as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the vehicle-mounted image processing method for adverse weather conditions as described in the first aspect of the present invention.

[0017] The above one or more technical solutions have the following beneficial effects: This invention provides a method and system for vehicle-mounted image processing in adverse weather conditions. In the physical model parameter initialization stage, this invention innovatively introduces a specular interference suppression mechanism. In the joint optimization stage, this invention constructs an objective function that integrates special constraints of specular regions. In the post-processing denoising stage, this invention proposes a structure-adaptive denoising method linked to physical model noise estimation. In the color reconstruction stage, this invention proposes a chromaticity adaptive gain strategy based on local brightness ratio. Therefore, applying the method proposed in this invention can effectively enhance and denoise vehicle-mounted images in adverse weather conditions, while protecting the brightness and structural information of specular regions, thus improving the environmental perception reliability and safety of vehicle-mounted vision systems.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a flowchart of the method in Example 1.

[0021] Figure 2 This is the flowchart for the initialization of highlight suppression in Example 1.

[0022] Figure 3 This is a flowchart of the joint iterative optimization sub-process of Example 1.

[0023] Figure 4 The flowchart for the adaptive denoising sub-process of Example 1 is shown.

[0024] Figure 5 This is a flowchart of the color reconstruction and output sub-process of Example 1. Detailed Implementation

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] Terminology Explanation To facilitate understanding of the technical solution of this application, the core technical terms involved in this application will first be explained: YCbCr color space: A color space representation method that separates the luminance and chrominance information of an image. The Y channel represents luminance, and the Cb and Cr channels represent the blue and red chrominance components, respectively. This color space is widely used in image processing and video compression because it decouples the luminance and chrominance information, which are most important to human visual perception, facilitating their separate processing.

[0029] Luminance component: refers to the Y channel in the YCbCr color space, representing the brightness of each pixel in an image. Its value range is usually normalized to 0 to 1, where 0 represents pure black and 1 represents pure white. The luminance component carries the main structural and detailed information of the image.

[0030] Chromaticity components: These refer to the Cb and Cr channels in the YCbCr color space, representing the blue-yellow and red-green color differences, respectively. While chromaticity components carry color information for an image, they contribute relatively little to the overall image structure.

[0031] Atmospheric scattering model: A physical model describing the degradation of observed images caused by the scattering and absorption of light by suspended particles (such as haze, water droplets, etc.) in the atmosphere. The classic atmospheric scattering model is expressed as: , where is the observed degraded image, is the inherent brightness of the scene, is the atmospheric transmittance, and is the global atmospheric light value.

[0032] Dark Channel Prior: A prior knowledge for image dehazing based on statistical regularities of natural scenes. This prior assumes that in haze-free natural scene images, most local regions contain some pixels with very low brightness values ​​in at least one color channel. Using this prior, the image's transmittance and atmospheric light values ​​can be estimated, thereby achieving image dehazing.

[0033] Transmittance: Describes the ability of reflected light from a scene to penetrate the atmospheric medium and reach an observer, with a value ranging from 0 to 1. Transmittance is defined as follows: where is the atmospheric extinction coefficient and is the scene depth. Lower transmittance indicates a greater distance from the scene or denser fog / haze at that location.

[0034] Highlight regions: These are areas in an image with abnormally high brightness and smooth local gradient changes, typically corresponding to strong artificial light sources (such as car headlights, traffic lights, etc.) or specular reflections. The brightness of these regions is much higher than that of ambient light, which seriously violates the basic assumptions of the dark channel prior.

[0035] Multi-scale Gaussian difference operator: An operator for detecting highlight regions in images. By calculating the difference between Gaussian filtering results with different standard deviations, it can effectively detect regions with high brightness and smooth gradients. This operator has good robustness to illumination changes and noise.

[0036] Morphological closing operation: A morphological image processing operation that first dilates the image and then erodes it. This operation can connect adjacent regions, fill in small holes, and make the detected highlight areas more complete and continuous.

[0037] Alternating Direction Multiplier Method (ADMM): An iterative algorithm for solving convex optimization problems with multiple constraints. This algorithm decomposes the original problem into multiple subproblems, alternately optimizes each variable, and guarantees convergence through Lagrange multiplier updates. The ADMM algorithm has the advantages of fast convergence and wide applicability.

[0038] Adaptive bilateral filter: A bilateral filter that dynamically adjusts filtering parameters based on local image structural features. This filter uses a larger spatial kernel standard deviation for strong smoothing in flat regions and a smaller spatial kernel standard deviation to preserve details in edge regions, thus achieving a balance between noise reduction and detail preservation.

[0039] Example 1 With the continuous improvement of automotive electronics and intelligence, in-vehicle vision systems have become one of the core sensors for environmental perception, assisted driving, and even autonomous driving decision-making. Among these, electronic rearview mirrors, as a crucial component replacing traditional optical lenses, directly impact driving safety and the driving experience through their image quality. However, when vehicles travel in real-world road environments, they inevitably encounter adverse weather conditions such as fog, haze, rain, and snow. These conditions lead to an increase in atmospheric suspended particles, causing severe degradation in images captured by in-vehicle cameras, including decreased contrast, color distortion, blurred details, and increased noise. This significantly weakens the reliability and safety of the vision system. Therefore, how to perform real-time and effective enhancement and denoising processing on in-vehicle images captured under adverse weather conditions has become a critical technical problem that urgently needs to be solved in the field of intelligent vehicles.

[0040] Currently, the technological development in this field is mainly evolving in two directions: one is data-driven deep learning model methods, and the other is image restoration methods based on physical models.

[0041] In the field of deep learning modeling methods, researchers are dedicated to constructing end-to-end neural networks to learn the complex mapping relationship from degraded images to clear images from massive amounts of data. For example, Chinese patent document CN119295328A, titled "Method, Device, Equipment and Storage Medium for Visual Enhancement of Electronic Rearview Mirror Images in Severe Weather," proposes an end-to-end adaptive image enhancement model. This scheme analyzes the input image through a hyperparameter estimation network and dynamically adjusts the parameters of subsequent enhancement modules, aiming to achieve adaptive processing for various severe weather conditions. Similarly, some studies have proposed a multi-scale adaptive detail enhancement dehazing network, which integrates modules such as initial feature extraction, multi-scale adaptive feature learning, and terminal detail enhancement to specifically improve the quality of autonomous driving perception images under foggy conditions. Although such methods can achieve good visual results in some cases, they are essentially "black box" systems, and their processing lacks clear physical meaning. The model is heavily reliant on the distribution of training data. When encountering extreme scenarios not fully covered in the training set (such as rainy or foggy weather with strong, non-uniform artificial light sources), its performance may degrade unpredictably, and its generalization ability is limited.

[0042] Physical model-based methods attempt to solve for a clear image by working backward from the physical principles of imaging, modeling processes such as atmospheric scattering and noise introduction. These methods typically offer better interpretability and theoretical support. For example, authorized patent CN116503454B, "Infrared and Visible Light Image Fusion Method, Apparatus, Electronic Device, and Storage Medium," proposes an image fusion framework for extremely complex scenes. One of its innovations is the introduction of a transmission map estimation model based on an atmospheric scattering model, which decomposes the visible light image into a structural layer and a contrast layer to separate interference information. Chinese patent CN119991474B, "A Method and System for Enhancing Severe Weather Images Based on Physical Prior Knowledge," also embodies this approach, learning weather components through a sharpening module and introducing physical prior knowledge for enhancement. The core of these solutions relies on the accurate estimation of key parameters in the atmospheric scattering physical model.

[0043] However, in-depth research and practice have revealed shortcomings in existing physical model-based methods for real-world vehicle-mounted adverse weather scenarios: conflicts between localized highlight interference and the prior assumptions of the physical model. Specifically, existing technologies generally rely on statistical prior assumptions such as the "dark channel prior" when estimating global atmospheric light values ​​and initial transmittance. These assumptions are usually valid in natural scenes with uniform daytime lighting. However, in nighttime, twilight, or urban road environments with complex lighting, images often contain localized artificial light sources with intensities far exceeding ambient light, such as taillights of vehicles ahead, high beams of oncoming vehicles, traffic lights, and specular reflections from rain-covered road surfaces. These bright areas severely violate the fundamental premise of the dark channel prior, leading to two serious consequences: first, the global atmospheric light value is significantly overestimated, incorrectly biased towards the brightness values ​​of these bright spots; second, the estimated transmittance in and around the bright areas is abnormally low, misclassified by the model as extremely close objects or completely opaque areas.

[0044] This technical problem will cause a chain of negative impacts and safety hazards in practical work. First, in subsequent image restoration calculations, the model will attempt to "correct" this erroneous physical parameter, causing the highlight areas and their adjacent objects (such as vehicles and pedestrians) to be abnormally over-darkened or severely distorted in color, losing crucial texture and contour information. Second, the intense brightness of the highlight areas will be incorrectly decomposed and treated as extremely severe noise or interference, interfering with the stability of overall denoising and contrast enhancement, resulting in local information confusion or artifacts in the final enhanced image. For in-vehicle vision systems, especially electronic rearview mirrors and front-facing cameras, this means a fatal flaw: on rainy or foggy nights, the system may not be able to accurately render the clear outline and reasonable brightness of the taillights of vehicles ahead, or correctly recover roadside pedestrians or obstacles "submerged" by the halo of oncoming headlights, thus providing misleading visual information to the driver or autonomous driving system, greatly increasing driving risks.

[0045] Therefore, existing technologies urgently need an innovative in-vehicle image enhancement and denoising method that can, while inheriting the advantages of interpretability of physical models, fundamentally overcome the destructive impact of local strong light interference on model parameter estimation, and ensure that enhanced images with clear details, natural colors, and reasonable preservation of highlight areas can be stably and reliably restored under various extreme and complex real-world harsh weather lighting conditions.

[0046] Based on this, this invention proposes a "recognition-isolation-differentiated processing" framework for suppressing specular interference. The core idea of ​​this framework is as follows: First, through multi-scale brightness and gradient analysis, a specular region mask is dynamically detected and generated; then, using this mask, in the key steps of estimating the global atmospheric light value and the initial transmittance map, interference from specular region pixels is actively eliminated to obtain corrected initial parameters that are more in line with physical reality; finally, this mask is embedded as prior knowledge into the subsequent joint optimization objective function, and differentiated constraint strategies are adopted for specular and non-spectral regions to achieve a balance between "interference removal" and "fidelity preservation".

[0047] The vehicle-mounted image enhancement and denoising method for adverse weather conditions proposed in this invention has the following overall technical approach: After the input image is separated in color space, the luminance component is processed sequentially through the core chain of "spectral robust physical model initialization" - "joint optimization restoration with embedded specular constraints" - "noise and structure adaptive post-processing denoising". The processed luminance component is then fused with the chrominance component through the "saturation adaptive compensation" strategy to finally output the enhanced result.

[0048] The core difference between this invention and existing technologies lies in: First, in the physical model parameter initialization stage, this invention innovatively introduces a high-light interference suppression mechanism; Second, in the joint optimization stage, this invention constructs an objective function that integrates special constraints of the highlight region; Third, in the post-processing denoising stage, this invention proposes a structure-adaptive denoising method that is linked to the noise estimation of the physical model; Fourth, in the color reconstruction stage, this invention proposes a chromaticity adaptive gain strategy based on local luminance ratio.

[0049] The vehicle-mounted image processing method for adverse weather conditions provided in this embodiment includes the following steps: S1. Data Acquisition and Component Separation: The system acquires raw severe weather images captured by the vehicle-mounted image sensor, converts them to the target color space, and separates the luminance and chrominance components. S2. Construction and initialization of robust physical degradation model: The luminance component is received, and parameters are initialized based on the physical degradation model. During this process, specular interference suppression initialization is performed, specifically as follows: Spectrum region detection is performed on the brightness component to generate a spectrum region mask; Based on this mask, interference from the high-light region is eliminated when estimating the global atmospheric light value and the initial atmospheric transmittance map, thus obtaining the corrected global atmospheric light value and the initial atmospheric transmittance map. Simultaneously, the statistical distribution parameters of image noise are estimated; The final output is the physical degradation model that has been initialized and incorporates atmospheric scattering and noise statistics, along with its corrected initial parameters. S3. Model-driven joint optimization of sharpness and luminance components: Receive the initialized physical degradation model and its corrected initial parameters, take the scene's inherent brightness and atmospheric transmittance map as variables to be optimized, and construct an objective function containing data fidelity terms and prior constraint terms for joint iterative optimization; During the optimization process, differentiated constraint strategies are adopted for highlight areas and non-highlight areas, and the final output is a preliminary restored clear scene brightness component; S4. Adaptive structural denoising: The system receives the initially recovered clear scene brightness component and the noise distribution parameters, constructs an adaptive filter for denoising based on the noise intensity and local image structure characteristics, and outputs the final enhanced clear brightness component. S5. Color Reconstruction and Output: The final enhanced clarity luminance component and the chrominance component separated in step S1 are received, fused and color corrected, and the final enhanced color vehicle image is output.

[0050] The specific implementation method of step S1 is as follows: The system input receives raw RGB color images of severe weather directly output from the vehicle-mounted image sensor; Linearly convert the RGB image from the RGB color space to the YCbCr color space; In the YCbCr color space, the two-dimensional matrix of the Y channel is extracted as the luminance component. Simultaneously, the two-dimensional matrices of the Cb and Cr channels are extracted to form the chromaticity component group. ; This brightness component As the only image data input for step S2, the chromaticity component group The color is temporarily stored for the final step S5 color reconstruction.

[0051] The specific implementation of the specular interference suppression initialization sub-step in step S2 is as follows: The luminance component received from step S1 As input; Using multi-scale Gaussian difference operators Perform convolution operations to calculate the brightness gradient magnitude of each pixel location at multiple scales; Set a dynamic brightness threshold This threshold is dynamically determined based on the percentile of the overall brightness histogram of the image; Those brightness values ​​exceeding Furthermore, pixel regions whose sum of gradient magnitudes in their surrounding neighborhood is lower than the smoothing threshold are marked as candidate highlight regions; A morphological closing operation is performed on the candidate highlight region to connect adjacent regions and fill small holes, generating a binarized highlight region mask. In this mask, the pixel value in the highlight area is 1, and the pixel value in the non-highlight area is 0. Pixels will be used for subsequent parameter estimation.

[0052] The specific implementation of the robust parameter initialization sub-step in step S2 is as follows: With brightness component and highlight area mask As input; First, calculate the dark channel image. However, when calculating the minimum value within each local window, the mask is ignored. Highlight pixel with an indicator value of 1; Based on the dark channel image after excluding highlights Select the pixels with the highest brightness values, which are the top 0.1%. From the original luminance components corresponding to these pixels Within the specified location, points within the highlight area are excluded again, and the average brightness of the remaining pixels is calculated as the corrected global atmospheric light value. ; Then, using the formula:

[0053] Estimate the initial transmittance, where An adjustment factor ranging from 0 to 1 is used to obtain the corrected initial atmospheric transmittance map. ; At the same time, The luminance variance is calculated for the indicated non-highlight flat regions, combined with... and An initial estimate of the noise variance is obtained by back-engineering from the physical model. ; Output the corrected parameter set , and .

[0054] The specific implementation of the objective function construction sub-step in step S3 is as follows: Receive the luminance component from step S2 Corrected parameters , , and highlight area mask ; Define the inherent brightness of the scene to be solved as: Atmospheric transmittance ; Constructing the energy function ; Among them, data fidelity item , Based on noise variance The robust norm function is designed; Gradient sparsity term yes Image gradient Norm; Transmittance smoothing term yes Image gradient Norm; Specular constraint This is used to constrain the relative ratio of the recovered brightness to the original observed brightness in the highlight region to be close to a preset value. , To prevent division by zero for extremely small constants; , , These are the preset weighting coefficients.

[0055] The specific implementation method of the differential constraint implementation and iterative solution sub-step in step S3 is as follows: The constructed energy function Initial estimation and masks As input; An iterative solution is performed using the Alternating Direction Multiplier Method (ADMM) framework. Each iteration is divided into two sub-problems: The first subproblem is to fix the current transmittance. Solving makes Minimized At this time, the specular constraint term Effective, guiding the highlight area The value converges towards a reasonable range; The second subproblem is fixed after the update Solving makes Minimized This process utilizes Provide scene structure information to refine transmittance estimation; After each iteration, update the noise variance based on the current residual. Local estimation; The iteration termination condition is two consecutive iterations. The change norm is less than the threshold Or reach the maximum number of iterations ; The final output is the scene's inherent brightness after convergence. As a preliminary restoration of the clear scene luminance component.

[0056] The specific implementation method of step S4 is as follows: Receive the sharp scene luminance component from the preliminary recovery in step S3 and the basic parameters of noise distribution from step S2 ; First, calculate Local gradient magnitude map ; Construct an adaptive bilateral filter that is applied at pixel location. Spatial domain kernel standard deviation Based on gradient magnitude Dynamic adjustment, the relationship is as follows: , in , and These are preset parameters; Standard deviation of the luminance domain kernel Then directly with Linear association, let it be , This is the proportionality coefficient; Using this adaptive bilateral filter pair Filtering is performed to smooth out noise. While representing random noise, it preserves the gradient. Indicates a strong edge structure; Filtered image As the final enhanced clarity luminance component output.

[0057] The specific implementation of the color saturation adaptive adjustment sub-step in step S5 is as follows: Receive the raw chromaticity components from step S1 , The final enhanced sharpness component from step S4 and the original luminance component from step S1 ; First, calculate separately within the local window. and The mean of the two values ​​is used to obtain the local mean ratio. , in To prevent division by zero for extremely small constants, Indicated by A local window centered on the user; Then, using this ratio, an adaptive gain is applied to the original chromaticity component to calculate the enhanced chromaticity component. and ,in It is a saturation enhancement intensity coefficient ranging from 0 to 1, used to control the degree of color restoration; This operation results in a proportional increase in color saturation in areas where brightness is significantly enhanced.

[0058] The specific implementation method of the color space inverse conversion and output sub-step in step S5 is as follows: Receive the enhanced chroma component after saturation adjustment , And ultimately enhanced clarity and brightness components. ; These three components , , Combined, they form an enhanced YCbCr image; Subsequently, a non-linear inverse transformation from the YCbCr color space to the standard RGB color space is performed; This inverse transformation strictly follows the transformation matrix and gamma correction function defined by the broadcast television standard ITU-R BT.601 or BT.709, converting the YCbCr signal back to the R, G, and B channels of the sRGB color space; The final result is an enhanced color image with natural colors, clear details, and reasonable preservation of highlights, which is then output to the visual processing module of the in-vehicle display system or the Advanced Driver Assistance System (ADAS).

[0059] 1. This proposal puts forward a "recognition-isolation-differentiated processing" framework for suppressing specular interference as a prerequisite and embedded constraint for the physical model. This technique first dynamically detects and generates a specular region mask through multi-scale brightness and gradient analysis; then, using this mask, in the key steps of estimating global atmospheric light values ​​and initial transmittance maps, it actively eliminates interference from specular region pixels to obtain corrected initial parameters that are more in line with physical reality; finally, this mask is embedded as prior knowledge into the subsequent joint optimization objective function.

[0060] The technical effect achieved by these methods is that they fundamentally avoid the destruction of classical physical priors (such as dark channel priors) by local strong light sources, ensure the robustness of model parameter initialization, lay a correct foundation for subsequent high-quality restoration, and solve the core problem of inaccurate restoration under complex lighting conditions such as nighttime rain and fog in existing technologies.

[0061] 2. This proposal puts forward a joint optimization model and method that integrates special constraints for highlight regions. The technical approach lies in constructing a unified objective function, which innovatively introduces a differentiated constraint term based on a highlight mask. For highlight regions, this constraint term does not force gradient sparsity, but rather constrains the recovered brightness to maintain a reasonable relative proportion to the original observed brightness, while ensuring internal smoothness and a natural transition with surrounding areas. For non-highlight regions, traditional prior constraints such as sparse gradients are applied.

[0062] The technical effect achieved by this approach is that, within the same mathematical model framework, it can effectively remove atmospheric scattering and noise while intelligently identifying and protecting the brightness and structural information of inherent strong light sources or highly reflective objects in the scene. This achieves a balance between "interference removal" and "fidelity preservation," outputting a physically consistent and visually reasonable preliminary clear image.

[0063] 3. This proposal puts forward a structure-adaptive post-processing denoising method linked to the noise estimation of the physical model. The technique involves combining the noise distribution parameters estimated during the previous physical model optimization process with the local structural features (such as gradient magnitude) of the preliminary restored image to dynamically construct the parameters of the adaptive filter.

[0064] Specifically, the spatial domain smoothing intensity is adaptively adjusted based on the strength of the local gradient, and the brightness domain similarity tolerance is set based on the estimated noise level.

[0065] The technical effect achieved by this approach is the spatial adaptive variation of denoising intensity: sufficient smoothing is performed in flat areas and areas with high noise estimates, while smoothing is weakened as much as possible in areas with rich edge contours and textures to preserve details. This avoids the problems of detail blurring or noise residue caused by traditional global filtering, and is a refined enhancement of the physical model restoration results.

[0066] 4. This proposal presents a complete image enhancement and denoising method, from physical model-driven approach to perceptual quality optimization. The overall technical path of this method is as follows: After the input image is separated in color space, the luminance component is processed sequentially through the core chain of "highlight robust physical model initialization" -> "joint optimization restoration with embedded highlight constraints" -> "noise and structure adaptive post-processing denoising". The processed luminance component is then fused with the chrominance component through the "saturation adaptive compensation" strategy to finally output the enhanced result.

[0067] The overall technical effect of this approach is the formation of an end-to-end, highly interpretable, and synergistic technical solution across all stages. It not only systematically solves the specific problem of localized high-light interference, but also, through rigorous physical modeling and targeted algorithm design, comprehensively improves the clarity, contrast, color fidelity, and visual realism of the output images from the vehicle system in various adverse weather conditions such as fog, haze, rain, and snow, especially under complex lighting conditions. This significantly enhances the environmental perception reliability and safety of vision-based driver assistance systems.

[0068] The following is a specific application scenario to illustrate the solution of this embodiment: 1. Data acquisition and component separation.

[0069] The vehicle-mounted image sensor captures a frame of raw RGB image containing nighttime fog and the bright taillights of vehicles ahead. The system reads the raw Bayer array data of this image, and after de-mosaicing, white balance, and gamma correction, obtains a standard sRGB color image. .

[0070] Will Convert from sRGB color space to YCbCr color space. The conversion follows the ITU-R BT.709 standard.

[0071] in These are linear RGB values ​​that have undergone inverse gamma correction.

[0072] After separation, the luminance component matrix is ​​obtained. (Size 1920×1080, data type 32-bit floating point, theoretical range [0,1]), and chromaticity component matrix. and .

[0073] In this frame, The global statistical mean is 0.15, with a local peak value of 0.95 in the taillight area of ​​the vehicle.

[0074] 2. Construction and initialization of robust physical degradation model (the following will be combined with...) Figure 2 (Explanation to be provided).

[0075] 2.1 Highlight region detection and mask generation.

[0076] With brightness component For input.

[0077] First, use the standard deviation as and Two Gaussian kernels Filtering is performed to obtain and Calculate the difference of Gaussian image. .

[0078] Simultaneously, calculation Grad magnitude $Grad$ in a 3×3 neighborhood.

[0079] Dynamic brightness threshold Pick The 99.5 quantile of the histogram was calculated to be 0.82. Smoothing threshold. Set it to 0.01.

[0080] Highlight area mask The generation rules are as follows: For each pixel position If both conditions are met and and If a point is found to be a candidate point, it is marked as such. A morphological closing operation is performed on the resulting binary candidate image, using a 5×5 elliptical kernel to connect adjacent taillight halo regions.

[0081] The final generated specular mask In the image, pixels with a value of 1 account for approximately 0.8% of the total area, mainly corresponding to the taillights of the two vehicles and their halos.

[0082] 2.2 Robust parameter initialization.

[0083] Calculate the corrected dark channel: for For each pixel, find the minimum value within its 15×15 local neighborhood, but completely ignore [the local neighborhood] during the search. The pixels. For a neighborhood consisting entirely of highlight pixels, the original pixels within that neighborhood are taken. The minimum value is obtained. The corrected dark channel image is then obtained. .

[0084] Estimate and correct global atmospheric light values: Find The top 0.1% of pixels with the highest brightness. Tracing back to these positions in the original... The brightness value in the image is then excluded again. The position. The average brightness of the remaining pixels is obtained. This value is significantly lower than the direct value. The result for the brightest pixel (approximately 0.95).

[0085] Estimate corrected initial transmittance: Pick According to the formula Calculation. In the sky area, Approximately 0.18, Approximately 0.19; in the near-field road surface area where there is no highlight, Approximately 0.05, It is approximately 0.77.

[0086] Estimate the baseline noise level: exist Furthermore, for areas with smooth textures (such as hazy skies), select 100 10×10 image patches, and then apply the formula... Estimate the noise variance for each block and take the median. .

[0087] At this point, model initialization is complete, and the parameter set is output: .

[0088] 3. Model-driven joint optimization of sharpness and luminance components (the following will combine...) Figure 3 (Explanation to be provided).

[0089] The constructed energy function is:

[0090] in: (Using L2 norm).

[0091] (The sum of the L1 norms of the J gradients).

[0092] (The sum of the L2 norms of the t gradients).

[0093] This constraint limits the brightness recovery of the highlight areas. Approximately the original observed brightness It is 1.1 times that of the original, allowing it to be slightly boosted rather than incorrectly darkened.

[0094] The ADMM algorithm is used to solve the problem, and a penalty parameter is set. The iterative process is as follows: initialization: , .

[0095] kth iteration: a. Update J: fixed Solve the subproblems Gradient descent is used with a step size of 0.01 and 20 iterations. In this step, pixels in the highlight region are... The function of the item, its The value is constrained to be near the original value.

[0096] b. Update t: fixed Solve the subproblems This is a quadratic optimization problem, which is obtained by solving a system of five diagonal linear equations. The solution is obtained using the preconditional conjugate gradient method.

[0097] c. Determining convergence: calculate .

[0098] After 15 iterations, the relative change is less than the threshold. Stop iterating.

[0099] Output optimized scene inherent brightness At this point, the smog effect was greatly reduced, road signs in the distance became visible, and the brightness value of the vehicle taillight area (highlight) remained at a reasonably high level (about 0.98), without any abnormal darkening phenomenon.

[0100] 4. Adaptive structural denoising (which will be combined with the following) Figure 4 (Explanation to be provided).

[0101] enter: , .

[0102] First, calculate Sobel gradient magnitude plot .

[0103] Configure adaptive bilateral filter parameters: , , , .

[0104] For each pixel position : Spatial domain standard deviation: In flat areas ( ), Perform strong smoothing; at strong edges ( ), The smoothing effect is very weak.

[0105] Luminance range standard deviation: .

[0106] Using a 9x9 sliding window, as described above and Calculate the bilateral filter weights, for Filtering is performed. After filtering, patchy noise in the image is effectively suppressed, while structures such as road edges and vehicle outlines remain clear. The final output is an enhanced, sharp luminance component. .

[0107] 5. Color reconstruction and output (the following will be combined with...) Figure 5 (Explanation to be provided).

[0108] 5.1 Adaptive adjustment of color saturation.

[0109] Input: Original chroma , Original brightness Enhance brightness .

[0110] Set local window It is 15×15. Saturation enhancement strength coefficient .

[0111] For each pixel position : Calculate separately and In the window mean within and .

[0112] Calculate local brightness ratio In the distance, thick with smog, Maybe more 1.5 times larger Nearby, Close to 1.0.

[0113] Calculate the enhanced chromaticity:

[0114]

[0115] This means that in the areas where the defogging effect is most pronounced, color saturation will be increased proportionally by up to 35%. ).

[0116] 5.2 Inverse color space conversion and output.

[0117] Will , , Combined to form an enhanced YCbCr image.

[0118] Perform the inverse transformation from YCbCr to linear RGB (using the BT.709 inverse matrix):

[0119] in , , .

[0120] Finally, gamma correction is performed on the linear RGB values ​​( This process yields the final enhanced sRGB image. The image boasts high overall contrast, natural colors, low noise, and accurate preservation of key light sources (taillights). This image is then transmitted to the vehicle's instrument panel display and ADAS target detection module for further processing.

[0121] This embodiment verifies, through the specific data and operation process described above, that the method of the present invention can effectively enhance image details, remove noise, and correctly preserve important light source information in real in-vehicle adverse weather scenarios, especially when there is localized high-light interference, demonstrating excellent practical effects.

[0122] Example 2 This embodiment discloses an in-vehicle image processing system for use in adverse weather conditions.

[0123] Vehicle-mounted image processing systems for use in adverse weather conditions include: The original image acquisition module is configured to: acquire the original image and separate the luminance component and chrominance component from the original image; The highlight region suppression module is configured to: detect highlight regions on the brightness component, correct the global atmospheric light value and the initial atmospheric transmittance map based on the highlight region detection results, and obtain the initial parameters after the physical degradation model is corrected. The iterative optimization module is configured to: based on the initial parameters corrected by the physical degradation model, take the scene's inherent brightness and atmospheric transmittance map as variables to be optimized, construct an objective function and perform joint iterative optimization to obtain the initially restored clear scene brightness components; The denoising module is configured to: construct an adaptive filter to denoise the initially recovered sharp scene luminance component to obtain the final enhanced sharp luminance component; The fusion module is configured to fuse and color correct the final enhanced clarity luminance component and chrominance component to obtain the final enhanced color in-vehicle image.

[0124] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same.

[0125] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0126] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the vehicle-mounted image processing method for adverse weather conditions as described in Embodiment 1 of this disclosure.

[0127] Example 4 The purpose of this embodiment is to provide an electronic device.

[0128] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the vehicle-mounted image processing method for adverse weather conditions as described in Embodiment 1 of this disclosure.

[0129] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0130] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0131] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A vehicle-mounted image processing method for use in adverse weather conditions, characterized in that, Includes the following steps: Acquire the original image and separate the luminance and chrominance components from it; Highlight regions are detected for the brightness component. Based on the highlight region detection results, the global atmospheric light value and the initial atmospheric transmittance map are corrected to obtain the initial parameters after the physical degradation model is corrected. Based on the initial parameters corrected by the physical degradation model, the scene's inherent brightness and atmospheric transmittance map are used as variables to be optimized. An objective function is constructed and jointly iteratively optimized to obtain the initially restored clear scene brightness components. An adaptive filter is constructed to denoise the initially recovered sharp scene luminance component, resulting in the final enhanced sharp luminance component. The enhanced luminance and chrominance components are then fused and color-corrected to obtain the final enhanced color in-vehicle image.

2. The vehicle-mounted image processing method for adverse weather conditions as described in claim 1, characterized in that, Obtain the original image, and separate the luminance and chrominance components from the original image, specifically including: Linearly convert the original image from the RGB color space to the YCbCr color space; In the YCbCr color space, the two-dimensional matrix of the Y channel is extracted as the luminance component. Simultaneously, the two-dimensional matrices of the Cb and Cr channels are extracted to form the chromaticity component group. .

3. The vehicle-mounted image processing method for adverse weather conditions as described in claim 1, characterized in that, Spectrum regions are detected for the luminance component. Based on the spectrum region detection results, the global atmospheric light value and the initial atmospheric transmittance map are corrected. Specifically, this includes: Spectrum region detection is performed on the brightness component to generate a spectrum region mask; Based on the highlight region mask, the interference of the highlight region is eliminated when estimating the global atmospheric light value and the initial atmospheric transmittance map, so as to obtain the corrected global atmospheric light value and the initial atmospheric transmittance map. The specific generation process of the highlight area mask includes: A multi-scale Gaussian difference operator is used to perform convolution operations on the luminance components to calculate the luminance gradient magnitude at each pixel location at multiple scales. Set a dynamic brightness threshold; Pixel regions whose brightness values ​​exceed the dynamic brightness threshold and whose sum of gradient magnitudes in the surrounding neighborhood is below the smoothing threshold are marked as candidate highlight regions. A morphological closing operation is performed on the candidate highlight region to generate a binarized highlight region mask, where the pixel value of the highlight region is 1 and the pixel value of the non-highlight region is 0.

4. The vehicle-mounted image processing method for adverse weather conditions as described in claim 3, characterized in that, The process of obtaining the initial parameters after the physical degradation model is corrected also includes: When calculating the dark channel image, when calculating the minimum value within each local window, ignore the highlight pixels with a mask indicator of 1 in the highlight region. Based on the dark channel image after excluding highlights, select the pixels with the highest brightness values ​​from the previous set percentage; From the positions of the original luminance components corresponding to the previously set percentage of pixels, points in the highlight area are excluded again, and the average luminance of the remaining pixels is calculated as the corrected global atmospheric light value. Based on the corrected global atmospheric light value, the corrected initial atmospheric transmittance map is estimated. The luminance variance is calculated in the non-highlight flat region indicated by the high-light region mask, and the initial estimate of the noise variance is obtained by back-reasoning through the physical model; The initial estimate of noise variance, the corrected global atmospheric light value, and the corrected initial atmospheric transmittance map are used as the initial parameters after the physical degradation model is corrected.

5. The vehicle-mounted image processing method for adverse weather conditions as described in claim 1, characterized in that, Define the inherent brightness of the scene to be solved as: Atmospheric transmittance The objective function constructed is as follows: ； in: Data fidelity item , Based on noise variance The robust norm function is designed; For the luminance component; This is an initial estimate of the noise variance; This is the corrected global atmospheric light value; Gradient sparsity term yes Image gradient Norm; Transmittance smoothing term yes Image gradient Norm; Specular constraint This is used to constrain the relative ratio of the recovered brightness to the original observed brightness in the highlight region to be close to a preset value. ; Masking for highlight areas; It is a constant; , , These are the preset weighting coefficients.

6. The vehicle-mounted image processing method for adverse weather conditions as described in claim 5, characterized in that, During the joint iterative optimization of the objective function, differentiated constraint strategies are adopted for highlight and non-highlight regions, ultimately yielding the initially recovered sharp scene brightness components: An iterative solution is performed using the Alternating Direction Multiplier Method (ADMM) framework. Each iteration is divided into two sub-problems: The first subproblem is to fix the current transmittance. Solving makes Minimize the next round At this time, the specular constraint term Effective, guiding the highlight area The value converges towards a reasonable range; The second subproblem is fixed after the update Solving makes Minimize the next round This process utilizes Provide scene structure information to refine transmittance estimation; After each iteration, update the noise variance based on the current residual. Local estimation; The iteration termination condition is two consecutive iterations. The change norm is less than the threshold or the maximum number of iterations has been reached; The final output is the scene's inherent brightness after convergence, which is used as the initial recovered sharp scene brightness component.

7. The vehicle-mounted image processing method for adverse weather conditions as described in claim 1, characterized in that, Calculate the local gradient magnitude map of the initially restored sharp scene brightness component, dynamically adjust the spatial domain kernel parameters based on the local gradient magnitude map, and then construct an adaptive filter to denoise the initially restored sharp scene brightness component. Among them, the adaptive filter at the pixel position Spatial domain kernel standard deviation Based on local gradient magnitude Dynamic adjustment, the relationship is as follows: ； in , and These are preset parameters; the standard deviation of the luminance domain kernel is directly linearly related to the noise variance.

8. A vehicle-mounted image processing system for use in adverse weather conditions, characterized in that, include: The original image acquisition module is configured to: acquire the original image and separate the luminance component and chrominance component from the original image; The highlight region suppression module is configured to: detect highlight regions on the brightness component, correct the global atmospheric light value and the initial atmospheric transmittance map based on the highlight region detection results, and obtain the initial parameters after the physical degradation model is corrected. The iterative optimization module is configured to: based on the initial parameters corrected by the physical degradation model, take the scene's inherent brightness and atmospheric transmittance map as variables to be optimized, construct an objective function and perform joint iterative optimization to obtain the initially restored clear scene brightness components; The denoising module is configured to: construct an adaptive filter to denoise the initially recovered sharp scene luminance component to obtain the final enhanced sharp luminance component; The fusion module is configured to fuse and color correct the final enhanced clarity luminance component and chrominance component to obtain the final enhanced color in-vehicle image.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the vehicle-mounted image processing method for adverse weather conditions as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the vehicle-mounted image processing method for adverse weather conditions as described in any one of claims 1-7.

Images