Camera vision optimization method for rainy days

Abstract

The present invention relates to the field of camera technology, and in particular to a camera rainy day vision optimization method, comprising the following steps: S1, rainy day detection: detecting the current weather conditions through an environmental sensor; S2, image acquisition and preprocessing: using a camera to acquire image data of the current environment; S3, raindrop dynamic prediction and repair: identifying and predicting the area where raindrops appear in advance, and dynamically repairing the image damage caused by raindrops through an image repair algorithm; S4, image enhancement: performing image enhancement on the image data after raindrop repair processing; S5, real-time feedback: feeding back the enhanced image data to a monitoring device in real time; S6, dynamic adjustment: based on the real-time feedback image data, combined with actual environmental changes, evaluating the effect of the image data, and dynamically adjusting according to the result of the effect evaluation. The present invention effectively improves the image quality of the camera in a rainy day environment.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and in particular to a method for optimizing camera vision in rainy days. Background Art

[0002] With the widespread application of intelligent monitoring systems in various fields, it has become crucial for cameras to provide high-quality monitoring images in all-weather environments. However, in rainy environments, raindrops often adhere to the camera lens, causing blurred and distorted images, seriously affecting the image quality and reliability of the monitoring system. How to effectively optimize the visual effect of the camera in rainy environments has become a problem that needs to be solved urgently.

[0003] In the prior art, although there are some methods that attempt to improve the camera image quality in rainy environments, such as reducing the impact of raindrops by heating the lens or using a hydrophobic coating, or removing raindrops and enhancing the image through image processing algorithms, these methods have many shortcomings. Physical methods are difficult to completely avoid raindrop adhesion and are costly. Traditional image processing algorithms are often unable to detect and repair raindrop areas in real time and accurately, and the processing effect is unsatisfactory and cannot meet the needs of practical applications. In addition, the prior art generally lacks the ability to dynamically adapt to environmental changes and cannot maintain the stability of image quality under different lighting and rain intensity conditions.

[0004] The purpose of the present invention is to provide a camera vision optimization method in rainy days, so as to ensure that the camera can provide high-quality and stable monitoring images under various weather conditions, and significantly improve the reliability and practicality of the monitoring system in bad weather. Summary of the invention

[0005] Based on the above objectives, the present invention provides a method for optimizing camera vision in rainy days.

[0006] The camera rainy day vision optimization method includes the following steps:

[0007] S1, rain detection: detect the current weather conditions through environmental sensors and determine whether it is rainy;

[0008] S2, image acquisition and preprocessing: using the camera to acquire image data of the current environment, and preprocessing the acquired image data, including denoising and color correction;

[0009] S3, raindrop dynamic prediction and repair: Perform raindrop dynamic prediction on the preprocessed image data, identify and predict the area where raindrops will appear in advance, and dynamically repair the image damage caused by raindrops through the image repair algorithm, including:

[0010] S31, Raindrop Detection: Use the Convolutional Neural Network (CNN) model to detect raindrop areas and identify areas blocked by raindrops in image data;

[0011] S32, dynamic prediction: predict the dynamic changes of the raindrop area and identify the area where raindrops appear in advance;

[0012] S33, image restoration: after the raindrop area is detected, the image restoration algorithm is used to process the raindrop area, and the area blocked by the raindrop is filled according to the surrounding pixel information;

[0013] S34, multi-frame fusion: through the multi-frame image fusion technology, the unobstructed image data in the continuous frames are partially fused into the current frame to eliminate the influence of raindrops on the single-frame image data;

[0014] S4, image enhancement: image enhancement is performed on the image data after raindrop repair processing, including local contrast enhancement and multi-scale image fusion;

[0015] S5, real-time feedback: feeding back the enhanced image data to the monitoring equipment in real time;

[0016] S6, dynamic adjustment: Based on real-time feedback of image data and combined with actual environmental changes, the effect of the image data is evaluated, including contrast, brightness, and sharpening intensity, and dynamic adjustments are made based on the results of the effect evaluation.

[0017] Optionally, the image acquisition and preprocessing in S2 includes:

[0018] S21, image acquisition: using a camera to collect image data of the current environment in real time, the image data includes multiple continuous frames;

[0019] S22, denoising: performing denoising on the collected image data using a Gaussian filtering algorithm;

[0020] S23, color correction: performing color correction on the image data after contrast enhancement using a white balance algorithm.

[0021] Optionally, the convolutional neural network (CNN) model in S31 includes:

[0022] S311, data preprocessing and enhancement: The data diversity is increased by using data expansion technology on the input image data, which is expressed as:

[0023] ;

[0024] in, is the original image data, is the preprocessed image data, is the data enhancement function;

[0025] S312, convolution layer: dilated convolution is introduced in the convolution layer to expand the receptive field without increasing the number of parameters, expressed as:

[0026] ;

[0027] in, is the convolution output, For input image data, is the convolution kernel, is the expansion rate, It is half the size of the convolution kernel;

[0028] S313, introduction of attention mechanism: Introducing the attention mechanism in the convolutional network, expressed as:

[0029] ;

[0030] ;

[0031] in, is the attention weight, Score feature relevance, is the correlation function, and They are the convolutional layer output and the attention mechanism input respectively;

[0032] S314, dual loss function: A dual loss function is used, including classification loss and positioning loss, expressed as:

[0033] ;

[0034] ;

[0035] ;

[0036] in, is the classification loss, For the positioning loss, is the balance coefficient, is the real category, is the predicted probability, is the true bounding box, To predict the bounding box;

[0037] S315, mixed pooling layer: Introduce the mixed pooling layer in the convolutional network, expressed as:

[0038] ;

[0039] in, is the mixed pooling output, is the maximum pooling output, is the average pooling output, is the weight coefficient.

[0040] Optionally, the dynamic prediction in S32 includes:

[0041] S321, temporal feature extraction: extract temporal features from continuous frame image data, and use the long short-term memory network (LSTM) to model the image frame sequence, expressed as:

[0042] ;

[0043] in, is the hidden state at the current moment, is the current input frame, is the hidden state at the previous moment, and is the weight matrix, is the bias term, is the activation function;

[0044] S322, predicting raindrop area: using the extracted time series features to dynamically predict the raindrop area, generating the area where raindrops appear in the future frame, expressed as;

[0045] ;

[0046] in, is the predicted raindrop area, is the output layer weight matrix, is the bias term, is the activation function;

[0047] S323, dynamically adjust the prediction result: Combine the characteristics of the current frame image data and the prediction result of the raindrop area to dynamically adjust the raindrop area, which is expressed as:

[0048] ;

[0049] in, is the adjusted raindrop area prediction result, is the balance coefficient, It is the feature extracted by the convolutional neural network (CNN) of the current frame image data.

[0050] Optionally, the image restoration in S33 includes:

[0051] S331, raindrop area detection: After detecting the raindrop area, mark the pixel points in the raindrop occlusion area, expressed as:

[0052] ;

[0053] in, is the binary mask of the raindrop occluded area, 1 represents the raindrop area and 0 represents the normal area;

[0054] S332, neighborhood pixel feature extraction: extract pixel information around the raindrop occlusion area for reference during repair, expressed as:

[0055] ;

[0056] in, The raindrop area The set of surrounding unoccluded pixels, Represents pixels Neighborhood of;

[0057] S333, repair image: Use image repair algorithm to repair the raindrop area, expressed as:

[0058] ;

[0059] in, is the pixel value of the restored image, is a convolutional neural network for image restoration, is the original image data, Mask the raindrop occluded area;

[0060] S334, pixel value filling: fill the area blocked by raindrops according to the neighborhood pixel information and the output of the image restoration algorithm, expressed as:

[0061] ;

[0062] in, is the output repair pixel value, is the average value of the unoccluded pixels in the neighborhood, is the balance coefficient.

[0063] Optionally, the multi-frame fusion in S34 includes:

[0064] S341, feature extraction: extract features from continuous frame image data, and extract the features of each frame image data through the convolutional neural network (CNN) model, expressed as:

[0065] ;

[0066] in, For the Frame image data Extracted feature map;

[0067] S342, feature alignment: align the features of the continuous frame image data, estimate the inter-frame motion using the optical flow algorithm, and align the feature maps, which is expressed as:

[0068] ;

[0069] in, For Frame to Optical flow estimation of frames, is the alignment operation of the feature map, is the aligned feature map;

[0070] S343, weight calculation: Calculate the weight of each frame of image data, and assign fusion weights according to the clarity of the image data and the raindrop occlusion, expressed as:

[0071] ;

[0072] in, For the The fusion weight of the frame, For the Raindrop intensity of frame image data, To adjust the parameters;

[0073] S344, image fusion: weighted fusion of features of multiple frames of image data to generate current frame image data after raindrop removal, expressed as:

[0074] ;

[0075] ;

[0076] in, is the fused feature map, is the image data after removing raindrops, Convolutional neural network for decoding.

[0077] Optionally, the image enhancement in S4 includes:

[0078] S41, local contrast enhancement: Adaptive histogram equalization algorithm (CLAHE) is used to perform local contrast enhancement on the repaired image data, which is expressed as:

[0079] ;

[0080] in, The restored image data is at position The pixel value at is the enhanced pixel value, is the cumulative distribution function, is the minimum non-zero cumulative distribution value, is the gray level;

[0081] S42, multi-scale feature extraction: extract multi-scale features from the enhanced image data, and extract image features at different scales through a multi-scale convolutional neural network (MSCNN), expressed as:

[0082] ;

[0083] in, For scale The feature map extracted below, It is a multi-scale convolutional neural network;

[0084] S43, multi-scale image fusion: feature maps at different scales are fused to generate enhanced image data, expressed as:

[0085] ;

[0086] ;

[0087] in, is the feature map after multi-scale fusion, is the enhanced image data, is the weight coefficient at different scales, Convolutional neural network for decoding.

[0088] Optionally, the dynamic adjustment in S6 includes:

[0089] S61, environmental parameter detection: real-time detection of environmental parameters, including light intensity and rain intensity, through sensors;

[0090] S62, image effect evaluation: based on the real-time detected environmental parameters, the image effect of the current image data is evaluated, including the contrast, brightness and sharpness of the image data;

[0091] S63, Image Adjustment: Dynamically adjust the contrast, brightness and sharpness of the image data based on the results of the image effect evaluation.

[0092] Optionally, the image effect evaluation in S62 includes:

[0093] S621, contrast evaluation: based on the real-time detected environmental parameters, the contrast of the current image data is calculated using the standard deviation method;

[0094] S622, brightness evaluation: based on the real-time detected environmental parameters, the brightness of the current image data is calculated using the average gray value method;

[0095] S623, sharpening strength evaluation: based on the real-time detected environmental parameters, the sharpening strength of the current image data is calculated using the Laplace operator method.

[0096] Optionally, the image adjustment in S63 includes:

[0097] S631, contrast adjustment: dynamically adjusting the contrast of the image data through linear transformation based on the contrast result of the image effect evaluation;

[0098] S632, brightness adjustment: based on the brightness result of the image effect evaluation, dynamically adjust the brightness of the image data by adjusting the gray value of the image;

[0099] S633, sharpening intensity adjustment: based on the sharpening intensity result of the image effect evaluation, the sharpening intensity of the image data is dynamically adjusted through a convolution operation.

[0100] Beneficial effects of the present invention:

[0101] The present invention effectively improves the image quality of the camera in a rainy environment through rainy day detection, image acquisition and preprocessing, raindrop dynamic prediction and repair, image enhancement, real-time feedback and dynamic adjustment. It uses a convolutional neural network to detect and repair raindrops, combines multi-frame fusion technology to eliminate the influence of raindrops on single-frame images, ensures the integrity and clarity of the image, and enhances the image through adaptive histogram equalization and multi-scale convolutional neural network to further improve the local contrast and overall quality of the image.

[0102] The present invention detects environmental parameters in real time, evaluates the contrast, brightness and sharpness of the image, and dynamically adjusts the image parameters based on the evaluation results, thereby ensuring that the camera can continuously provide high-quality images under different lighting and rain intensity conditions. The real-time adjustment mechanism enables the system to adapt to environmental changes, optimize image effects, and improve imaging stability and consistency.

[0103] The present invention significantly improves the reliability and practicability of the monitoring system under adverse weather conditions by optimizing and adjusting in combination with environmental changes and real-time image feedback. It adopts data enhancement technology and multi-level feature extraction and repair algorithms to enhance the system's ability to detect and repair raindrop areas. The introduction of hybrid pooling layers and dual loss functions ensures the accuracy of classification and positioning, improves the robustness and efficiency of the model in complex environments, and enables the monitoring system to provide stable and clear monitoring images under various weather conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0105] Figure 1 A schematic diagram of the optimization method flow of an embodiment of the present invention;

[0106] Figure 2 Schematic diagram of raindrop dynamic prediction and repair according to an embodiment of the present invention. DETAILED DESCRIPTION

[0107] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0108] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0109] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0110] like Figure 1-Figure 2 As shown, the camera rainy day vision optimization method includes the following steps:

[0111] S1, rain detection: detect the current weather conditions through environmental sensors and determine whether it is rainy;

[0112] S2, image acquisition and preprocessing: using the camera to acquire image data of the current environment, and preprocessing the acquired image data, including denoising and color correction, to preliminarily improve the image quality;

[0113] S3, raindrop dynamic prediction and repair: Perform raindrop dynamic prediction on the preprocessed image data, identify and predict the area where raindrops will appear in advance, and dynamically repair the image damage caused by raindrops through the image repair algorithm to restore the integrity of the image, including:

[0114] S31, Raindrop Detection: Use the Convolutional Neural Network (CNN) model to detect raindrop areas and identify areas blocked by raindrops in image data;

[0115] S32, dynamic prediction: predict the dynamic changes of the raindrop area and identify the area where raindrops appear in advance to facilitate rapid processing;

[0116] S33, image restoration: after the raindrop area is detected, the image restoration algorithm is used to process the raindrop area, and the area blocked by the raindrop is filled according to the surrounding pixel information;

[0117] S34, multi-frame fusion: through the multi-frame image fusion technology, the unobstructed image data in the continuous frames are partially fused into the current frame to eliminate the influence of raindrops on the single-frame image data;

[0118] S4, image enhancement: image enhancement is performed on the image data after raindrop repair processing, including local contrast enhancement and multi-scale image fusion;

[0119] S5, real-time feedback: the enhanced image data is fed back to the monitoring device in real time for users to observe and use;

[0120] S6, dynamic adjustment: Based on real-time feedback of image data and combined with actual environmental changes, the image data effect is evaluated, including contrast, brightness and sharpening intensity, and dynamic adjustment is made according to the results of the effect evaluation to adapt to different rain intensity and lighting conditions, ensuring the stability and consistency of the optimization effect;

[0121] Through the above steps, the imaging quality of the camera in rainy environments is effectively improved, the clarity and stability of the monitoring images are ensured, and the system can adapt to different rain intensities and lighting conditions, which significantly improves the reliability and practicality of the monitoring system in severe weather conditions.

[0122] Image acquisition and preprocessing in S2 include:

[0123] S21, image acquisition: using a camera to collect image data of the current environment in real time, the image data includes multiple continuous frames;

[0124] S22, denoising: using Gaussian filtering algorithm to denoise the collected image data, expressed as:

[0125] ;

[0126] in, is the pixel value of the denoised image, is the standard deviation of the Gaussian function, is the size of the filter window;

[0127] S23, color correction: use a white balance algorithm to perform color correction on the image data after contrast enhancement, expressed as:

[0128] ;

[0129] ;

[0130] ;

[0131] in, are the red, green, and blue channel values ​​of the original image, is the channel value after correction, are the average values ​​of the red, green and blue channels respectively, is the overall brightness average of the image;

[0132] Through the above content, it is ensured that the collected image data has better basic quality in rainy environments, providing purer and more accurate image data for subsequent image enhancement and restoration, thereby significantly improving the clarity and color accuracy of the final image.

[0133] The convolutional neural network (CNN) models in S31 include:

[0134] S311, data preprocessing and enhancement: The data diversity is increased by using data expansion technology on the input image data, which is expressed as:

[0135] ;

[0136] in, is the original image data, is the preprocessed image data, is a data augmentation function, including rotation, scaling, translation, and mirroring;

[0137] Data augmentation function Specifically include:

[0138] Rotation: Rotate the image by a certain angle, expressed as:

[0139] ;

[0140] in, is the original pixel coordinate, is the pixel coordinate after rotation, is the rotation angle;

[0141] Scaling: Scale the image by a certain ratio, expressed as:

[0142] ;

[0143] in, and are the scaling ratios in the horizontal and vertical directions respectively;

[0144] Translation: The image is translated according to a certain displacement, expressed as:

[0145] ;

[0146] in, and are the translation amounts in the horizontal and vertical directions respectively;

[0147] Mirror: Mirror the image horizontally or vertically, expressed as:

[0148] ;

[0149] ;

[0150] in, is the image width, is the image height;

[0151] Data augmentation function It is expressed as:

[0152] ;

[0153] in, Represents the image By angle To rotate, Represents the image According to the proportion and To zoom, Represents the image According to displacement and To translate, Represents the image Perform horizontal or vertical mirroring operations;

[0154] S312, convolution layer: dilated convolution is introduced in the convolution layer to expand the receptive field without increasing the number of parameters and enhance the model's ability to detect raindrop morphology, expressed as:

[0155] ;

[0156] in, is the convolution output, For input image data, is the convolution kernel, is the expansion rate, It is half the size of the convolution kernel;

[0157] S313, introduction of attention mechanism: Introducing the attention mechanism into the convolutional network to improve the model's attention to important feature areas, especially the sensitivity to raindrop areas, expressed as:

[0158] ;

[0159] ;

[0160] in, is the attention weight, Score feature relevance, is the correlation function, and They are the convolutional layer output and the attention mechanism input respectively;

[0161] Correlation function It is expressed as:

[0162] ;

[0163] in, is the input feature vector, The input vector for the attention mechanism, express The transpose of express The vector itself;

[0164] S314, dual loss function: A dual loss function, including classification loss and positioning loss, is used to improve the accuracy of raindrop area detection, which is expressed as:

[0165] ;

[0166] ;

[0167] ;

[0168] in, is the classification loss, For the positioning loss, is the balance coefficient, is the real category, is the predicted probability, is the true bounding box, To predict the bounding box;

[0169] S315, mixed pooling layer: Introducing the mixed pooling layer in the convolutional network, combining the advantages of maximum pooling and average pooling, improving the robustness of feature extraction and the ability to detect raindrop areas, expressed as:

[0170] ;

[0171] in, is the mixed pooling output, is the maximum pooling output, is the average pooling output, is the weight coefficient, which controls the ratio of maximum pooling and average pooling;

[0172] Through the above contents, the detection and repair capabilities of raindrop areas are greatly enhanced. The dilated convolution expands the receptive field and improves the recognition of raindrop morphology. The attention mechanism enhances the focus on key areas. The dual loss function ensures the accuracy of classification and positioning. The hybrid pooling layer combines the advantages of maximum pooling and average pooling to improve the robustness of feature extraction, enabling the model to process images more accurately and efficiently in complex rainy environments, thereby significantly improving image quality and system reliability.

[0173] Dynamic prediction in S32 includes:

[0174] S321, temporal feature extraction: extract temporal features from continuous frame image data, and use the long short-term memory network (LSTM) to model the image frame sequence, expressed as:

[0175] ;

[0176] in, is the hidden state at the current moment, is the current input frame, is the hidden state at the previous moment, and is the weight matrix, is the bias term, is the activation function;

[0177] S322, predicting raindrop area: using the extracted time series features to dynamically predict the raindrop area, generating the area where raindrops appear in the future frame, expressed as;

[0178] ;

[0179] in, is the predicted raindrop area, is the output layer weight matrix, is the bias term, is the activation function;

[0180] S323, dynamically adjust the prediction result: Combine the characteristics of the current frame image data and the prediction result of the raindrop area, dynamically adjust the raindrop area to improve the accuracy of the prediction, which is expressed as:

[0181] ;

[0182] in, is the adjusted raindrop area prediction result, is the balance coefficient, The features extracted by the convolutional neural network (CNN) for the current frame image data;

[0183] Through the above steps, the dynamic changes of the raindrop area can be accurately predicted, and the raindrop area that may appear in the future frame can be identified in advance, providing a reliable basis for subsequent image restoration, thereby improving the image quality of the camera in rainy environments.

[0184] Image restoration in S33 includes:

[0185] S331, raindrop area detection: After detecting the raindrop area, mark the pixel points in the raindrop occlusion area, expressed as:

[0186] ;

[0187] in, is the binary mask of the raindrop occluded area, 1 represents the raindrop area and 0 represents the normal area;

[0188] S332, neighborhood pixel feature extraction: extract pixel information around the raindrop occlusion area for reference during repair, expressed as:

[0189] ;

[0190] in, The raindrop area The set of surrounding unoccluded pixels, Represents pixels Neighborhood of;

[0191] S333, repair image: Use image repair algorithm to repair the raindrop area, expressed as:

[0192] ;

[0193] in, is the pixel value of the restored image, is a convolutional neural network for image restoration, is the original image data, Mask the raindrop occluded area;

[0194] Convolutional Neural Networks for Image Inpainting Specifically include:

[0195] Input feature extraction: Take the image and raindrop occlusion mask as input, and extract features through the convolution layer, expressed as:

[0196] ;

[0197] ;

[0198] in, is the original image, is the raindrop occlusion area mask, and is the convolutional layer weight, and is the bias, * indicates the convolution operation, is the activation function, and are the feature maps of the image and mask respectively;

[0199] Feature fusion: The extracted image features and mask features are fused, expressed as:

[0200] ;

[0201] in, represents the feature concatenation operation, is the fused feature map;

[0202] Feature recovery: Through multiple layers of convolution and deconvolution layers, the repaired image is restored and generated, which is expressed as:

[0203] ;

[0204] ;

[0205] in, is the convolutional layer weight, is the bias, represents the deconvolution operation, is the pixel value of the restored image;

[0206] Loss function: A combination of adversarial loss and reconstruction loss is used to train the image restoration network, expressed as:

[0207] ;

[0208] ;

[0209] ;

[0210] in, For adversarial loss, is the reconstruction loss, is the discriminator, and is the balance coefficient, represents mathematical expectation;

[0211] S334, pixel value filling: fill the area blocked by raindrops according to the neighborhood pixel information and the output of the image restoration algorithm, expressed as:

[0212] ;

[0213] in, is the output repair pixel value, is the average value of unoccluded pixels in the neighborhood, is the balance coefficient;

[0214] Through the above content, combined with feature extraction and repair capabilities, and the effective supplement of neighborhood pixel information, it is possible to effectively repair image areas blocked by raindrops in rainy environments and significantly improve the image quality of the camera.

[0215] Multi-frame fusion in S34 includes:

[0216] S341, feature extraction: extract features from continuous frame image data, and extract the features of each frame image data through the convolutional neural network (CNN) model, expressed as:

[0217] ;

[0218] in, For the Frame image data Extracted feature map;

[0219] S342, feature alignment: align the features of the continuous frame image data, estimate the inter-frame motion using the optical flow algorithm, and align the feature maps, which is expressed as:

[0220] ;

[0221] in, For Frame to Optical flow estimation of frames, is the alignment operation of the feature map, is the aligned feature map;

[0222] S343, weight calculation: Calculate the weight of each frame of image data, and assign fusion weights according to the clarity of the image data and the raindrop occlusion, expressed as:

[0223] ;

[0224] in, For the The fusion weight of the frame, For the Raindrop intensity of frame image data, To adjust the parameters;

[0225] S344, image fusion: weighted fusion of features of multiple frames of image data to generate current frame image data after raindrop removal, expressed as:

[0226] ;

[0227] ;

[0228] in, is the fused feature map, is the image data after removing raindrops, To decode the convolutional neural network;

[0229] Decoding Convolutional Neural Networks Specifically include:

[0230] Deconvolution layer: The input feature map is converted into a higher resolution image through a deconvolution operation, which is expressed as:

[0231] ;

[0232] in, is the image after deconvolution, is the deconvolution kernel weight, is the input feature map, is the bias, represents the deconvolution operation, is the ReLU activation function;

[0233] Upsampling layer: The feature map is enlarged by upsampling operation, and then a convolution operation is performed, which is expressed as:

[0234] ;

[0235] in, is the upsampled image, is the convolution kernel weight, is the upsampling operation (bilinear interpolation), is the input feature map, is the bias, is the convolution operation, is the ReLU activation function;

[0236] Through the above steps, the multi-frame fusion technology can utilize the information of continuous frame images to effectively eliminate the influence of raindrops on single-frame images, generate clear raindrop-free images, and improve the image quality of the camera in rainy environments.

[0237] Image enhancements in S4 include:

[0238] S41, local contrast enhancement: Adaptive histogram equalization algorithm (CLAHE) is used to perform local contrast enhancement on the repaired image data, which is expressed as:

[0239] ;

[0240] in, The restored image data is at position The pixel value at is the enhanced pixel value, is the cumulative distribution function, is the minimum non-zero cumulative distribution value, is the gray level;

[0241] S42, multi-scale feature extraction: extract multi-scale features from the enhanced image data, and extract image features at different scales through a multi-scale convolutional neural network (MSCNN), expressed as: ;

[0242] in, For scale The feature map extracted below, It is a multi-scale convolutional neural network;

[0243] Multi-scale Convolutional Neural Network Specifically include:

[0244] Input image: the original input image ;

[0245] Multi-scale convolution layer: Convolution operation is performed on the input image using convolution kernels of different scales, expressed as: ;

[0246] in, For scale Next The feature map extracted by the convolution kernel, For scale Next The weight of the convolution kernel, For scale Next The bias of the convolution kernel, is the ReLU activation function, Represents the convolution operation;

[0247] Feature map fusion: The feature maps extracted by multiple convolution kernels at the same scale are fused, expressed as: ;

[0248] in, For scale The fused feature map below: For scale The number of convolution kernels under

[0249] S43, multi-scale image fusion: feature maps at different scales are fused to generate enhanced image data, expressed as:

[0250] ;

[0251] ;

[0252] in, is the feature map after multi-scale fusion, is the enhanced image data, is the weight coefficient at different scales, To decode the convolutional neural network;

[0253] Through the above steps, the repaired image can be further enhanced to improve the local contrast and overall quality of the image, making the image captured by the camera in a rainy environment clearer and easier to analyze.

[0254] Dynamic adjustments in S6 include:

[0255] S61, environmental parameter detection: real-time detection of environmental parameters, including light intensity and rain intensity, through sensors, and recording of environmental parameters for subsequent use;

[0256] S62, image effect evaluation: based on the real-time detected environmental parameters, the image effect of the current image data is evaluated, including the contrast, brightness and sharpness of the image data;

[0257] S63, Image Adjustment: Based on the results of image effect evaluation, dynamically adjust the contrast, brightness and sharpness of image data to achieve the best image quality;

[0258] Through the above content, it is ensured that the camera can continue to provide high-quality images under different lighting and rain intensity conditions. Combined with environmental changes and real-time image feedback, the image parameters are optimized to make the imaging clearer and more stable, thereby improving the reliability and practicality of the monitoring system in severe weather conditions.

[0259] Image quality evaluation in S62 includes:

[0260] S621, contrast evaluation: Based on the real-time detected environmental parameters, the standard deviation method is used to calculate the contrast of the current image data to measure the dispersion of the image grayscale value, expressed as: ;

[0261] in, is the image contrast, is the total number of image pixels, For the The gray value of a pixel, is the average gray value of the image;

[0262] S622, brightness evaluation: Based on the real-time detected environmental parameters, the brightness of the current image data is calculated using the average gray value method to measure the overall brightness of the image, expressed as: ;

[0263] in, is the image brightness, is the total number of image pixels, For the The gray value of pixels;

[0264] S623, sharpening strength evaluation: Based on the real-time detected environmental parameters, the sharpening strength of the current image data is calculated using the Laplace operator method to measure the clarity of the image edge, which is expressed as: ;

[0265] in, is the image sharpening strength, is the total number of image pixels, For the The Laplacian result of pixels;

[0266] Through the above content, accurate image quality parameters can be provided, providing a basis for subsequent image optimization and dynamic adjustment, thereby ensuring that the camera can continuously output high-quality images under different environmental conditions.

[0267] Image adjustments in the S63 include:

[0268] S631, contrast adjustment: Based on the contrast result of image effect evaluation, the contrast of image data is dynamically adjusted through linear transformation, which is expressed as: ;

[0269] in, For the adjusted image, is the original image, is the contrast adjustment factor, is the average gray value of the image;

[0270] S632, brightness adjustment: Based on the brightness result of the image effect evaluation, the brightness of the image data is dynamically adjusted by adjusting the gray value of the image, which is expressed as: ;

[0271] in, For the adjusted image, is the original image, is the brightness adjustment amount;

[0272] S633, sharpening intensity adjustment: Based on the sharpening intensity result of the image effect evaluation, the sharpening intensity of the image data is dynamically adjusted through the convolution operation, which is expressed as: ;

[0273] in, For the adjusted image, is the original image, is the sharpening strength adjustment factor, is the Laplacian operator result of the image;

[0274] Through the above steps, image adjustment can adjust the contrast, brightness and sharpness of the image in real time based on the image effect evaluation to ensure the best quality of the image under different environmental conditions.

[0275] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0276] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A camera rainy day vision optimization method, characterized in that: The following steps are involved: S1, rain detection: detect the current weather conditions through environmental sensors and determine whether it is rainy; S2, image acquisition and preprocessing: using the camera to acquire image data of the current environment, and preprocessing the acquired image data, including denoising and color correction; S3, raindrop dynamic prediction and repair: Perform raindrop dynamic prediction on the preprocessed image data, identify and predict the area where raindrops will appear in advance, and dynamically repair the image damage caused by raindrops through the image repair algorithm, including: S31, raindrop detection: using the convolutional neural network model to detect raindrop areas and identify areas blocked by raindrops in the image data; S32, dynamic prediction: predict the dynamic changes of the raindrop area and identify the area where raindrops appear in advance; S33, image restoration: after the raindrop area is detected, the image restoration algorithm is used to process the raindrop area, and the area blocked by the raindrop is filled according to the surrounding pixel information; S34, multi-frame fusion: through the multi-frame image fusion technology, the unobstructed image data in the continuous frames are partially fused into the current frame to eliminate the influence of raindrops on the single-frame image data; S4, image enhancement: image enhancement is performed on the image data after raindrop repair processing, including local contrast enhancement and multi-scale image fusion; S5, real-time feedback: feeding back the enhanced image data to the monitoring equipment in real time; S6, dynamic adjustment: Based on the real-time feedback of image data and the actual environmental changes, the image data effect is evaluated, including contrast, brightness and sharpening intensity, and dynamic adjustment is made according to the results of the effect evaluation; The convolutional neural network model in S31 includes: S311, data preprocessing and enhancement: The data diversity is increased by using data expansion technology on the input image data, which is expressed as: ； in, is the original image data, is the preprocessed image data, is the data enhancement function; S312, convolution layer: dilated convolution is introduced in the convolution layer to expand the receptive field without increasing the number of parameters, expressed as: ; in, is the convolution output, For input image data, is the convolution kernel, is the expansion rate, is half the size of the convolution kernel; S313, introduction of attention mechanism: Introducing the attention mechanism in the convolutional network, expressed as: ； ； in, is the attention weight, Score feature relevance, is the correlation function, and They are the convolutional layer output and the attention mechanism input respectively; S314, dual loss function: A dual loss function is used, including classification loss and positioning loss, expressed as: ； ； ； in, is the classification loss, is the positioning loss, is the balance coefficient, is the real category, is the predicted probability, is the true bounding box, To predict the bounding box; S315, mixed pooling layer: Introduce the mixed pooling layer in the convolutional network, expressed as: ； in, is the mixed pooling output, is the maximum pooling output, is the average pooling output, is the weight coefficient; The dynamic prediction in S32 includes: S321, temporal feature extraction: extract temporal features from continuous frame image data, and use long short-term memory network to model the image frame sequence, expressed as: ； in, is the hidden state at the current moment, is the current input frame, is the hidden state at the previous moment, and is the weight matrix, is the bias term, is the activation function; S322, predicting raindrop area: using the extracted time series features to dynamically predict the raindrop area, generating the area where raindrops appear in the future frame, expressed as; ； in, is the predicted raindrop area, is the output layer weight matrix, is the bias term, is the activation function; S323, dynamically adjust the prediction result: Combine the characteristics of the current frame image data and the prediction result of the raindrop area to dynamically adjust the raindrop area, which is expressed as: ； in, is the adjusted raindrop area prediction result, is the balance coefficient, The features extracted by the convolutional neural network for the current frame image data; The image restoration in S33 includes: S331, raindrop area detection: After detecting the raindrop area, mark the pixel points in the raindrop occlusion area, expressed as: ； in, is the binary mask of the raindrop occluded area, 1 represents the raindrop area and 0 represents the normal area; S332, neighborhood pixel feature extraction: extract pixel information around the raindrop occlusion area for reference during repair, expressed as: ； in, The raindrop area The set of surrounding unoccluded pixels, Represents pixels Neighborhood of; S333, repair image: Use image repair algorithm to repair the raindrop area, expressed as: ； in, is the pixel value of the restored image, is a convolutional neural network for image restoration, is the original image data, Mask the raindrop occluded area; S334, pixel value filling: fill the area blocked by raindrops according to the neighborhood pixel information and the output of the image restoration algorithm, expressed as: ； in, is the output repair pixel value, is the average value of unoccluded pixels in the neighborhood, is the balance coefficient.

2. The camera rainy day vision optimization method according to claim 1 is characterized in that: The image acquisition and preprocessing in S2 include: S21, image acquisition: using a camera to collect image data of the current environment in real time, the image data includes multiple continuous frames; S22, denoising: performing denoising on the collected image data using a Gaussian filtering algorithm; S23, color correction: performing color correction on the image data after contrast enhancement using a white balance algorithm.

3. The camera rainy day vision optimization method according to claim 1 is characterized in that: The multi-frame fusion in S34 includes: S341, feature extraction: extract features from continuous frame image data, and extract the features of each frame image data through the convolutional neural network model, expressed as: ； in, For the Frame image data Extracted feature map; S342, feature alignment: align the features of the continuous frame image data, estimate the inter-frame motion using the optical flow algorithm, and align the feature maps, which is expressed as: ； in, For Frame to Optical flow estimation of frames, is the alignment operation of the feature map, is the aligned feature map; S343, weight calculation: Calculate the weight of each frame of image data, and assign fusion weights according to the clarity of the image data and the raindrop occlusion, expressed as: ； in, For the The fusion weight of the frame, For the Raindrop intensity of frame image data, To adjust the parameters; S344, image fusion: weighted fusion of features of multiple frames of image data to generate current frame image data after raindrop removal, expressed as: ； ； in, is the fused feature map, is the image data after removing raindrops, Convolutional neural network for decoding.

4. The camera rainy day vision optimization method according to claim 3 is characterized in that: The image enhancement in S4 includes: S41, local contrast enhancement: Adaptive histogram equalization algorithm is used to perform local contrast enhancement on the repaired image data, which is expressed as: ； in, The restored image data is at position The pixel value at is the enhanced pixel value, is the cumulative distribution function, is the minimum non-zero cumulative distribution value, is the gray level; S42, multi-scale feature extraction: extract multi-scale features from the enhanced image data, and extract image features at different scales through a multi-scale convolutional neural network, expressed as: ； in, For scale The feature map extracted below, It is a multi-scale convolutional neural network; S43, multi-scale image fusion: feature maps at different scales are fused to generate enhanced image data, expressed as: ； ； in, is the feature map after multi-scale fusion, is the enhanced image data, is the weight coefficient at different scales, Convolutional neural network for decoding.

5. The camera rainy day vision optimization method according to claim 1 is characterized in that: The dynamic adjustment in S6 includes: S61, environmental parameter detection: real-time detection of environmental parameters, including light intensity and rain intensity, through sensors; S62, image effect evaluation: based on the real-time detected environmental parameters, the image effect of the current image data is evaluated, including the contrast, brightness and sharpness of the image data; S63, Image Adjustment: Dynamically adjust the contrast, brightness and sharpness of the image data based on the results of the image effect evaluation.

6. The camera rainy day vision optimization method according to claim 5 is characterized in that: The image effect evaluation in S62 includes: S621, contrast evaluation: based on the real-time detected environmental parameters, the contrast of the current image data is calculated using the standard deviation method; S622, brightness evaluation: based on the real-time detected environmental parameters, the brightness of the current image data is calculated using the average gray value method; S623, sharpening strength evaluation: based on the real-time detected environmental parameters, the sharpening strength of the current image data is calculated using the Laplace operator method.

7. The camera rainy day vision optimization method according to claim 6 is characterized in that: The image adjustment in S63 includes: S631, contrast adjustment: dynamically adjusting the contrast of the image data through linear transformation based on the contrast result of the image effect evaluation; S632, brightness adjustment: based on the brightness result of the image effect evaluation, dynamically adjust the brightness of the image data by adjusting the gray value of the image; S633, sharpening intensity adjustment: based on the sharpening intensity result of the image effect evaluation, the sharpening intensity of the image data is dynamically adjusted through a convolution operation.

Images