Lens dirt monitoring method, computer equipment and storage medium

Abstract

The invention discloses a lens smudginess monitoring method, computer equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: controlling an image collection device to shoot a plurality of images with different exposure degrees, and forming an image sequence from overexposure to underexposure; processing the image sequence, and respectively extracting a smudginess feature based on pixel brightness variance, a smudginess feature based on block brightness gradient change and a smudginess feature based on frequency domain enhancement; and fusing the extracted multiple smudginess features to generate smudginess area information. By collecting a multi-exposure image sequence and respectively extracting pixel brightness variance, block brightness gradient change and frequency domain enhancement features, light transmission abnormity, local response abnormity and edge texture features of a dirty area in exposure change are effectively captured, multi-dimensional features are fused, the accuracy, sensitivity and robustness of recognition of various pollutants are improved, and the recognition accuracy, sensitivity and robustness of various pollutants are improved. The problem of misjudgment of single-frame detection in a complex environment is solved, and reliable and comprehensive monitoring of lens smudginess is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for monitoring lens dirt, a computer device, and a storage medium. Background Technology

[0002] Imaging devices deployed in open environments often expose their camera lenses to complex and changing natural conditions, inevitably leading to the accumulation of various contaminants such as rainwater, dust, mud, and snowflakes. These contaminants, once covering the lens surface, significantly degrade image quality, causing prominent issues such as blurry images, loss of detail, and reduced contrast. Especially in applications with extremely high visual reliability requirements, such as outdoor action cameras, security monitoring, and autonomous driving, lens contamination not only affects basic imaging performance but also further interferes with the performance of advanced visual algorithms such as facial recognition, behavior analysis, and object tracking, potentially leading to functional failures or even safety hazards.

[0003] Currently, lens cleanliness maintenance primarily relies on traditional manual inspection methods. This approach is not only inefficient but also requires significant investment of manpower and resources, especially in applications where equipment is widely distributed, in harsh environments, or where frequent access is difficult, posing serious challenges to both feasibility and economy. Therefore, developing an efficient and reliable automated lens contamination detection technology to achieve real-time monitoring and timely alarms is of great practical significance for ensuring the continuous and stable operation of imaging systems.

[0004] While several automated detection solutions have been proposed, such as traditional algorithms based on single-frame image processing attempting to identify smudged areas through gradient analysis or frequency domain transformation, or machine learning methods relying on large amounts of labeled data to train classification models to determine dirt levels, and optical aids such as changes in infrared reflectance intensity to indirectly detect changes in lens transmittance, each method has its limitations. For example, single-frame analysis methods struggle to handle interference from complex scenes or low-contrast environments; machine learning methods heavily rely on the quality and scale of training samples and have poor generalization ability across different scenarios; and optical solutions require additional hardware modules, leading to increased costs and integration complexity.

[0005] In summary, existing lens contamination detection technologies generally suffer from problems such as weak environmental adaptability, limited detection accuracy, high implementation costs, or poor system compatibility. Therefore, there is an urgent need to propose a lens contamination monitoring method to effectively ensure the reliable operation of vision systems in complex and open environments. Summary of the Invention

[0006] This invention provides a method, computer device, and storage medium for monitoring lens dirt, aiming to solve the problem of how to accurately detect lens dirt.

[0007] In a first aspect, embodiments of the present invention provide a method for monitoring lens contamination, comprising:

[0008] S1. Control the image acquisition device to capture multiple images with different exposure levels, forming an image sequence from overexposed to underexposed;

[0009] S2. Process the image sequence to extract dirt features based on pixel brightness variance, dirt features based on block brightness gradient changes, and dirt features based on frequency domain enhancement, respectively.

[0010] S3. Combine the various dirt features extracted in step S2 to generate dirt area information.

[0011] A further technical solution is that, in step S1, images with different exposures are obtained by adjusting the brightness of the fill light or adjusting the exposure parameters of the image acquisition device; before step S1, the step also includes: turning off the white balance and automatic image quality enhancement algorithm of the image acquisition device.

[0012] A further technical solution is that, in step S2, dirt features based on pixel brightness variance are extracted, including:

[0013] Calculate the luminance variance of each pixel across all images in the image sequence;

[0014] Thresholding is performed on the calculated variance values ​​to obtain the first dirt mask.

[0015] A further technical solution is that the threshold segmentation of the calculated variance value includes:

[0016] An adaptive thresholding algorithm is used for threshold segmentation, and morphological opening operations are performed on the segmentation results to remove noise.

[0017] A further technical solution is that, in step S2, dirt features based on changes in block brightness gradient are extracted, including:

[0018] Each image is divided into multiple blocks;

[0019] For each block, calculate its brightness variation curve in the image sequence;

[0020] Calculate the gradient difference in brightness change between each block and its neighboring blocks;

[0021] Blocks with gradient differences exceeding a set gradient threshold are marked as suspicious dirty blocks, forming a second dirty mask.

[0022] A further technical solution is that, in step S2, the dirt features based on frequency domain enhancement are extracted, including:

[0023] Each image in the image sequence is subjected to high-pass filtering to obtain the corresponding high-frequency enhanced image;

[0024] The obtained multiple high-frequency enhanced images are then superimposed.

[0025] Threshold segmentation is performed on the superimposed results to obtain the third dirt mask.

[0026] A further technical solution is that performing high-pass filtering on each image in the image sequence includes:

[0027] Gaussian blurring is applied to the images in the image sequence to obtain low-frequency components;

[0028] Subtract the low-frequency component from the image in the image sequence to obtain the high-frequency component;

[0029] Histogram equalization is performed on the high-frequency components to enhance them.

[0030] A further technical solution is that the fusion in step S3 involves performing a logical AND operation on the first dirty mask obtained based on the pixel brightness variance, the second dirty mask obtained based on the block brightness gradient change, and the third dirty mask obtained based on frequency domain enhancement, and taking the intersection region as the dirty region information.

[0031] Secondly, embodiments of the present invention also provide a lens dirt monitoring device, which includes a unit for performing the above-described method.

[0032] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0033] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0034] This invention provides a method, computer device, and storage medium for monitoring lens contamination. The method includes: controlling an image acquisition device to capture multiple images at different exposure levels, forming an image sequence from overexposed to underexposed; processing the image sequence to extract contamination features based on pixel brightness variance, block brightness gradient changes, and frequency domain enhancement; and fusing the extracted contamination features to generate contamination area information. This invention effectively captures abnormal light transmittance, abnormal local response, and edge texture features of contamination areas during exposure changes by acquiring multiple exposure image sequences and extracting pixel brightness variance, block brightness gradient changes, and frequency domain enhancement features. By fusing multi-dimensional features, the accuracy, sensitivity, and robustness of identifying various contaminants are significantly improved, overcoming the misjudgment problem of single-frame detection in complex environments, and achieving reliable and comprehensive monitoring of lens contamination. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A schematic flowchart illustrating a method for monitoring lens contamination according to an embodiment of the present invention;

[0037] Figure 2 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0040] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0041] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0042] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0043] Please see Figure 1 This invention provides a method for monitoring lens contamination, the method comprising the following steps:

[0044] S1. Control the image acquisition device to capture multiple images with different exposure levels, forming an image sequence from overexposed to underexposed.

[0045] In practice, multiple images with different exposure levels are captured by controlling the image acquisition device, forming an image sequence from overexposed to underexposed. This method effectively expands the dynamic range of image information, providing rich and continuous illumination response data for subsequent processing. This step enables dirt with different light transmittance characteristics to exhibit distinguishable brightness features under different exposure conditions, laying a data foundation for accurately identifying various pollutants.

[0046] In some preferred embodiments, in step S1, images with different exposures are obtained by adjusting the brightness of the fill light or adjusting the exposure parameters of the image acquisition device; before step S1, the method further includes the step of turning off the white balance and automatic image enhancement algorithm of the image acquisition device.

[0047] In this embodiment, images with different exposure levels are obtained by adjusting the brightness of the fill light or the exposure parameters of the image acquisition device, and white balance and automatic image enhancement algorithms are turned off. This step ensures the originality and consistency of the acquired image sequence, avoids interference from automatic algorithms on the image content, and thus provides reliable data input for subsequent feature extraction.

[0048] Adjusting the fill light or exposure parameters is a flexible and low-cost approach suitable for different types of image acquisition devices, enhancing the method's versatility and deployability. Disabling the automatic optimization algorithm ensures that the image response curve accurately reflects the physical state of the lens surface, further improving the recognizability and detection accuracy of dirt features.

[0049] S2. Process the image sequence to extract dirt features based on pixel brightness variance, dirt features based on block brightness gradient changes, and dirt features based on frequency domain enhancement.

[0050] In practice, by processing the image sequence and extracting dirt features based on pixel brightness variance, the system can effectively capture the sluggish brightness changes in dirty areas due to decreased light transmittance. Normal areas exhibit significant brightness fluctuations with exposure changes, while dirty areas show significantly reduced brightness changes due to light obstruction. This characteristic is quantified through variance calculation, thus enabling preliminary localization of the contaminated area.

[0051] Furthermore, extracting dirt features based on block brightness gradient changes enables the system to identify anomalies from local response patterns. By dividing the image into multiple blocks and analyzing the brightness change curve of each block in the exposure sequence, this method can detect brightness distribution anomalies caused by dirt. The gradient difference calculation between blocks effectively highlights the response differences between the dirty area and the surrounding environment, enhancing the sensitivity to identify low-contrast or fine contaminants.

[0052] Furthermore, frequency-domain enhanced dirt feature extraction uses high-pass filtering to highlight high-frequency details in the image, enhancing the salience of dirt edges and textures. Dirty areas often have distinct contour features, and frequency-domain processing methods can effectively separate this high-frequency information from low-frequency background content, thereby improving the ability to identify contaminants with obvious morphological features, such as fingerprints and water stains.

[0053] In some preferred embodiments, step S2, extracting dirt features based on pixel brightness variance, includes the following steps: calculating the brightness variance of each pixel across all images in the image sequence; and performing threshold segmentation on the calculated variance values ​​to obtain a first dirt mask.

[0054] This embodiment defines the specific steps for extracting dirt features based on pixel brightness variance, including calculating the brightness variance of each pixel in the image sequence and performing threshold segmentation on the variance values ​​to obtain a first dirt mask. Since the brightness response of dirty areas is weak during exposure changes, their variance values ​​are generally lower than those of surrounding normal areas. This method can effectively capture this statistical characteristic, thereby achieving preliminary dirt area localization. Through pixel-level variance calculation, areas with abnormal light transmittance can be identified globally, exhibiting strong resistance to scene interference, and is particularly suitable for dirt detection against uniform backgrounds.

[0055] In some preferred embodiments, the above step "thresholding the calculated variance value" specifically includes the following steps: performing thresholding using an adaptive thresholding algorithm, and performing morphological opening operations on the segmentation results to remove noise.

[0056] In this embodiment, an adaptive thresholding algorithm (such as the Otsu algorithm, which is not specifically limited in this invention) is used for variance segmentation to binarize low-variance regions, and morphological opening operations are performed on the results. The adaptive thresholding dynamically adjusts the segmentation threshold based on local image characteristics, adapting to changes in lighting and scene content, thereby improving the accuracy and stability of threshold segmentation. The morphological opening operation effectively removes false detection regions caused by noise or minute textures, smooths the mask contour, improves the integrity and consistency of dirty areas, and further enhances the robustness and practicality of the method.

[0057] Specifically, the adaptive thresholding algorithm can employ the Otsu's algorithm. The Otsu's algorithm automatically analyzes the gray-level histogram distribution of this "variance image" to find an optimal threshold. This threshold maximizes the inter-class variance between the two classes of pixels segmented by it (i.e., the low-variance class and the high-variance class). In other words, the algorithm finds the critical value that best distinguishes between "slow-changing" and "normal-changing" regions.

[0058] Binarization result: After applying the optimal threshold calculated by Otsu, the variance image is converted into a binary mask (first dirt mask). Low-variance pixels (values ​​below the threshold) are marked white (e.g., representing the foreground or target, i.e., suspected dirt areas), while high-variance pixels (values ​​above the threshold) are marked black (representing the background). This initially separates potential dirt areas.

[0059] In some preferred embodiments, step S2, extracting dirt features based on block brightness gradient changes, specifically includes the following steps: dividing each image into multiple blocks; for each block, calculating its brightness change curve in the image sequence; calculating the gradient difference between each block and its neighboring blocks in terms of brightness change; marking blocks with gradient differences exceeding a set gradient threshold as suspected dirty blocks, forming a second dirt mask. The gradient threshold can be dynamically set according to experimental data or actual application scenarios, and is not specifically limited in this invention.

[0060] In this embodiment, specific steps are defined for extracting dirt features based on changes in block brightness gradients, including image segmentation, calculation of the brightness change curve of each block in the exposure sequence, calculation of the gradient difference between blocks, and marking of suspicious areas according to gradient thresholds.

[0061] The formula for determining the gradient difference of a suspicious region is: Gradient = |Mean(B_i) - Mean(B_j)| > T; where Gradient is the gradient difference, B_i is the current block, B_j is the neighboring block, T is the gradient threshold, Mean(B_i) is the average brightness of the current block B_i, and Mean(B_j) is the average brightness of the neighboring block B_j.

[0062] The purpose of the above formula is to identify areas with abnormally abrupt changes in brightness that are inconsistent with the surrounding environment, and to mark them as suspected dirty areas.

[0063] Technical principle: In a sequence of images from overexposed to underexposed, the brightness changes of adjacent areas in a natural scene captured by a clean lens are typically gradual and continuous (e.g., the gradient from sky to trees). Even with edges, the gradient changes conform to physical laws.

[0064] The manifestation of dirt: Dirty areas (such as a clump of mud or a drop of water) are foreign objects adhering to the lens surface. They locally and abnormally alter light transmission, causing abrupt and unnatural differences in brightness between the area (B_i) where the dirt is located and the corresponding area (B_j) of the surrounding clean area. This difference persists with exposure changes.

[0065] Formula workflow:

[0066] The system iterates through each block B_i in the image.

[0067] For each B_i, calculate the absolute value of the average brightness difference between it and one or more neighboring blocks B_j.

[0068] The following formula is used for judgment: If the difference is greater than the threshold T, it means that the B_i region is extremely inconsistent with the surrounding environment and there is an abnormal abrupt change in brightness.

[0069] Based on this, block B_i is marked as a suspected dirty block, and finally all suspected dirty blocks are merged to generate a second dirty mask.

[0070] In summary, the above formula is an anomaly detector. It effectively captures unnatural local brightness abrupt changes caused by lens dirt by quantifying and judging whether the brightness gradient between local areas exceeds a reasonable natural threshold. It is one of the key technical means to achieve high-precision dirt detection.

[0071] This method analyzes the response patterns of local areas during exposure to identify abnormal brightness changes caused by dirt, thus achieving dirt detection at the block level. Gradient difference calculation effectively highlights the response differences between dirt and the surrounding environment, enhancing the ability to identify low-contrast dirt. At the same time, block-based processing reduces computational complexity and improves the execution efficiency of the method.

[0072] In some preferred embodiments, step S2, extracting dirt features based on frequency domain enhancement, specifically includes the following steps: performing high-pass filtering on each image in the image sequence to obtain a corresponding high-frequency enhanced image; superimposing multiple high-frequency enhanced images; and performing threshold segmentation on the superimposed result to obtain a third dirt mask.

[0073] This embodiment defines the steps for extracting dirt features based on frequency domain enhancement, including high-pass filtering of each image, superimposing multiple high-frequency enhanced images, and thresholding the superimposed results to obtain a third dirt mask. High-pass filtering effectively extracts high-frequency detail components in the image, highlighting dirt edges and texture features; multi-image superimposition strengthens the consistency of these high-frequency components in the sequence, further distinguishing dirt from scene content; finally, thresholding achieves quantitative identification of dirt regions. Specifically, an adaptive thresholding algorithm (such as the Otsu algorithm) is used to binarize the high-frequency regions to obtain the third dirt mask. This method is particularly suitable for detecting dirt types with obvious edge or texture features, such as dust and water droplets, and maintains high detection accuracy even in complex backgrounds.

[0074] Specifically, the adaptive thresholding algorithm can employ the Otsu algorithm. The Otsu algorithm analyzes the grayscale histogram of the "high-frequency enhanced image." It automatically calculates an optimal threshold that best classifies the pixels in the image into two categories: pixels with significant high-frequency features (brighter) and pixels with insignificant high-frequency features (darker).

[0075] Binarization result: After applying this threshold, the high-frequency enhanced image is converted into another binary mask (the third dirt mask). Pixels with strong high-frequency components and high brightness (values ​​above the threshold) are marked as white (representing the target, i.e., the edges / textures of suspected dirt), while low-frequency background areas are marked as black. This yields the outline of the dirt region composed of prominent edges.

[0076] In some preferred embodiments, the above step "perform high-pass filtering on each image in the image sequence" specifically includes the following steps: performing Gaussian blur processing on the images in the image sequence to obtain low-frequency components; subtracting the low-frequency components from the images in the image sequence to obtain high-frequency components; and performing histogram equalization processing on the high-frequency components to achieve enhancement.

[0077] This embodiment clarifies the specific steps of high-pass filtering, including Gaussian blurring to obtain low-frequency components, subtracting low-frequency components from the original image to obtain high-frequency components, and histogram equalization to enhance the high-frequency components. This processing effectively separates low-frequency background information and high-frequency detail components in the image, and enhances the contrast of high-frequency regions through histogram equalization, making dirt features more prominent. This method strengthens the difference between dirt and normal areas in the frequency domain, improves the detection capability of fine and transparent dirt, and has strong resistance to changes in illumination and background interference.

[0078] S3. Combine the various dirt features extracted in step S2 to generate dirt area information.

[0079] In practice, by fusing the aforementioned multiple features to generate information on contaminated areas, the system can comprehensively utilize multi-dimensional information in the spatial, temporal, and frequency domains, significantly improving the accuracy and robustness of the detection results. The feature fusion process effectively avoids the limitations of single-feature detection methods, and reduces the probability of false alarms and missed detections through multi-evidence joint decision-making, enabling the system to adapt to detection needs in various complex environments, ultimately achieving reliable and comprehensive monitoring of lens contamination.

[0080] In some preferred embodiments, the above step "integrating multiple dirt features extracted in step S2 to generate dirt region information" specifically includes the following steps: performing a logical AND operation on the first dirt mask obtained based on pixel brightness variance, the second dirt mask obtained based on block brightness gradient change, and the third dirt mask obtained based on frequency domain enhancement, and taking the intersection region as the dirt region information.

[0081] In this embodiment, a logical AND operation is performed on the first, second, and third dirt masks, and the intersection region is taken as the final dirt region information. This fusion strategy fully utilizes the complementarity of different feature extraction methods: variance features are good at identifying regions with abnormal light transmission, gradient features are good at capturing local response anomalies, and frequency domain features enhance edge and texture information. Through the logical AND operation, only candidate regions jointly indicated by the three features are retained, which greatly improves the reliability and accuracy of the detection results, significantly reduces the false alarm rate, and ensures the integrity and consistency of the output dirt region, thereby enhancing the reliability and practical value of the entire monitoring system.

[0082] Furthermore, if the area of ​​the intersection region is greater than a preset area threshold, it is determined that the lens is dirty, and a lens cleaning reminder message is issued to remind the user to clean the lens in time.

[0083] This invention proposes a method for monitoring lens contamination, comprising: controlling an image acquisition device to capture multiple images at different exposure levels, forming an image sequence from overexposed to underexposed; processing the image sequence to extract contamination features based on pixel brightness variance, block brightness gradient changes, and frequency domain enhancement; and fusing the extracted contamination features to generate contamination area information. This invention effectively captures the abnormal light transmittance, abnormal local response, and edge texture features of contamination areas during exposure changes by acquiring multiple exposure image sequences and extracting pixel brightness variance, block brightness gradient changes, and frequency domain enhancement features. By fusing multi-dimensional features, the accuracy, sensitivity, and robustness of identifying various contaminants are significantly improved, overcoming the misjudgment problem of single-frame detection in complex environments, and achieving reliable and comprehensive monitoring of lens contamination.

[0084] Please see Figure 2 , Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0085] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0086] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to perform a method for monitoring lens contamination.

[0087] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0088] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can perform a method for monitoring lens dirt.

[0089] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0090] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following steps:

[0091] S1. Control the image acquisition device to capture multiple images with different exposure levels, forming an image sequence from overexposed to underexposed;

[0092] S2. Process the image sequence to extract dirt features based on pixel brightness variance, dirt features based on block brightness gradient changes, and dirt features based on frequency domain enhancement, respectively.

[0093] S3. Combine the various dirt features extracted in step S2 to generate dirt area information.

[0094] In some preferred embodiments, in step S1, images with different exposures are obtained by adjusting the brightness of the fill light or adjusting the exposure parameters of the image acquisition device; before step S1, the method further includes the step of turning off the white balance and automatic image enhancement algorithm of the image acquisition device.

[0095] In some preferred embodiments, step S2, extracting dirt features based on pixel brightness variance, includes:

[0096] Calculate the luminance variance of each pixel across all images in the image sequence;

[0097] Thresholding is applied to the calculated variance values ​​to obtain the first dirt mask.

[0098] In some preferred embodiments, thresholding the calculated variance values ​​includes:

[0099] An adaptive thresholding algorithm is used for threshold segmentation, and morphological opening operations are performed on the segmentation results to remove noise.

[0100] In some preferred embodiments, step S2, extracting dirt features based on block brightness gradient changes, includes:

[0101] Each image is divided into multiple blocks;

[0102] For each block, calculate its brightness variation curve in the image sequence;

[0103] Calculate the gradient difference in brightness change between each block and its neighboring blocks;

[0104] Blocks with gradient differences exceeding a set gradient threshold are marked as suspicious dirty blocks, forming a second dirty mask.

[0105] In some preferred embodiments, step S2, extracting dirt features based on frequency domain enhancement, includes:

[0106] Each image in the image sequence is subjected to high-pass filtering to obtain the corresponding high-frequency enhanced image;

[0107] The obtained multiple high-frequency enhanced images are then superimposed.

[0108] Threshold segmentation is performed on the superimposed results to obtain the third dirt mask.

[0109] In some preferred embodiments, the high-pass filtering process for each image in the image sequence includes:

[0110] Gaussian blurring is applied to the images in the image sequence to obtain low-frequency components;

[0111] Subtract the low-frequency component from the image in the image sequence to obtain the high-frequency component;

[0112] Histogram equalization is performed on the high-frequency components to enhance them.

[0113] In some preferred embodiments, the fusion in step S3 involves performing a logical AND operation on the first dirty mask obtained based on pixel brightness variance, the second dirty mask obtained based on block brightness gradient change, and the third dirty mask obtained based on frequency domain enhancement, and taking the intersection region as the dirty region information.

[0114] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0115] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0116] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the following steps:

[0117] S1. Control the image acquisition device to capture multiple images with different exposure levels, forming an image sequence from overexposed to underexposed;

[0118] S2. Process the image sequence to extract dirt features based on pixel brightness variance, dirt features based on block brightness gradient changes, and dirt features based on frequency domain enhancement, respectively.

[0119] S3. Combine the various dirt features extracted in step S2 to generate dirt area information.

[0120] In some preferred embodiments, in step S1, images with different exposures are obtained by adjusting the brightness of the fill light or adjusting the exposure parameters of the image acquisition device; before step S1, the method further includes the step of turning off the white balance and automatic image enhancement algorithm of the image acquisition device.

[0121] In some preferred embodiments, step S2, extracting dirt features based on pixel brightness variance, includes:

[0122] Calculate the luminance variance of each pixel across all images in the image sequence;

[0123] Thresholding is applied to the calculated variance values ​​to obtain the first dirt mask.

[0124] In some preferred embodiments, thresholding the calculated variance values ​​includes:

[0125] An adaptive thresholding algorithm is used for threshold segmentation, and morphological opening operations are performed on the segmentation results to remove noise.

[0126] In some preferred embodiments, step S2, extracting dirt features based on block brightness gradient changes, includes:

[0127] Each image is divided into multiple blocks;

[0128] For each block, calculate its brightness variation curve in the image sequence;

[0129] Calculate the gradient difference in brightness change between each block and its neighboring blocks;

[0130] Blocks with gradient differences exceeding a set gradient threshold are marked as suspicious dirty blocks, forming a second dirty mask.

[0131] In some preferred embodiments, step S2, extracting dirt features based on frequency domain enhancement, includes:

[0132] Each image in the image sequence is subjected to high-pass filtering to obtain the corresponding high-frequency enhanced image;

[0133] The obtained multiple high-frequency enhanced images are then superimposed.

[0134] Threshold segmentation is performed on the superimposed results to obtain the third dirt mask.

[0135] In some preferred embodiments, the high-pass filtering process for each image in the image sequence includes:

[0136] Gaussian blurring is applied to the images in the image sequence to obtain low-frequency components;

[0137] Subtract the low-frequency component from the image in the image sequence to obtain the high-frequency component;

[0138] Histogram equalization is performed on the high-frequency components to enhance them.

[0139] In some preferred embodiments, the fusion in step S3 involves performing a logical AND operation on the first dirty mask obtained based on pixel brightness variance, the second dirty mask obtained based on block brightness gradient change, and the third dirty mask obtained based on frequency domain enhancement, and taking the intersection region as the dirty region information.

[0140] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0143] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0145] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0146] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0147] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for monitoring lens dirt, characterized in that, include: S1. Control the image acquisition device to capture multiple images with different exposure levels, forming an image sequence from overexposed to underexposed; S2. Process the image sequence to extract dirt features based on pixel brightness variance, dirt features based on block brightness gradient changes, and dirt features based on frequency domain enhancement, respectively. S3. Combine the various dirt features extracted in step S2 to generate dirt area information.

2. The method for monitoring lens contamination according to claim 1, characterized in that, In step S1, images with different exposures are obtained by adjusting the brightness of the fill light or adjusting the exposure parameters of the image acquisition device; before step S1, there is also a step: turning off the white balance and automatic image enhancement algorithm of the image acquisition device.

3. The method for monitoring lens contamination according to claim 1, characterized in that, In step S2, dirt features based on pixel brightness variance are extracted, including: Calculate the luminance variance of each pixel across all images in the image sequence; Thresholding is performed on the calculated variance values ​​to obtain the first dirt mask.

4. The method for monitoring lens contamination according to claim 3, characterized in that, The threshold segmentation of the calculated variance values ​​includes: An adaptive thresholding algorithm is used for threshold segmentation, and morphological opening operations are performed on the segmentation results to remove noise.

5. The method for monitoring lens contamination according to claim 1, characterized in that, In step S2, dirt features based on changes in block brightness gradient are extracted, including: Each image is divided into multiple blocks; For each block, calculate its brightness variation curve in the image sequence; Calculate the gradient difference in brightness change between each block and its neighboring blocks; Blocks with gradient differences exceeding a set gradient threshold are marked as suspicious dirty blocks, forming a second dirty mask.

6. The method for monitoring lens contamination according to claim 1, characterized in that, In step S2, the dirt features based on frequency domain enhancement are extracted, including: Each image in the image sequence is subjected to high-pass filtering to obtain the corresponding high-frequency enhanced image; The obtained multiple high-frequency enhanced images are then superimposed. Threshold segmentation is performed on the superimposed results to obtain the third dirt mask.

7. The method for monitoring lens contamination according to claim 6, characterized in that, The high-pass filtering process for each image in the image sequence includes: Gaussian blurring is applied to the images in the image sequence to obtain low-frequency components; Subtract the low-frequency component from the image in the image sequence to obtain the high-frequency component; Histogram equalization is performed on the high-frequency components to enhance them.

8. The method for monitoring lens contamination according to claim 1, characterized in that, The fusion in step S3 involves performing a logical AND operation on the first dirty mask obtained based on pixel brightness variance, the second dirty mask obtained based on block brightness gradient change, and the third dirty mask obtained based on frequency domain enhancement, and taking the intersection region as the dirty region information.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-8.

Images