Lens shielding detection method, image acquisition equipment and storage medium

Abstract

The invention provides a lens shielding detection method, image acquisition equipment and a storage medium. The method comprises the following steps: segmenting a target image to obtain at least two block images; each block image has the same size and shape; obtaining the regional high-frequency confidence degree of each block image corresponding to the predetermined three components; obtaining a region bright color ratio corresponding to each block image; the regional bright color ratio is the ratio of pixel points with the brightness greater than the preset brightness in the block image; obtaining a region motion probability corresponding to each block image; the region motion probability is color brightness change in the block image; according to the current region high-frequency confidence coefficient, the historical region high-frequency confidence coefficient, the region bright color duty ratio and the region motion probability, the region shielding probability is obtained through calculation; and determining a lens shielding state according to the region shielding probabilities corresponding to the plurality of block images. The accuracy of detecting whether the lens is shielded or not can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a lens occlusion detection method, an image acquisition device, and a storage medium. Background Art

[0002] The application scenarios of cameras are very wide. Shopping malls, factories, hospitals and other public places need to install cameras for shooting, identification, tracking, security, etc.

[0003] However, when using a camera, it is common for the camera lens to be blocked, which may be caused by a variety of unexpected factors. Therefore, lens blockage detection is crucial. Summary of the Invention

[0004] The present application provides a lens occlusion detection method, an image acquisition device, and a storage medium to improve the accuracy of detecting whether a lens is occluded.

[0005] In a first aspect, the present application provides a lens occlusion detection method, comprising the steps of:

[0006] Segmenting the target image to obtain at least two block images; each of the block images has the same size and shape;

[0007] Obtaining high-frequency confidence of the area corresponding to each of the three predetermined components of each block image;

[0008] Obtaining a bright color ratio of a region corresponding to each of the block images; the bright color ratio of the region is a ratio of pixels in the block image whose brightness is greater than a preset brightness;

[0009] Obtaining a regional motion probability corresponding to each of the block images; the regional motion probability is a color brightness change in the block image;

[0010] Calculating the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the bright color duty cycle of the region, and the regional motion probability;

[0011] The lens occlusion state is determined according to the regional occlusion probabilities corresponding to the plurality of block images.

[0012] In some embodiments, obtaining the high-frequency confidence of the region corresponding to each of the three predetermined components of each block image includes the steps of:

[0013] Downsampling each of the block images according to a horizontal downsampling rate and a vertical downsampling rate to obtain a multi-level sub-block image; the horizontal resolution of the sub-block image of the previous level is equal to a first integer multiple of the horizontal resolution of the sub-block image of the next level, and the vertical resolution of the sub-block image of the previous level is equal to a second integer multiple of the vertical resolution of the sub-block image of the next level;

[0014] Inputting each level of the sub-block image into a high-pass filter to obtain a corresponding high-frequency sub-block image, and calculating the high-frequency absolute value mean corresponding to each of the predetermined three components according to the high-frequency sub-block image of each level; the predetermined three components are any one of the HSV three components and the YUV three components;

[0015] The high-frequency confidence of the region corresponding to each of the block images is calculated based on the high-frequency upper threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high-frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images.

[0016] In some embodiments, the step of calculating the high-frequency absolute value means corresponding to each of the predetermined three components according to the high-frequency sub-block image at each level comprises the following steps:

[0017] The position coordinates, horizontal filter radius, vertical filter radius, filter coefficient of the high-pass filter, and the pixel coordinates, image width, image height, and pixel value of the high-frequency sub-block image are substituted into the following formula to calculate the high-frequency absolute value mean:

[0018]

[0019] Wherein, (i, j) is the pixel coordinate of the high-frequency sub-block image, (x, y) is the position coordinate, r1 is the horizontal filtering radius, r2 is the vertical filtering radius, wth is the image width of the high-frequency sub-block image, hgt is the image height of the high-frequency sub-block image, p is the pixel value of the high-frequency sub-block image, flt is the filtering coefficient, and E N is the average of the high-frequency absolute values ​​of the high-frequency sub-block images of level N;

[0020] The step of calculating the high frequency confidence of the region corresponding to each of the block images based on the high frequency upper limit threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images comprises the following steps:

[0021] Substitute the following formula to calculate the high-frequency confidence of the area:

[0022]

[0023] Wherein, max_fq is the high frequency upper limit threshold, n is the number of levels of the sub-block image, K is the total number of downsampling levels, and are respectively the high frequency absolute value means corresponding to the predetermined three components of the sub-block image of the nth level.

[0024] In some embodiments, obtaining the bright color ratio of each area corresponding to each block image includes the steps of:

[0025] Obtaining three histograms corresponding to the predetermined three components according to the block image, and obtaining a fused histogram according to the three histograms; each of the three histograms corresponds to one fused histogram;

[0026] Obtaining a filtered histogram based on the fused histogram corresponding to the first region image and the fused histogram corresponding to the second region image; the first region image is any one of the at least two block images, the second region image is the other one of the at least two block images, and the second region image is adjacent to the first region image;

[0027] Obtaining a first proportion of first-category pixels in the fused histogram corresponding to the first region image, and a second proportion of the first-category pixels in the filtered histogram; the first-category pixels are pixels having a pixel value of zero;

[0028] A first product is obtained by multiplying the first mixing coefficient by the first proportion, a second product is obtained by multiplying the second mixing coefficient by the second proportion, and the first product and the second product are added to obtain the regional bright color duty cycle.

[0029] In some embodiments, the first mixing coefficient and the second mixing coefficient are both related to the local features and global features of the block image, and the size of the first mixing coefficient is positively correlated with the importance level of the local features, and the size of the second mixing coefficient is positively correlated with the importance level of all features.

[0030] In some embodiments, obtaining the regional motion probability corresponding to each of the block images includes:

[0031] obtaining a difference image corresponding to each of the three preset color components based on a previous region image and a current region image; the previous region image is any one of the at least two block images obtained by segmenting the target image at a first moment, and the current region image is any one of the at least two block images obtained by segmenting the target image at a second moment, the previous region image and the current region image having the same position coordinates in the target image, the first moment being the start time of a preset time period, and the second moment being the end time of the preset time period;

[0032] performing binarization processing on the difference images corresponding to the three predetermined components to obtain binarized images;

[0033] Obtain a binary difference image according to the binarized images corresponding to the three predetermined components; the pixel value of the binary difference image is the maximum value in the binarized images corresponding to the three predetermined components;

[0034] Obtaining a motion probability image according to the binarized difference image, filtering the motion probability image, and segmenting the filtered motion probability image to obtain at least two partition images, wherein the segmentation method of the filtered motion probability image is the same as the segmentation method of the target image;

[0035] The pixel mean of each subarea image is calculated, and the regional motion probability of the block image corresponding to the subarea image is obtained according to the pixel mean and the sum of the pixel values ​​of each subarea image.

[0036] The regional occlusion probability is calculated based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability, including:

[0037] Get the regional coefficient of the region of interest;

[0038] Substituting the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability corresponding to the block image into the following formula, the regional occlusion probability corresponding to the block image is calculated;

[0039]

[0040] in, + is the regional high-frequency confidence of the history, is the current region high frequency confidence, E H is the bright color duty cycle of the area, E Fis the motion probability of the region, a, b, c, d are the corresponding set coefficients, e is the coefficient of the region of interest, idx is the index of the block image, Occ idx The area occlusion probability corresponding to the block image corresponding to the index is index.

[0041] In some embodiments, determining the lens occlusion state according to the regional occlusion probabilities corresponding to the plurality of block images includes:

[0042] Comparing the area occlusion probability corresponding to the block image with a preset occlusion threshold;

[0043] If the area occlusion probability corresponding to the block image exceeds the preset occlusion threshold, confirming that the image occlusion state of the block image is that the block image is occluded;

[0044] If the area occlusion probability corresponding to the block image does not exceed the preset occlusion threshold, confirming the image occlusion state of the block image is that the block image is not occluded;

[0045] According to the regional occlusion states corresponding to all the block images in the target image, an expansion algorithm is used to obtain an updated occluded block image;

[0046] According to the updated occluded block image, a maximum connected area is calculated using a connected domain;

[0047] Comparing the calculated maximum connected area with a preset area threshold;

[0048] If the maximum connected area exceeds the preset area threshold, determining that the target image is blocked;

[0049] If the maximum connected area does not exceed the preset area threshold, it is determined that the target image is not blocked.

[0050] In a second aspect, the present application further provides an image acquisition device, comprising:

[0051] A segmentation module, configured to segment the target image to obtain at least two block images; each of the block images has the same size and shape;

[0052] An acquisition module is configured to acquire a high-frequency confidence level of the region corresponding to each of the three predetermined components of each block image; acquire a bright color ratio of the region corresponding to each of the block images; the bright color ratio of the region is the ratio of pixels in the block image whose brightness is greater than a preset brightness; and acquire a motion probability of the region corresponding to each of the block images; the motion probability of the region is a color brightness change in the block image.

[0053] The processing module is used to calculate the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle and the regional motion probability; and determine the lens occlusion state based on the regional occlusion probabilities corresponding to the multiple block images.

[0054] In a second aspect, the present application further provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes the steps of the lens occlusion detection method described in the first aspect.

[0055] The lens occlusion detection method, image acquisition device, and storage medium provided by the present application segment a target image to obtain at least two block images; each block image has the same size and shape; obtains the regional high-frequency confidence corresponding to each of the three predetermined components of each block image; obtains the regional bright color ratio corresponding to each block image; the regional bright color ratio is the ratio of pixels in the block image with brightness greater than a preset brightness; obtains the regional motion probability corresponding to each block image; the regional motion probability is the color brightness change in the block image; calculates the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability; and determines the lens occlusion status based on the regional occlusion probabilities corresponding to multiple block images. The present application can improve the accuracy of detecting whether the lens is occluded. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 This is a flowchart of a lens occlusion detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0059] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0060] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, well-known processes will not be elaborated in detail to avoid obscuring the description of the embodiments of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0061] Lens occlusion detection techniques can be broadly categorized into three types: the first is based on image depth, analyzing the image's depth information to detect and identify lens occlusion. The second is based on gradient calculation, using the image's histogram and gradient information to detect and identify lens occlusion. The third is based on neural networks, using trained neural network models to detect and identify lens occlusion. However, each of these methods has limitations. Image depth-based lens occlusion detection requires additional depth-sensing hardware and is unsuitable for cameras that lack such hardware. Gradient calculation-based lens occlusion detection is ineffective when faced with textured occlusions, such as leaves. While powerful, neural network-based lens occlusion detection methods suffer from high computational complexity, limiting their application in real-time systems, particularly on resource-constrained embedded devices. Furthermore, different types of occlusions, such as transparent or semi-transparent objects, moving objects, and objects with similar backgrounds, exhibit distinct characteristics, making a single detection algorithm difficult to fully adapt.

[0062] The lens occlusion detection method, image acquisition device and storage medium of the present application are described below in conjunction with the drawings to solve the above problems.

[0063] Reference Figure 1 As shown, Figure 1 This is a flow chart of the lens occlusion detection method provided in the embodiment of the present application. Figure 1The flowcharts shown in the figures or other figures show a logical order, but in some cases, the steps shown or described may be performed in a different order than shown in the figures. The lens occlusion detection method provided in the embodiment of the present application is applied to an image acquisition device, the image acquisition device includes a plurality of image acquisition related devices, the plurality of image acquisition related devices include an image signal processor, such as Figure 1 As shown, the lens occlusion detection method includes the steps of:

[0064] In a first aspect, the present application provides a lens occlusion detection method, comprising the steps of:

[0065] S100, segmenting a target image to obtain at least two block images; each of the block images has the same size and shape;

[0066] S200, obtaining high-frequency confidence of the region corresponding to each of the three predetermined components of each block image;

[0067] S300, obtaining a bright color ratio of a region corresponding to each of the block images; the bright color ratio of the region is a ratio of pixels in the block image whose brightness is greater than a preset brightness;

[0068] S400, obtaining a regional motion probability corresponding to each of the block images; the regional motion probability is a color brightness change in the block image;

[0069] S500, calculating a regional occlusion probability based on a current regional high-frequency confidence, a historical regional high-frequency confidence, a bright color duty cycle of the region, and a regional motion probability;

[0070] S600: Determine a lens occlusion state according to regional occlusion probabilities corresponding to the plurality of block images.

[0071] Specifically, the present application counts the color brightness change rate of each block image, and calculates the regional motion probability in combination with the time series information. Specifically, the regional motion probability of each block image is counted through the color brightness change of pixels between adjacent image frames. The regional motion probability reflects the dynamic characteristics of the object in the regional image, which helps to distinguish between stationary occluders and moving occluders. The present application combines multiple features such as high-frequency confidence, bright color duty cycle, motion probability, etc., and can analyze occlusion situations from different angles, thereby improving the robustness of detection. The present application combines multiple features such as high-frequency confidence, bright color duty cycle, motion probability, etc., does not rely on depth information, avoids complex depth calculations, is suitable for application in resource-constrained environments, can cope with occluders of different textures, colors and transparencies, and has strong versatility.

[0072] In some embodiments, obtaining the high-frequency confidence of the region corresponding to each of the three predetermined components of each block image includes the steps of:

[0073] Downsampling each of the block images according to a horizontal downsampling rate and a vertical downsampling rate to obtain a multi-level sub-block image; the horizontal resolution of the sub-block image of the previous level is equal to a first integer multiple of the horizontal resolution of the sub-block image of the next level, and the vertical resolution of the sub-block image of the previous level is equal to a second integer multiple of the vertical resolution of the sub-block image of the next level;

[0074] Inputting each level of the sub-block image into a high-pass filter to obtain a corresponding high-frequency sub-block image, and calculating the high-frequency absolute value mean corresponding to each of the predetermined three components according to the high-frequency sub-block image of each level; the predetermined three components are any one of the HSV three components and the YUV three components;

[0075] The high-frequency confidence of the region corresponding to each of the block images is calculated based on the high-frequency upper threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high-frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images.

[0076] Specifically, the target image is divided into multiple sub-areas to obtain block images, specifically, the horizontal division number is set to N, the vertical division number is set to M, and the entire target image is evenly divided into MxN block images by equal division. Then, each block image is multi-level downsampled according to the downsampling rate in the horizontal and vertical directions. The resolution of each level of sub-block image after downsampling satisfies the following relationship: the resolution of the previous level sub-block image in the horizontal direction is the first integer multiple of the horizontal resolution of the next level sub-block image. The resolution of the previous level sub-block image in the vertical direction is the second integer multiple of the vertical resolution of the next level sub-block image. This downsampling method can gradually reduce the resolution of the image while retaining key information. Pyramid structure sampling is implemented on the block image to construct a multi-level pyramid image. The sampling rate can be different in the horizontal and vertical directions, and the sampling rate of each layer of the pyramid can also be different. For example,

[0077] The first-level pyramid has a horizontal downsampling rate of 4 and a vertical downsampling rate of 2;

[0078] The second-level pyramid has a horizontal downsampling rate of 2 and a vertical downsampling rate of 2.

[0079] If the resolution of the original block image is AxB, the resolution of the second-level pyramid will be A / 4x, B / 2, and the resolution of the third-level pyramid will be A / 8x, B / 4. The sub-block images after each downsampling level are then input into a high-pass filter to extract high-frequency information. A high-pass filter removes low-frequency components (such as smooth areas) from an image while retaining high-frequency components (such as edges and details). Common high-pass filters include Sobel filters. After processing through the high-pass filter, high-frequency sub-block images corresponding to each level of sub-block images are obtained. For each component of each high-frequency sub-block image (such as H, S, V in HSV or Y, U, V in YUV), the high-frequency absolute value mean corresponding to each of the three components is calculated. The high-frequency confidence of the region corresponding to each sub-block image is then calculated based on the high-frequency upper threshold, the number of sub-block image levels, the total number of downsampling levels, and the high-frequency absolute value mean corresponding to each of the three components of each sub-block image level. This application can perform multi-level downsampling on each block image, extract high-frequency information, and calculate the regional high-frequency confidence, which can effectively reflect the distribution of high-frequency information in the image and is often used for feature extraction in image analysis and target detection.

[0080] In some embodiments, the step of calculating the high-frequency absolute value means corresponding to each of the predetermined three components according to the high-frequency sub-block image at each level comprises the following steps:

[0081] The position coordinates, horizontal filter radius, vertical filter radius, filter coefficient of the high-pass filter, and the pixel coordinates, image width, image height, and pixel value of the high-frequency sub-block image are substituted into the following formula to calculate the high-frequency absolute value mean:

[0082]

[0083] Wherein, (i, j) is the pixel coordinate of the high-frequency sub-block image, (x, y) is the position coordinate, r1 is the horizontal filtering radius, r2 is the vertical filtering radius, wth is the image width of the high-frequency sub-block image, hgt is the image height of the high-frequency sub-block image, p is the pixel value of the high-frequency sub-block image, flt is the filtering coefficient, and E N is the average of the high-frequency absolute values ​​of the high-frequency sub-block images of level N;

[0084] For block images, distinguish between brightness and color and calculate the high-frequency absolute average of the three components, such as the three-component YUV and three-component HSV. The horizontal and vertical filter radii and filter coefficients of the high-pass filters used for the three components of brightness and color can be different.

[0085] The step of calculating the high frequency confidence of the region corresponding to each of the block images based on the high frequency upper limit threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images comprises the following steps:

[0086] Substitute the following formula to calculate the high-frequency confidence of the area:

[0087]

[0088] Wherein, max_fq is the high frequency upper limit threshold, n is the number of levels of the sub-block image, K is the total number of downsampling levels, and are respectively the high frequency absolute value means corresponding to the predetermined three components of the sub-block image of the nth level.

[0089] For the high-frequency absolute average of the three components obtained at each level of the pyramid image, first calculate the maximum high-frequency absolute average of the three components of the pyramid at different levels, then set the high-frequency upper limit threshold to calculate the ratio, and finally use the above formula to calculate the regional high-frequency confidence E F .

[0090] In some embodiments, obtaining the bright color ratio of each area corresponding to each block image includes the steps of:

[0091] Obtaining three histograms corresponding to the predetermined three components according to the block image, and obtaining a fused histogram according to the three histograms; each of the three histograms corresponds to one fused histogram;

[0092] Obtaining a filtered histogram based on the fused histogram corresponding to the first region image and the fused histogram corresponding to the second region image; the first region image is any one of the at least two block images, the second region image is the other one of the at least two block images, and the second region image is adjacent to the first region image;

[0093] Obtaining a first proportion of first-category pixels in the fused histogram corresponding to the first region image, and a second proportion of the first-category pixels in the filtered histogram; the first-category pixels are pixels having a pixel value of zero;

[0094] A first product is obtained by multiplying the first mixing coefficient by the first proportion, a second product is obtained by multiplying the second mixing coefficient by the second proportion, and the first product and the second product are added to obtain the regional bright color duty cycle.

[0095] Specifically, for the three components of brightness and color (such as Y, U, V or H, S, V in YUV or HSV color space), a preset step size is used to count the histogram corresponding to the block image. The histograms of the three components are fused to form a fused histogram. During the fusion process, the histograms of the three components can be combined by weighted averaging or other mathematical methods to obtain a single histogram containing all color and brightness information. Considering the similarity between regions, the fused histogram of the spatially adjacent regions of the current block image is obtained. The fused histogram of the current block image is fused with the fused histogram of the spatially adjacent block images to obtain a filtered histogram. The filtering method can be implemented by taking weighted averaging or median filtering, etc., in order to improve the stability and reliability of the histogram. The duty cycle of the fused histogram and the filtered histogram of the current block image are calculated respectively. The duty cycle refers to the proportion of non-zero values ​​in the histogram, as shown in the following formula, where hist i is the value of the i-th point in the histogram, T is the maximum statistical number of the histogram, and hist_rto is the duty cycle of the bright color in the area.

[0096]

[0097] The preset mixing coefficient is used to mix the duty ratios of the fused histogram and the filtered histogram of the current block image to obtain the regional bright color duty ratio.

[0098] In some embodiments, the first mixing coefficient and the second mixing coefficient are both related to the local features and the global features of the block image, and the magnitude of the first mixing coefficient is positively correlated with the importance level of the local features, and the magnitude of the second mixing coefficient is positively correlated with the importance level of all features. The mixing coefficient can be adjusted according to the needs of the actual application scenario to balance the importance of local features and global features. For example, if local features are more important, a larger first mixing coefficient can be assigned to the duty cycle of the fusion histogram; conversely, if global features are more important, the weight of the filter histogram duty cycle can be increased, and a larger second mixing coefficient can be assigned to the filter histogram duty cycle.

[0099] In some embodiments, obtaining the regional motion probability corresponding to each of the block images includes:

[0100] obtaining a difference image corresponding to each of the three preset color components based on a previous region image and a current region image; the previous region image is any one of the at least two block images obtained by segmenting the target image at a first moment, and the current region image is any one of the at least two block images obtained by segmenting the target image at a second moment, the previous region image and the current region image having the same position coordinates in the target image, the first moment being the start time of a preset time period, and the second moment being the end time of the preset time period;

[0101] performing binarization processing on the difference images corresponding to the three predetermined components to obtain binarized images;

[0102] Obtain a binary difference image according to the binarized images corresponding to the three predetermined components; the pixel value of the binary difference image is the maximum value in the binarized images corresponding to the three predetermined components;

[0103] Obtaining a motion probability image according to the binarized difference image, filtering the motion probability image, and segmenting the filtered motion probability image to obtain at least two partition images, wherein the segmentation method of the filtered motion probability image is the same as the segmentation method of the target image;

[0104] The pixel mean of each subarea image is calculated, and the regional motion probability of the block image corresponding to the subarea image is obtained according to the pixel mean and the sum of the pixel values ​​of each subarea image.

[0105] Specifically, in order to reveal the changes that occur in the area within a preset time period, a three-component difference image is obtained by taking the difference between the first and last two frames of the preset time period. Thresholds are set for the three components respectively, and the three-component difference image is binarized according to the threshold to obtain a three-component binary image. For example, for each color component (such as RGB) in the difference image, a threshold is set, and the pixel value above the threshold is set to 1 (or 255), and the pixel value below the threshold is set to 0 (or 0), thereby obtaining a binary image. The three-component binary image is subjected to the pixel-level maximum value operation to obtain a binary difference image, which is specifically as follows, wherein i and j are image coordinates, and are the pixel values ​​of the binarized images corresponding to the three predetermined components, is the pixel value of the binarized difference image.

[0106]

[0107] Accumulate multiple binary difference images to obtain a motion probability image. The number of accumulated motion probability images is a preset value. The calculation formula for each pixel in the specific motion probability image is as follows, where max_val is the maximum pixel value of the set motion probability image (for example, for 8-bit, it can be set to 255), W is the number of accumulated motion probability images, i and j are the binary image coordinates, and n is the index of the binary difference image. is the pixel value of the binary difference image corresponding to index n, is the pixel value of the motion probability image; the motion probability image is subsequently continuously updated according to the binary difference image;

[0108]

[0109] The motion probability image is filtered to reduce the effects of noise and outliers. For example, this can be achieved by using filters such as Gaussian filtering or median filtering to smooth the image and enhance the characteristics of motion areas. The filtered motion probability image is segmented based on the region divisions to obtain motion probability block images corresponding to the original block image. To reflect the intensity and frequency of motion within the region, the mean of each motion probability block image is calculated to obtain the motion probability for that region.

[0110] In some embodiments, the calculating of the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability includes:

[0111] Get the regional coefficient of the region of interest;

[0112] Substituting the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability corresponding to the block image into the following formula, the regional occlusion probability corresponding to the block image is calculated;

[0113]

[0114] in, + is the regional high-frequency confidence of the history, is the current region high frequency confidence, E H is the bright color duty cycle of the area, E F is the motion probability of the region, a, b, c, d are the corresponding set coefficients, e is the coefficient of the region of interest, idx is the index of the block image, Occ idx The area occlusion probability corresponding to the block image corresponding to the index is index.

[0115] In some embodiments, determining the lens occlusion state according to the regional occlusion probabilities corresponding to the plurality of block images includes:

[0116] Comparing the area occlusion probability corresponding to the block image with a preset occlusion threshold;

[0117] If the area occlusion probability corresponding to the block image exceeds the preset occlusion threshold, confirming that the image occlusion state of the block image is that the block image is occluded;

[0118] If the area occlusion probability corresponding to the block image does not exceed the preset occlusion threshold, confirming the image occlusion state of the block image is that the block image is not occluded;

[0119] According to the regional occlusion states corresponding to all the block images in the target image, an expansion algorithm is used to obtain an updated occluded block image;

[0120] According to the updated occluded block image, a maximum connected area is calculated using a connected domain;

[0121] Comparing the calculated maximum connected area with a preset area threshold;

[0122] If the maximum connected area exceeds the preset area threshold, determining that the target image is blocked;

[0123] If the maximum connected area does not exceed the preset area threshold, it is determined that the target image is not blocked.

[0124] Specifically, the dilation algorithm is a morphological image processing method used to expand the white area (or occluded area) in the image. Its function is to connect adjacent occluded blocks to form a larger occluded area. A connected domain refers to a set of interconnected pixels in an image.

[0125] First, for each image block, the corresponding regional occlusion probability is calculated. This probability reflects the likelihood that the image block is occluded. A preset occlusion threshold (e.g., 0.5 or 0.6) is used to determine the occlusion status. The regional occlusion probability is compared with the preset threshold. If the regional occlusion probability of a particular image block exceeds the preset threshold, the block is considered occluded. If the regional occlusion probability of a particular image block does not exceed the preset threshold, the block is considered unoccluded. This block-by-block determination method can preliminarily determine the occlusion status of each image block. Next, the occlusion status of the entire target image is further analyzed. The specific steps are as follows: A dilation algorithm is used to dilate the blocks initially determined to be occluded, making the occluded area more complete and avoiding fragmentation caused by low local occlusion probabilities. The mathematical expression of the dilation algorithm can be simply understood as expanding the boundaries of the occluded area outward by a certain range, usually by defining a structuring element (e.g., a 3×3 rectangle). After dilation, the connectivity of the occluded areas needs to be further analyzed. In occlusion detection, connected domain analysis is used to determine the size and distribution of occluded areas. Connected domain analysis locates the largest connected domain in the dilated occluded block image and calculates its area to obtain the maximum connected area. The maximum connected area is compared with a preset area threshold to determine the occlusion status of the entire target image. An area threshold is preset to determine whether the entire image is occluded. The maximum connected area is compared with the preset area threshold. If the maximum connected area exceeds the preset area threshold, the entire target image is considered occluded. If the maximum connected area does not exceed the threshold, the target image is considered unoccluded.

[0126] This application considers both local occlusion and global occlusion status through block-by-block judgment and overall connected domain analysis. This application uses an expansion algorithm to effectively connect occluded areas and avoid misjudgments caused by local noise or insufficient occlusion probability. The preset occlusion threshold and preset area threshold of this application can be adjusted according to the actual application scenario to improve the adaptability and robustness of the algorithm. This application is very practical in the fields of video surveillance, autonomous driving, robot vision, etc., and can effectively detect whether the lens is blocked, thereby triggering an alarm or taking corresponding measures in time.

[0127] This application can keenly capture the sudden change of high-frequency information caused by transparent object occlusion by calculating the ratio of the high-frequency confidence of the current area to the high-frequency confidence of the historical area. Although transparent objects will not completely block the image content, the texture and edge information of their surface will form different high-frequency features from the background. The change of this feature will be amplified in the ratio calculation, making transparent object occlusion easier to detect. This method has enhanced adaptability and does not rely on specific image content or background information. Instead, it judges the occlusion situation by analyzing the changing trend of high-frequency features. Therefore, it has good adaptability to transparent object occlusion in different scenarios.

[0128] This application combines the current regional high-frequency confidence and the regional bright color duty cycle, and based on the principle of color brightness diversity, can effectively identify close-fitting occlusions, such as stain occlusions and covering occlusions. Stains and coverings usually form local changes in color and brightness on the image. These changes will affect the bright color duty cycle of the area, and the high-frequency confidence reflects the detailed characteristics of these changes. By comprehensively analyzing these two indicators, the existence of close-fitting occlusions can be accurately judged. This method is highly robust and can maintain high robustness when faced with complex image environments and various occlusion types. Even in the presence of interference factors such as lighting changes and image noise, close-fitting occlusions can still be accurately identified through a comprehensive analysis of color brightness diversity and high-frequency features.

[0129] This application calculates regional motion probability. Beyond addressing general, close-fitting occlusion, it further optimizes the problem of close-range texture occlusion. For ultra-close-range texture occlusion, such as leaf occlusion, the texture features may be very close to those of the background, making it difficult to distinguish using color and brightness alone. Regional motion probability, however, can capture how texture regions change over time, such as the subtle swaying of leaves, providing a powerful basis for detecting ultra-close-range texture occlusion.

[0130] In summary, for occlusion detection methods that rely on depth information, this application eliminates the reliance on depth devices and does not need to collect depth information, thereby significantly reducing costs and making lens occlusion detection more economical and practical. Secondly, for detection methods based on gradient calculation, this application not only introduces frequency information and diversity information of brightness and color, but also integrates the calculation of regional motion probability. This multi-dimensional analysis method greatly expands the detection range and improves detection accuracy. It can effectively identify close-fitting occlusions, and can also accurately detect transparent object occlusions and close-range occlusions, overcoming the limitations of traditional gradient calculation methods in these scenarios. Thirdly, compared with detection methods based on neural networks, this application does not need to consume a large amount of computing power resources, but can still achieve better detection results. This makes this application more valuable and feasible in resource-constrained environments, such as embedded devices or real-time processing systems.

[0131] The present application also provides an image acquisition device, comprising:

[0132] A segmentation module, configured to segment the target image to obtain at least two block images; each of the block images has the same size and shape;

[0133] An acquisition module is configured to acquire a high-frequency confidence level of the region corresponding to each of the three predetermined components of each block image; acquire a bright color ratio of the region corresponding to each of the block images; the bright color ratio of the region is the ratio of pixels in the block image whose brightness is greater than a preset brightness; and acquire a motion probability of the region corresponding to each of the block images; the motion probability of the region is a color brightness change in the block image.

[0134] The processing module is used to calculate the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle and the regional motion probability; and determine the lens occlusion state based on the regional occlusion probabilities corresponding to the multiple block images.

[0135] The present application also provides a storage medium having a computer program stored thereon, wherein the computer program is configured to execute the steps of the lens occlusion detection method described in the first aspect when the computer program is executed by a processor.

[0136] In specific implementation, the above units or modules can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same entity or several entities. The above units or modules can refer to the lens occlusion detection method in the previous method embodiment, and will not be repeated here.

[0137] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor to implement the above method embodiments. The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0138] In the above embodiments, the descriptions of each embodiment have their own specific focus. For portions not detailed in one embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for ease and brevity of description, the specific operating processes and beneficial effects of the data acquisition system and its corresponding units described above can be referenced to the description of the lens occlusion detection method in the above embodiments, and no further details will be given here.

[0139] The above is a detailed introduction to a lens occlusion detection method and system provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A lens occlusion detection method, characterized in that: Including steps: Segmenting the target image to obtain at least two block images; each of the block images has the same size and shape; Obtaining high-frequency confidence of the area corresponding to each of the three predetermined components of each block image; Obtaining the bright color ratio of each corresponding area of ​​each block image; The bright color ratio of the region is the ratio of pixels in the block image whose brightness is greater than a preset brightness; Obtaining a regional motion probability corresponding to each of the block images; the regional motion probability is a color brightness change in the block image; Calculating the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the bright color duty cycle of the region, and the regional motion probability; The lens occlusion state is determined according to the regional occlusion probabilities corresponding to the plurality of block images.

2. The lens occlusion detection method according to claim 1, wherein: The step of obtaining the high-frequency confidence of the region corresponding to each of the three predetermined components of each block image comprises the following steps: Downsampling each of the block images according to a horizontal downsampling rate and a vertical downsampling rate to obtain a multi-level sub-block image; the horizontal resolution of the sub-block image of the previous level is equal to a first integer multiple of the horizontal resolution of the sub-block image of the next level, and the vertical resolution of the sub-block image of the previous level is equal to a second integer multiple of the vertical resolution of the sub-block image of the next level; Inputting the sub-block images of each level into a high-pass filter to obtain corresponding high-frequency sub-block images, and calculating the high-frequency absolute value means corresponding to each of the predetermined three components according to the high-frequency sub-block images of each level; The predetermined three components are any one of the HSV three components and the YUV three components; The high-frequency confidence of the region corresponding to each of the block images is calculated based on the high-frequency upper threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high-frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images.

3. The lens occlusion detection method according to claim 2, wherein: The step of calculating the high frequency absolute value means corresponding to each of the predetermined three components according to the high frequency sub-block image at each level comprises the following steps: The position coordinates, horizontal filter radius, vertical filter radius, filter coefficient of the high-pass filter, and the pixel coordinates, image width, image height, and pixel value of the high-frequency sub-block image are substituted into the following formula to calculate the high-frequency absolute value mean: Wherein, (i, j) is the pixel coordinate of the high-frequency sub-block image, (x, y) is the position coordinate, r1 is the horizontal filtering radius, r2 is the vertical filtering radius, wth is the image width of the high-frequency sub-block image, hgt is the image height of the high-frequency sub-block image, p is the pixel value of the high-frequency sub-block image, flt is the filtering coefficient, and E N is the average of the high-frequency absolute values ​​of the high-frequency sub-block images of level N; The step of calculating the high frequency confidence of the region corresponding to each of the block images based on the high frequency upper limit threshold, the number of levels of the sub-block images, the total number of downsampling levels, and the average of the high frequency absolute values ​​corresponding to the predetermined three components of each level of the sub-block images comprises the following steps: Substitute the following formula to calculate the high-frequency confidence of the area: Wherein, max_fq is the high frequency upper limit threshold, n is the number of levels of the sub-block image, K is the total number of downsampling levels, and are respectively the high frequency absolute value means corresponding to the predetermined three components of the sub-block image of the nth level.

4. The lens occlusion detection method according to claim 1, wherein: The step of obtaining the bright color ratio of each corresponding area of ​​each block image comprises the following steps: Obtaining three histograms corresponding to the predetermined three components according to the block image, and obtaining a fused histogram according to the three histograms; each of the three histograms corresponds to one fused histogram; Obtaining a filtered histogram according to the fused histogram corresponding to the first region image and the fused histogram corresponding to the second region image; The first region image is any one of the at least two block images, the second region image is the other one of the at least two block images, and the second region image is adjacent to the first region image; Obtaining a first proportion of first-category pixels in the fused histogram corresponding to the first region image, and a second proportion of the first-category pixels in the filtered histogram; the first-category pixels are pixels having a pixel value of zero; A first product is obtained by multiplying the first mixing coefficient by the first proportion, a second product is obtained by multiplying the second mixing coefficient by the second proportion, and the first product and the second product are added to obtain the regional bright color duty cycle.

5. The lens occlusion detection method according to claim 4, wherein: The first mixing coefficient and the second mixing coefficient are both related to the local features and global features of the block image, and the size of the first mixing coefficient is positively correlated with the importance level of the local features, and the size of the second mixing coefficient is positively correlated with the importance level of all features.

6. The lens occlusion detection method according to claim 1, wherein: The obtaining of the regional motion probability corresponding to each of the block images includes: obtaining a difference image corresponding to each of the three preset color components based on a previous region image and a current region image; the previous region image is any one of the at least two block images obtained by segmenting the target image at a first moment, and the current region image is any one of the at least two block images obtained by segmenting the target image at a second moment, the previous region image and the current region image having the same position coordinates in the target image, the first moment being the start time of a preset time period, and the second moment being the end time of the preset time period; performing binarization processing on the difference images corresponding to the three predetermined components to obtain binarized images; Obtain a binary difference image according to the binarized images corresponding to the three predetermined components; the pixel value of the binary difference image is the maximum value in the binarized images corresponding to the three predetermined components; Obtaining a motion probability image according to the binarized difference image, filtering the motion probability image, and segmenting the filtered motion probability image to obtain at least two partition images, wherein the segmentation method of the filtered motion probability image is the same as the segmentation method of the target image; The pixel mean of each subarea image is calculated, and the regional motion probability of the block image corresponding to the subarea image is obtained according to the pixel mean and the sum of the pixel values ​​of each subarea image.

7. The lens occlusion detection method according to claim 1, wherein: The regional occlusion probability is calculated based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability, including: Get the regional coefficient of the region of interest; Substituting the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle, and the regional motion probability corresponding to the block image into the following formula, the regional occlusion probability corresponding to the block image is calculated; in, + is the regional high-frequency confidence of the history, is the current region high frequency confidence, E H is the bright color duty cycle of the area, E F is the motion probability of the region, a, b, c, d are the corresponding set coefficients, e is the coefficient of the region of interest, idx is the index of the block image, Occ idx The area occlusion probability corresponding to the block image corresponding to the index is index.

8. The lens occlusion detection method according to claim 1, wherein: Determining the lens occlusion state according to the regional occlusion probabilities corresponding to the plurality of block images includes: Comparing the area occlusion probability corresponding to the block image with a preset occlusion threshold; If the area occlusion probability corresponding to the block image exceeds the preset occlusion threshold, confirming that the image occlusion state of the block image is that the block image is occluded; If the area occlusion probability corresponding to the block image does not exceed the preset occlusion threshold, confirming the image occlusion state of the block image is that the block image is not occluded; According to the regional occlusion states corresponding to all the block images in the target image, an expansion algorithm is used to obtain an updated occluded block image; According to the updated occluded block image, a maximum connected area is calculated using a connected domain; Comparing the calculated maximum connected area with a preset area threshold; If the maximum connected area exceeds the preset area threshold, determining that the target image is blocked; If the maximum connected area does not exceed the preset area threshold, it is determined that the target image is not blocked.

9. An image acquisition device, characterized in that: include: A segmentation module, configured to segment the target image into at least two block images; Each of the block images has the same size and shape; An acquisition module is configured to acquire a high-frequency confidence level of the region corresponding to each of the three predetermined components of each block image; and acquire a bright color ratio of the region corresponding to each of the block images; the bright color ratio of the region is the ratio of pixels in the block image having a brightness greater than a preset brightness; Obtaining a regional motion probability corresponding to each of the block images; the regional motion probability is a color brightness change in the block image; The processing module is used to calculate the regional occlusion probability based on the current regional high-frequency confidence, the historical regional high-frequency confidence, the regional bright color duty cycle and the regional motion probability; and determine the lens occlusion state based on the regional occlusion probabilities corresponding to the multiple block images.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lens occlusion detection method according to any one of claims 1 to 8 are executed.

Images