Method, device and equipment for detecting lens with target object and readable storage medium

Abstract

The application discloses a method and device for detecting whether a lens is attached with a target object, equipment and a readable storage medium, and aims to solve the problem that the target object attached on the lens cannot be found in time, resulting in low imaging quality and inaccurate detection. The method comprises the following steps: detecting an original image sequence collected by a lens to be processed, and responding to at least two suspected lens blocking images detected in the original image sequence; determining whether the lens to be processed is attached with a target object based on a suspected blocking area contained in the at least two suspected lens blocking images.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to methods, apparatus, devices, and readable storage media for detecting objects attached to lenses. Background Technology

[0002] Currently, the widespread adoption of smart video parking racks simplifies payment procedures for car owners, saves time, and reduces labor costs. However, in severe weather or due to vandalism, the detection lenses are prone to localized dirt accumulation, leading to inaccurate vehicle detection information.

[0003] Therefore, in the existing technology, there is a problem that the lens detection device for smart video parking posts and other devices cannot detect dirt or other obstructions on the lens in a timely manner, resulting in inaccurate detection. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for detecting objects attached to a lens, in order to solve the problem that objects attached to a lens cannot be detected in a timely manner, resulting in low image quality and inaccurate detection.

[0005] In a first aspect, this application provides a method for detecting a target object attached to a lens, the method comprising:

[0006] The original image sequence captured by the lens to be processed is detected, and at least two suspected lens occlusion images are detected in the original image sequence.

[0007] Based on the suspected occlusion areas contained in the at least two suspected lens occlusion images, it is determined whether there is a target object attached to the lens to be processed.

[0008] In the solution provided in the above-described embodiments of this application, suspected lens occlusion images can be determined by detecting the original image sequence acquired by the lens to be processed, and then, based on the suspected occlusion area of ​​the suspected lens occlusion image, it can be determined that there is a target object attached to the lens to be processed, thereby achieving the purpose of timely detection of a target object attached to the lens.

[0009] One possible implementation, prior to detecting at least two suspected lens occlusion images in the original image sequence, includes:

[0010] Determine the blur coefficient of each original image in the original image sequence; wherein the blur coefficient indicates whether the original image is distorted;

[0011] Original images with a blur coefficient less than the blur threshold are identified as blurred images;

[0012] Then, in response to the detection of at least two suspected lens occlusion images in the original image sequence, including:

[0013] In response to the detection of at least two suspected lens occlusion images in the blurred image.

[0014] Before detecting at least two suspected lens-occluded images, the at least two original images are determined to be at least two blurred images based on the blur coefficient. Subsequent target detection based on the blurred images can significantly improve detection efficiency.

[0015] One possible implementation, wherein determining whether a target object is attached to the lens to be processed based on the suspected occlusion area contained in the at least two suspected lens occlusion images, includes:

[0016] Determine the similarity value of the suspected occlusion regions contained in the at least two suspected lens occlusion images; wherein the similarity value indicates the degree of similarity between the suspected occlusion regions contained in the different suspected lens occlusion images;

[0017] If the similarity value is greater than the fourth threshold, it is determined that the lens to be processed has a target object attached; otherwise, the lens to be processed does not have a target object.

[0018] One possible implementation is that the suspected occlusion area is the largest connected component in the corresponding suspected lens occlusion image;

[0019] The response to detecting at least two suspected lens occlusion images in the blurred image includes:

[0020] When the maximum number of connected components in the blurred image exceeds a third threshold, the original image is determined to be a suspected occluded image; the maximum number of connected components is the area of ​​the maximum connected region.

[0021] One possible implementation, wherein determining the original image as a suspected occluded image when the maximum number of connected components in the blurred image exceeds a third threshold, includes:

[0022] The blurred image is divided into multiple image blocks containing an integer number of pixels;

[0023] Based on the average pixel value of each image block, the output pixel value of each image block is determined; wherein the output pixel value indicates the overall brightness of all pixels in the image block;

[0024] The foreground and background regions of the blurred image are determined based on the output pixel values ​​of each image block; wherein the output pixel values ​​of the image blocks included in the background region are greater than a second threshold; and the output pixel values ​​of the image blocks included in the foreground region are not greater than the second threshold.

[0025] Determine the maximum connected component and the maximum number of connected images in the foreground region of the blurred image; wherein the maximum connected component refers to the largest region composed of pixels with the same pixel value and adjacent positions; the maximum number of connected images is the area of ​​the maximum connected component;

[0026] When the number of the largest connected images in the blurred image is greater than the third threshold, the blurred image is determined to be a suspected lens occlusion image; otherwise, the blurred image is determined to be a non-suspected lens occlusion image.

[0027] One possible implementation involves detecting whether there are at least two suspected lens occlusion images among the at least two original images, including:

[0028] The first original image is identified as a suspected lens occlusion image, and the first original image is cached.

[0029] Based on the cached first original image, determine whether the second original image is a suspected lens occlusion image; if so, cache the second original image; if not, delete the first original image and the second original image; wherein, the second original image is the original image that follows the first original image and is adjacent to the first original image.

[0030] Secondly, this application provides an apparatus for detecting a target object attached to a lens, the apparatus comprising:

[0031] Detection unit: used to detect the original image sequence acquired by the lens to be processed, and responds to the detection of at least two suspected lens occlusion images in the original image sequence;

[0032] Determination unit: used to determine whether there is a target object attached to the lens to be processed based on the suspected occlusion area contained in the at least two suspected lens occlusion images.

[0033] In one possible implementation, the apparatus further includes a computing unit specifically configured to determine the blur coefficient of each original image in the original image sequence; wherein the blur coefficient indicates whether the original image is distorted; and original images with blur coefficients less than a blur threshold are determined as blurred images.

[0034] The detection unit is specifically used in response to detecting at least two suspected lens-occluded images in the blurred image.

[0035] In one possible implementation, the determining unit is specifically used to determine the similarity value of the suspected occlusion regions contained in the at least two suspected lens occlusion images; wherein the similarity value indicates the degree of similarity of the suspected occlusion regions contained in different suspected lens occlusion images; when the similarity value is greater than a fourth threshold, it is determined that the lens to be processed has a target object attached; otherwise, the lens to be processed has no target object.

[0036] One possible implementation is that the suspected occlusion area is the largest connected component in the corresponding suspected lens occlusion image;

[0037] The detection unit is further configured to determine the original image as a suspected occlusion image when the maximum number of connected components in the blurred image exceeds a third threshold; the maximum number of connected components is the area of ​​the maximum connected region. In one possible implementation, the detection unit is further configured to divide the blurred image into multiple image blocks containing an integer number of pixels; determine the output pixel value of each image block based on the average pixel value of each image block; wherein the output pixel value indicates the overall brightness of all pixels in the image block; determine the foreground and background regions of the blurred image based on the output pixel values ​​of each image block; wherein the output pixel values ​​of the image blocks included in the background region are greater than a second threshold; the output pixel values ​​of the image blocks included in the foreground region are not greater than the second threshold; determine the maximum connected region and the maximum number of connected components in the foreground region of the blurred image; wherein the maximum connected region refers to the largest region composed of pixels with the same pixel value and adjacent positions; when the maximum number of connected components in the blurred image is greater than the third threshold, the blurred image is determined to be a suspected lens occlusion image; otherwise, the blurred image is determined to be a non-suspected lens occlusion image.

[0038] In one possible implementation, the detection unit is further configured to determine that the first original image is a suspected lens occlusion image and cache the first original image; based on the cached first original image, determine whether the second original image is a suspected lens occlusion image; if so, cache the second original image; if not, delete the first original image and the second original image; wherein the second original image is the original image that follows the first original image and is adjacent to the first original image.

[0039] Thirdly, this application provides a device for detecting a target object attached to a lens, comprising:

[0040] The memory stores computer instructions;

[0041] The processor is configured to read the computer instructions and execute the following steps:

[0042] Detect the original image sequence captured by the lens to be processed;

[0043] In response to the detection of at least two suspected lens occlusion images in the original image sequence;

[0044] Based on the suspected occlusion areas contained in the at least two suspected lens occlusion images, it is determined whether there is a target object attached to the lens to be processed.

[0045] Fourthly, this application provides a readable storage medium, comprising:

[0046] memory,

[0047] The memory is used to store instructions that, when executed by a processor, cause the apparatus including the readable storage medium to perform the method as described in the first aspect and any of the implementations. Attached Figure Description

[0048] Figure 1 A flowchart of a method for detecting a target object attached to a lens, provided in this application;

[0049] Figure 2 This is a schematic diagram illustrating the detection of whether a lens has a stain (target object) based on the original image captured by the lens, as provided in this application.

[0050] Figure 3 This application provides a schematic diagram of a device for detecting a target object attached to a lens. Detailed Implementation

[0051] To address the problem in existing technologies where objects attached to surveillance lenses cannot be detected in a timely manner, resulting in low image quality and inaccurate detection, this application provides a method for detecting objects attached to lenses: by determining that the original image captured by the lens is a suspected occlusion image, and then by determining the similarity value of the occlusion areas of at least two suspected occlusion images, the presence of an object attached to the lens can be identified.

[0052] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0053] Please refer to Figure 1 This application provides a method for detecting objects attached to a lens, avoiding situations where objects attached to the lens cannot be detected in time, resulting in low image quality and inaccurate detection.

[0054] Step 101: Detect the original image sequence acquired by the lens to be processed, in response to the detection of at least two suspected lens occlusion images in the original image sequence.

[0055] Before detection, it is necessary to extract the original image sequence. In this embodiment, the original image sequence extracted by the lens to be processed can be composed of image frames from the monitoring video captured by the lens to be processed. The monitoring video can be acquired in real time or extracted from the stored video at set time intervals.

[0056] In addition, the original image sequence extracted in this step can also be composed of photos taken by the lens at set time intervals.

[0057] When a target object is attached to the lens, the image can be determined to be blurry. Image blurring is a form of image distortion caused by low image quality.

[0058] Therefore, before judging a suspected lens-occluded image, it is possible to first determine whether the image is blurred by checking if the blur coefficient is less than the corresponding blur threshold. The blur coefficient indicates whether the original image is distorted.

[0059] Specifically, the blurry image and the blur coefficient can be determined through the following three principles:

[0060] I. Based on pixel technology, analyze the statistical characteristics of pixel grayscale values ​​and the correlation between pixels, calculate the statistical characteristics of pixel grayscale values ​​and the correlation between pixels as blur coefficients, and determine image blur.

[0061] Second, based on transform domain technology, the principle that the more high-frequency components a transform domain component has, the clearer the image is, and the fewer high-frequency components a transform domain component has, the blurrier the image is. The proportion of high-frequency components in the image is calculated as the blur coefficient to determine the image blur.

[0062] Third, based on image gradient technology, the larger the gradient, the clearer the image. The gradient of the image edge is calculated as the blur coefficient to determine the image blur.

[0063] This application uses the first method described above to specifically describe the determination of blurred images and blur coefficients. First, the original image is converted to grayscale to obtain a grayscale image. Here, a grayscale image refers to an image in which the intensity of color of each pixel in the first original image can be represented by a pixel value (grayscale).

[0064] Next, for the grayscale image corresponding to the original image, the grayscale variance algorithm is used to calculate the grayscale variance value of the grayscale image as a blurring coefficient. Specifically, using the average grayscale value of all pixels in the grayscale image as a reference, the difference of the grayscale value of each pixel is calculated, then the sum of squares is calculated, and finally, the result is standardized using the total number of pixels to obtain the grayscale variance value of the grayscale image. Because variance represents the discreteness of a random variable, this sum of squares represents the average degree of change in the grayscale values ​​of all pixels in the image. The smaller the variance, the smaller the degree of grayscale change of each pixel in the image, and the more blurred the image. The formula for calculating the grayscale image variance is as follows:

[0065]

[0066]

[0067] Where f(x,y) is the gray value at (x,y) in the image, and Nx and Ny represent the number of rows and columns of the pixel in the image, respectively; is the average gray level of the image, and s is the variance of the gray level of the image.

[0068] When the grayscale variance (blur coefficient) is not less than the first threshold, the original image is considered clear.

[0069] When the grayscale variance (blur coefficient) is less than the first threshold, the original image is determined to be blurred.

[0070] If the original image is determined to be blurry, proceed to the next operation; otherwise, if the original image is determined to be clear, it can be determined that the original image is not a suspected lens-occluded image.

[0071] In image processing, an image can be divided into a foreground region and a background region. The subject captured by the lens is located in the foreground region of the image. When a target object is attached to the lens, the lens goes out of focus, and the captured image is distorted, resulting in distortion of the foreground region of the captured image.

[0072] Therefore, after identifying the blurred image, to determine the suspected lens occlusion region within it, it is first necessary to divide the blurred image into foreground and background regions. Then, based on the largest connected component in the foreground region of the corresponding grayscale image, the suspected lens occlusion region is identified. Here, a connected component in an image refers to a region composed of pixels with the same pixel value and adjacent positions. Therefore, the largest connected component is the largest region composed of pixels with the same pixel value and adjacent positions.

[0073] Specifically, regarding the division of the foreground and background regions, the blurred image is first divided into several image blocks. In this embodiment, the first grayscale image is divided into 128×128 image blocks. Generally, video images have a resolution of 1080P and an image size of 1920×1080 pixels. Therefore, to facilitate division, the first grayscale image corresponding to the blurred image determined in the above steps needs to be scaled, resulting in a size of 1024×1024 pixels. Then, the image is divided into 128×128 image blocks, with each block having a size of 8×8 pixels.

[0074] Next, based on the average pixel value of each image block, the output pixel value of each image block is determined; wherein the output pixel value indicates the overall brightness of all pixels in the image block.

[0075] In this embodiment, the output pixel value of each pixel block can be determined by: averaging the pixel values ​​of these pixels to obtain the average pixel value (Avg_Pixel_Value); finally, recording the average pixel value as the flag pixel value (Flag_Pixel_Value) of the corresponding small image. The flag pixel values ​​of all image blocks are compared sequentially with a flag threshold. When the flag pixel value is greater than the flag threshold, the pixel value with the largest corresponding pixel value in that image block is determined as the output pixel value. When the flag pixel value is less than or equal to the flag threshold, the pixel value with the smallest corresponding pixel value in that image block is determined as the output pixel value.

[0076] The grayscale values ​​of a grayscale image range from 0 to 255. In this embodiment, the median value of 127 is set as the labeling threshold. When the output pixel value is greater than 127, the pixel with the largest corresponding pixel value in the image block is selected as the output pixel value. When the labeling pixel value is less than or equal to 127, the pixel with the smallest corresponding pixel value in the image block is selected as the output pixel value.

[0077] Then, the foreground and background regions of the blurred image are determined based on the output pixel values ​​of each image block; that is, the region (foreground region or background region) to which each image block belongs is determined according to the output pixel values ​​of each image block. The output pixel values ​​of the image blocks are compared with a second threshold. If the output pixel value of an image block is greater than the second threshold, the image block is determined to be a background region; if the output pixel value of an image block is not greater than the second threshold, the image block is determined to be a foreground region. The formula for calculating the second threshold is as follows:

[0078] Threshold = (x + y) / 2

[0079] Where x is the minimum pixel value of the output pixel values ​​of all image blocks; y is the average value of the output pixel values ​​of all image blocks.

[0080] Finally, the maximum connected component and the maximum number of connected cells in the foreground region of the blurred image are determined. The maximum connected component refers to the largest region in the blurred image composed of pixels with the same pixel value and adjacent positions. The maximum number of connected cells is the area of ​​the maximum connected component. Specifically, after determining the regions to which all image patches belong, the connected components of the foreground and background regions in the image are determined according to an algorithm (e.g., dynamic programming). In the blurred image, the foreground region with the most image patches is extracted as the maximum connected component, and the area of ​​the image patches in the maximum connected component is recorded as the maximum number of connected cells (Num_Connection_Cell).

[0081] If the maximum number of connected components in the blurred image is greater than the third threshold, the original image corresponding to the blurred image is determined to be a suspected lens occlusion image, meaning there may be a target object attached to the lens. Otherwise, if the maximum number of connected components in the blurred image is not greater than the third threshold, the original image corresponding to the blurred image is determined to be a non-suspected lens occlusion image.

[0082] Generally, a potentially contaminated area is considered to exist when its potential area exceeds 12% of the total area. In this embodiment, the original image, after scaling, has an area of ​​128×128. Based on 128×128×12%≈2000, this embodiment can set 2000 as the third threshold. When the maximum number of connected images is greater than 2000, the original image corresponding to the blurred image is determined to be a suspected lens occlusion image, and the connected image area on this image is a suspected occlusion area, meaning there may be a target object attached to the lens. Therefore, the original image is cached. When the maximum number of connected images is not greater than 2000, the original image corresponding to the blurred image is determined to be a non-suspected lens occlusion image, and this image is no longer subject to subsequent operations.

[0083] It should be noted that, based on the original image obtained in step 101, when determining whether an image is a suspected occlusion image in this step, after determining that an original image is a suspected occlusion image, it is necessary to confirm whether the adjacent original image is a suspected occlusion image. If so, the subsequent operation steps are executed; if not, it is determined that the image is not a suspected occlusion image, and it can also be determined that the previous suspected occlusion image was not caused by a target object attached to the lens. At this time, neither of the two suspected occlusion images will participate in the subsequent operation.

[0084] Step 102: Based on the suspected occlusion areas contained in the at least two suspected lens occlusion images, determine whether there is a target object attached to the lens to be processed.

[0085] Once a suspected lens occlusion image is identified, it can be further confirmed as a lens occlusion image, i.e., an object is attached to the lens to be processed, by determining the similarity value of the suspected occlusion areas contained in the at least two suspected lens occlusion images. The similarity value indicates the degree of similarity between the suspected occlusion areas contained in the at least two suspected lens occlusion images.

[0086] In this embodiment, the similarity value of the suspected occlusion region is determined by calculating the intersection-over-union (IoU) ratio between each pair of at least two suspected lens occlusion images.

[0087] Intersection over Union (IoU) is a method of representing the overlap ratio of two regions by using the area of ​​their circumscribed rectangles. In this embodiment, the number of pixel blocks can be used for calculation. The formula for calculating IoU is as follows:

[0088]

[0089] Where A is the number of image blocks occupied by dirty areas in the first original image, B is the number of image blocks occupied by dirty areas in the second original image, and C is the number of overlapping image blocks in the first and second original images.

[0090] After determining the similarity value, the similarity value is further compared with the fourth threshold, that is, the intersection-union ratio is compared with the fourth threshold.

[0091] When the cross-union ratio (CUI) is greater than the fourth threshold, it is determined that dirt is occluding the same area in the first and second original images, and the suspected lens occlusion image is confirmed as a lens occlusion image, i.e., it is determined that there is a target object on the lens. When the CUI is not greater than the fourth threshold, it is determined that the suspected occlusion is not caused by a target object on the lens, i.e., there is no target object on the lens.

[0092] Following the above steps, a method for detecting a target object attached to a lens can be completed. Specific examples are provided below. Figure 2 The diagram shows a method for detecting whether a lens has a stain (target object) based on the original image captured by the lens.

[0093] First, the first raw image is obtained from the sequence of raw images captured by the lens to be processed.

[0094] Then, a blur determination is performed on the first original image. Specifically, the first original image needs to be converted to grayscale first. Then, the grayscale variance value of the first original image is calculated. If the grayscale variance value of the first original image is less than a first threshold, the first original image is determined to be blurry; otherwise, the first original image is determined to be clear, and there is no target object attached to the lens, and the detection is terminated.

[0095] After determining that the first original image is blurred, the foreground and background regions in the first original image are divided. Specifically, firstly, based on the size of the image captured by the lens, the first original image is divided into an integer number of image blocks. Currently, the resolution of the video image is 1080p, and the size is 1920×1080 (pixels). This image size is divided into 128×128 image blocks. To facilitate division, the first original image is scaled, and the scaled first original image has a size of 1024×1024 (pixels). Next, the average pixel value of each image block in the first original image is calculated, and the average pixel value is compared with a labeling threshold to determine the output pixel value for each image block. If the average pixel value is greater than the labeling threshold, the pixel value with the largest corresponding pixel value in the entire image block is determined as the output pixel value. If the labeling pixel value is less than or equal to the labeling threshold, the pixel value with the smallest corresponding pixel value in the entire image block is determined as the output pixel value. The output pixel value is compared with a second threshold. If the output pixel value is greater than the second threshold, the image block corresponding to the output pixel value is determined to be the background region; if the output pixel value is not greater than the second threshold, the image block corresponding to the output pixel value is determined to be the foreground region.

[0096] After determining the foreground and background regions of the first original image, a suspected occlusion judgment can be performed on the first original image. Here, the first original image is determined to be a suspected occluded image by calculating the largest connected component in the foreground region. When the largest number of connected components (the area of ​​the largest connected component) in the foreground region of the first original image is greater than a third threshold, the first original image is determined to be a suspected lens occlusion image; otherwise, the first original image is not a suspected lens occlusion image, and it is determined that there is no target object on the lens, and the detection process is exited.

[0097] After determining that the first original image is a suspected occluded image, the first original image needs to be cached.

[0098] Then, for the second original image that follows the first original image and is adjacent to the first original image, a blur judgment is made. If it is, the second original image is determined to be blurry; otherwise, the first original image is determined to be clear. It can be determined that there is no target object attached to the lens. Therefore, the first original image will no longer be used as a suspected occlusion image for subsequent judgment. At this time, the first original image and the second original image are deleted, and the detection program is exited.

[0099] After determining that the second original image is blurred, the foreground and background regions within it are segmented. Based on the maximum connected component of the foreground region, a suspected occlusion assessment is performed on the second original image. If the maximum number of connected components (the area of ​​the maximum connected component) in the foreground region of the second original image is greater than a third threshold, the second original image is determined to be a suspected lens occlusion image; otherwise, it is considered a non-suspected lens occlusion image, and since no target object is attached to the lens, the first original image is no longer considered a suspected occlusion image for further assessment. At this point, both the first and second original images are deleted, and the detection process exits.

[0100] The methods for blur detection and suspected occlusion detection of the second original image are the same as those for the first original image, and will not be repeated here.

[0101] After determining that both the first and second original images are suspected occlusion images, based on the principle that the location and shape of dirt (target object) on the lens do not change in the short term, occlusion can be determined by the fact that the detected suspected lens occlusion images contain suspected occlusion areas. Specifically, the similarity value of the suspected occlusion areas contained in the first and second original images can be represented by the intersection-over-union ratio (IoU). The IoU of the first and second original images is then calculated. When the IoU is greater than a fourth threshold, the first and second original images are determined to be occlusion images, i.e., dirt is attached to the lens; if the IoU is less than or equal to the fourth threshold, no dirt (target object) is attached to the lens.

[0102] Based on the same inventive concept, this application provides a device for detecting a target object attached to a lens, which is similar to the aforementioned device. Figure 1 The method for detecting a target object attached to the lens shown corresponds to this method. For a detailed description of the device, please refer to the foregoing description of the method embodiments. Repeated descriptions will not be repeated here. Figure 3 The device includes:

[0103] Detection unit 301: used to detect the original image sequence acquired by the lens to be processed, in response to the detection of at least two suspected lens occlusion images in the original image sequence.

[0104] The device for detecting a target object attached to a detection lens further includes a computing unit, specifically used to determine the blur coefficient of each original image in the original image sequence; wherein the blur coefficient indicates whether the original image is distorted; and original images with blur coefficients less than a blur threshold are determined to be blurred images.

[0105] The detection unit 301 is specifically used in response to detecting at least two suspected lens occlusion images in the blurred image.

[0106] The suspected occlusion area is the largest connected region in the corresponding suspected lens occlusion image;

[0107] The detection unit 301 is further configured to determine that the original image is a suspected occluded image when the maximum number of connected components in the blurred image exceeds a third threshold; the maximum number of connected components is the area of ​​the maximum connected region.

[0108] The detection unit 301 is further configured to divide the blurred image into multiple image blocks containing an integer number of pixels; determine the output pixel value of each image block based on the average pixel value of each image block; wherein the output pixel value indicates the overall brightness of all pixels in the image block; determine the foreground region and background region of the blurred image based on the output pixel value of each image block; wherein when the output pixel value of the image block is greater than a second threshold, the image block belongs to the background region; otherwise, the image block is determined to belong to the foreground region; determine the maximum connected component and the maximum number of connected images in the foreground region of the blurred image; wherein the maximum connected component refers to the largest region composed of pixels with the same pixel value and adjacent positions in the blurred image; when the number of the maximum connected images in the blurred image is greater than a third threshold, the blurred image is determined to be a suspected lens occlusion image.

[0109] The detection unit 301 is further configured to: determine that the first original image is a suspected lens occlusion image and cache the first original image; based on the cached first original image, determine whether the second original image is a suspected lens occlusion image; if so, cache the second original image; if not, delete the first original image and the second original image; wherein the second original image is the original image that follows the first original image and is adjacent to the first original image.

[0110] Determination unit 302: used to determine whether there is a target object on the lens to be processed based on the suspected occlusion area contained in the detected suspected lens occlusion image.

[0111] Specifically, it is used to determine the similarity value of the suspected occlusion regions contained in the at least two suspected lens occlusion images; wherein the similarity value indicates the degree of similarity of the suspected occlusion regions contained in different suspected lens occlusion images among the at least two suspected lens occlusion images;

[0112] If the similarity value is greater than the fourth threshold, it is determined that the lens to be processed has a target object attached to it; otherwise, the lens to be processed has no target object.

[0113] Based on the same inventive concept, embodiments of this application also provide a device for detecting a target object attached to a lens, comprising:

[0114] The memory stores computer instructions;

[0115] The processor is configured to read the computer instructions and execute the following steps:

[0116] Detect the original image sequence captured by the lens to be processed;

[0117] In response to the detection of at least two suspected lens occlusion images in the original image sequence;

[0118] Based on the suspected occlusion areas contained in the at least two suspected lens occlusion images, it is determined whether there is a target object attached to the lens to be processed.

[0119] Based on the same inventive concept, embodiments of this application also provide a readable storage medium, including:

[0120] memory,

[0121] The memory is used to store instructions that, when executed by a processor, cause the apparatus including the readable storage medium to perform the method described above for detecting a target object attached to a lens.

[0122] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0123] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: Universal Serial Bus flash disks, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0127] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting a target object attached to a lens, characterized in that, The method includes: The original image sequence captured by the lens to be processed is detected to determine the blur coefficient of each original image in the original image sequence; wherein, the blur coefficient indicates whether the original image is distorted; Original images with a blur coefficient less than the blur threshold are identified as blurred images; In response to the detection of at least two suspected lens occlusion images in the blurred image, including: The blurred image is divided into multiple image blocks containing an integer number of pixels; Based on the average pixel value of each image block, the output pixel value of each image block is determined; wherein the output pixel value indicates the overall brightness of all pixels in the image block; The foreground and background regions of the blurred image are determined based on the output pixel values ​​of each image block; wherein the output pixel values ​​of the image blocks included in the background region are greater than a second threshold; and the output pixel values ​​of the image blocks included in the foreground region are not greater than the second threshold. Determine the maximum connected component and the maximum number of connected images in the foreground region of the blurred image; wherein, the maximum connected component refers to the largest region composed of pixels with the same pixel value and adjacent positions; the maximum number of connected images is the area of ​​the maximum connected component; When the number of the largest connected images in the blurred image is greater than the third threshold, the blurred image is determined to be a suspected lens occlusion image; otherwise, the blurred image is determined to be a non-suspected lens occlusion image. Based on the suspected occlusion regions contained in the at least two suspected lens occlusion images, it is determined whether there is a target object attached to the lens to be processed; the suspected occlusion region is the largest connected region in the corresponding suspected lens occlusion image.

2. The method as described in claim 1, characterized in that, The step of determining whether a target object is attached to the lens to be processed based on the suspected occlusion area contained in the at least two suspected lens occlusion images includes: Determine the similarity value of the suspected occlusion regions contained in the at least two suspected lens occlusion images; wherein the similarity value indicates the degree of similarity between the suspected occlusion regions contained in the different suspected lens occlusion images; If the similarity value is greater than the fourth threshold, it is determined that the lens to be processed has a target object attached to it; otherwise, the lens to be processed has no target object.

3. The method as described in claim 1, characterized in that, After detecting the original image sequence acquired by the lens to be processed, the process further includes: The first original image is identified as a suspected lens occlusion image, and the first original image is cached. Based on the cached first original image, determine whether the second original image is a suspected lens occlusion image; if so, cache the second original image; if not, delete the first original image and the second original image; wherein, the second original image is the original image that follows the first original image and is adjacent to the first original image.

4. A device for detecting a target object attached to a lens, characterized in that, The device includes: The calculation unit is specifically used to determine the blur coefficient of each original image in the original image sequence; wherein the blur coefficient indicates whether the original image is distorted; and the original images with blur coefficients less than the blur threshold are determined as blurred images. Detection unit: used to detect the original image sequence acquired by the lens to be processed, in response to detecting at least two suspected lens occlusion images in the blurred image, including: dividing the blurred image into multiple image blocks containing an integer number of pixels; determining the output pixel value of each image block based on the average pixel value of each image block; wherein the output pixel value indicates the overall brightness of all pixels in the image block; determining the foreground region and background region of the blurred image based on the output pixel value of each image block; wherein the output pixel value of the image blocks included in the background region is greater than a second threshold; the output pixel value of the image blocks included in the foreground region is not greater than the second threshold; determining the maximum connected component and the maximum number of connected images in the foreground region of the blurred image; wherein the maximum connected component refers to the largest region composed of pixels with the same pixel value and adjacent positions; the maximum number of connected images is the area of ​​the maximum connected component; when the maximum number of connected images in the blurred image is greater than a third threshold, the blurred image is determined to be a suspected lens occlusion image; otherwise, the blurred image is determined to be a non-suspected lens occlusion image. Determination unit: used to determine whether there is a target object on the lens to be processed based on the suspected occlusion area contained in the at least two suspected lens occlusion images; the suspected occlusion area is the largest connected region in the corresponding suspected lens occlusion image.

5. The apparatus as described in claim 4, characterized in that, The detection unit is specifically used to determine the similarity value of the suspected occlusion areas contained in the at least two suspected lens occlusion images; wherein, the similarity value indicates the degree of similarity of the suspected occlusion areas contained in different suspected lens occlusion images; when the similarity value is greater than a fourth threshold, it is determined that there is a target object on the lens to be processed; otherwise, there is no target object on the lens to be processed.

6. A device for detecting a target object attached to a lens, characterized in that, include: Memory stores computer instructions; The processor is configured to read the computer instructions and execute the following steps: Detect the original image sequence captured by the lens to be processed; Determine the blur coefficient of each original image in the original image sequence; wherein the blur coefficient indicates whether the original image is distorted; Original images with a blur coefficient less than the blur threshold are identified as blurred images; In response to the detection of at least two suspected lens occlusion images in the blurred image, including: The blurred image is divided into multiple image blocks containing an integer number of pixels; Based on the average pixel value of each image block, the output pixel value of each image block is determined; wherein the output pixel value indicates the overall brightness of all pixels in the image block; The foreground and background regions of the blurred image are determined based on the output pixel values ​​of each image block; wherein the output pixel values ​​of the image blocks included in the background region are greater than a second threshold; and the output pixel values ​​of the image blocks included in the foreground region are not greater than the second threshold. Determine the maximum connected component and the maximum number of connected images in the foreground region of the blurred image; wherein, the maximum connected component refers to the largest region composed of pixels with the same pixel value and adjacent positions; the maximum number of connected images is the area of ​​the maximum connected component; When the number of the largest connected images in the blurred image is greater than the third threshold, the blurred image is determined to be a suspected lens occlusion image; otherwise, the blurred image is determined to be a non-suspected lens occlusion image. Based on the suspected occlusion regions contained in the at least two suspected lens occlusion images, it is determined whether there is a target object attached to the lens to be processed; the suspected occlusion region is the largest connected region in the corresponding suspected lens occlusion image.

7. A readable storage medium, characterized in that, include: memory, The memory is used to store instructions that, when executed by a processor, cause a device including the readable storage medium to perform the method as described in any one of claims 1 to 3.

Images