Precise detection method for lens dirt, terminal device and computer-readable storage medium

Abstract

The embodiment of the present invention discloses a method for accurately detecting lens dirt, a terminal device and a computer-readable storage medium; the method includes the steps of: S1, acquiring image data collected by a camera, and detecting all potential dirty target areas in the image data; S2, constructing image-specific description features for the dirty target areas; S3, screening the dirty target areas according to the image-specific description features; S4, grading and scoring the screened dirty target areas, and accurately grading the dirty target areas according to the grading scores. The water immersion method is used to detect all potential dirty target areas in the image data, which improves the detection accuracy and recall rate of lens dirt, constructs exclusive description features for the detected dirty target areas, determines the grading scores based on the exclusive description features, and accurately grades the dirty target areas according to the grading scores, which provides a basis and method for defective product classification and processing, and has good application prospects in the field of image processing technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of electrical data processing, and specifically relates to a method for accurately detecting lens dirt, a terminal device, and a computer-readable storage medium. Background Art

[0002] In modern digital imaging technology, the lens is a key component for capturing images and videos, and its cleanliness directly affects the quality of the final shooting effect. Dirty lenses, such as dust, fingerprints, water stains, etc., can cause problems such as blurred images, reduced contrast, and color distortion, which seriously affects user experience and imaging quality.

[0003] Traditional lens dirt detection methods mainly rely on manual inspection, which is not only inefficient but also easy to miss subtle dirt spots. With the development of computer vision and image processing technology, automated lens dirt detection technology has gradually become a research hotspot. Existing automated detection methods are mainly based on image processing technology, which determines whether the lens is dirty by comparing the difference between the original image captured by the camera and the image after specific processing. However, the accuracy and robustness of these methods still need to be improved when dealing with complex scenes or multiple types of dirt.

[0004] In recent years, with the development of deep learning, lens dirt detection technology has made significant progress. However, these methods have high requirements for computing resources when processing large amounts of data or real-time detection, and the materials required for model training are not easy to obtain. The model training and tuning process is also relatively complicated. More importantly, existing technologies often only focus on the presence or absence of dirt, but ignore the severity of the dirt.

[0005] In practical applications, the severity of lens dirt has different effects on image quality. Generally speaking, the larger the area of ​​the dirt spot and the higher the contrast, the greater its impact on image quality, so it needs to be processed with higher priority. However, most existing lens dirt detection methods simply determine whether the lens is dirty, without accurately evaluating the severity of the dirt. Summary of the invention

[0006] In view of this, on the one hand, some embodiments disclose a method for accurately detecting lens dirt, comprising the steps of:

[0007] S1, obtaining image data collected by a camera and detecting all potential dirty target areas in the image data;

[0008] S2, construct image-specific description features for the dirty target area;

[0009] S3, screening the dirty target area according to the image-specific description features;

[0010] S4. The screened dirty target areas are graded and scored, and the dirty target areas are accurately classified according to the graded scores.

[0011] On the other hand, some embodiments disclose a terminal device, including a memory and a processor; a computer program is stored in the memory, and when the processor executes the computer program, the terminal device executes a method for accurately detecting lens dirt.

[0012] On the other hand, some embodiments disclose a computer-readable storage medium storing a computer program. When the computer program is executed, a method for accurately detecting lens dirt is performed.

[0013] The method for accurately detecting lens dirt disclosed in the embodiment of the present invention uses the water immersion method to detect all potential dirty target areas in image data, which greatly improves the detection accuracy and recall rate of lens dirt, further constructs exclusive description features for the detected dirty target areas, determines a grading score based on the exclusive description features, and then accurately grades the dirty target areas according to the grading score, which provides a basis and method support for the classification and processing of defective products and has good application prospects in the field of image processing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of the method for accurately detecting lens dirtiness disclosed in Example 1;

[0015] Figure 2 Flow chart of the method for detecting all potential dirty target areas in image data using water immersion method disclosed in Example 2;

[0016] Figure 3 This is a flow chart of the method for constructing image-specific description features for a dirty target area disclosed in Example 3. DETAILED DESCRIPTION

[0017] The word "embodiment" is used here exclusively. Any embodiment described as "exemplary" need not be interpreted as being superior or better than other embodiments. Performance index tests in the embodiments of the present invention are conducted using conventional test methods in the art unless otherwise specified. It should be understood that the terms in the embodiments of the present invention are only used to describe specific implementation methods and are not intended to limit the contents disclosed in the embodiments of the present invention.

[0018] Unless otherwise specified, the technical and scientific terms used herein have the same meanings as commonly understood by ordinary technicians in the technical field to which the embodiments of the present invention belong; other experimental methods and technical means not specifically specified in the embodiments of the present invention refer to experimental methods and technical means commonly used by ordinary technicians in the field.

[0019] The terms "substantially" and "approximately" used herein are used to describe small fluctuations. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. The numerical data represented or presented in the range format herein are used only for convenience and brevity, and should therefore be flexibly interpreted as including not only the values ​​clearly listed as the limits of the range, but also all independent values ​​or sub-ranges contained in the range. For example, the numerical range of "1-5%" should be interpreted as including not only the clearly listed values ​​of 1% to 5%, but also the independent values ​​and sub-ranges within the range shown. Therefore, independent values ​​such as 2%, 3.5% and 4% and sub-ranges such as 1%-3%, 2%-4% and 3%-5% are included in this numerical range. This principle also applies to the range of only one numerical value. In addition, such an interpretation applies regardless of the width of the range or the characteristics described.

[0020] In this document, including in the claims, transitional words such as "comprises," "includes," "with," "having," "containing," "involving," "accommodating," etc. are understood to be open-ended, i.e., meaning "including but not limited to." Only the transitional words "consisting of" and "composed of" are closed transitional words.

[0021] In order to better illustrate the present invention, numerous specific details are provided in the specific examples below. It should be understood by those skilled in the art that the present invention can be implemented without certain specific details. In the embodiments, some methods, means, instruments, equipment, etc. well known to those skilled in the art are not described in detail in order to highlight the gist of the present invention.

[0022] Under the premise of no conflict, the technical features disclosed in the embodiments of the present invention may be arbitrarily combined, and the resulting technical solutions belong to the contents disclosed in the embodiments of the present invention.

[0023] In some embodiments, a method for accurately detecting lens dirtiness comprises the steps of:

[0024] S1, obtaining image data collected by a camera and detecting all potential dirty target areas in the image data;

[0025] S2, construct image-specific description features for the dirty target area;

[0026] S3, screening the dirty target area according to the image-specific description features;

[0027] S4. Grade and score the screened dirty target areas, and accurately grade the dirty target areas.

[0028] Furthermore, in some embodiments of the method for accurately detecting lens dirt, in step S1, all potential dirty target areas in the image data are detected using a water immersion method, specifically including:

[0029] S101, converting the YUV image data collected by the camera into a grayscale image and performing filtering processing; the grayscale image is usually filtered to make the image smoother and reduce the influence of noise, for example, filtering methods such as mean filtering, median filtering, Gaussian filtering, bilateral filtering, wavelet transform filtering, etc. can be selected;

[0030] S102, sorting the pixel values ​​of the grayscale image in ascending order and putting them into a pixel list;

[0031] S103, traverse the pixel list, use the current pixel value as the threshold to perform threshold segmentation on the grayscale image, binarize it, generate a black and white binary image, and store it in a binary image sequence; usually, threshold segmentation refers to dividing the image into black and white parts, and the pixels greater than the threshold are set to white, with a pixel value of 255, and the rest are all set to black, with a pixel value of 0; the grayscale image becomes a black and white image after threshold segmentation, in which the pixels have only black and white, 0 represents black, and 255 represents white;

[0032] S104, traverse the binary image sequence, perform connected domain analysis, and extract connected domains that meet the set conditions; usually, the set conditions for extracting connected domains can be determined as the area value range of the connected domain; usually, a connected domain refers to an area composed of pixels with the same pixel value and adjacent positions in the image. In a binary image, a connected domain usually refers to a continuous area composed of white pixels or black pixels, and these pixels are connected in space; generally, connectivity can be divided into 4-connected and 8-connected, where if two pixels are adjacent in the horizontal or vertical direction, they are 4-connected; if two pixels are adjacent in the horizontal, vertical or diagonal direction, they are 8-connected; usually, it is necessary to detect all connected domains in the binary image sequence, and then filter them according to preset conditions;

[0033] Usually, after detecting the connected domain, we can know the number of pixels in the connected domain, that is, the area of ​​the connected domain. The specific threshold depends on the specific task. For example, it can be determined that when the area of ​​the connected domain is less than 200, it belongs to the range of noise and is ignored, and is not considered to be a dirty target area;

[0034] S105, calculating the region information of the extracted connected domain, storing it in the dirty target region sequence, and obtaining all potential dirty target regions; the region information includes the area of ​​the connected domain, the width and height of the circumscribed rectangle, and the position coordinates.

[0035] In some embodiments of the disclosed lens dirt accurate detection method, step S2 constructs an image-specific description feature for the dirty target area, specifically including:

[0036] S201, traversing a dirty target region sequence, and intercepting image data of a corresponding region on the original grayscale image according to region information of the dirty target region; usually, the region information of the dirty target region includes coordinate position, target size, etc.;

[0037] S202, calculating the gray level co-occurrence matrix of the image data;

[0038] S203, extracting texture features of the image according to the gray level co-occurrence matrix to obtain image-specific description features;

[0039] Gray Level Co-occurrence Matrix (GLCM) is an image processing and texture analysis method that extracts texture features by analyzing the spatial relationship between gray levels in an image. For each pixel in the image, another pixel with a distance d and a direction θ from it is examined, and the probability of the gray levels of these two pixels occurring at the same time is counted. These probability values ​​are combined into a matrix, namely the gray level co-occurrence matrix. The gray level co-occurrence matrix can be used to extract the texture features of the image, such as energy, homogeneity, contrast, correlation, and dissimilarity. For example, the scikit-image library can be used in Python to calculate the gray level co-occurrence matrix.

[0040] Generally, one or more texture features may be selected as image transformation description features according to image processing requirements. In some embodiments, the selected texture features are contrast and dissimilarity of a gray-level co-occurrence matrix.

[0041] In some embodiments of the disclosed lens dirt accurate detection method, the image-specific description feature also includes the “center-to-whole” contrast of the dirty target area in the image, and the calculation formula of the “center-to-whole” contrast is:

[0042] C=|M a -M c |

[0043]

[0044] Among them, M a is the overall pixel mean of the target area, M c is the mean pixel value in the center of the target area, C is the “center-overall” contrast; H a is the height of the target area's bounding rectangle, W a is the width of the target area's bounding rectangle, H c is the height of the center of the target area's circumscribed rectangle, W cis the center width of the circumscribed rectangle of the target area, and I(i,j) is the pixel value of the i-th row and j-th column of the image in the target area; usually, in the "center-overall" contrast, the center refers to the central area of ​​the dirty target area, and the body refers to the entire dirty target area. Generally, the width and height of the central area can be set to half of the width and height of the entire area, respectively.

[0045] In some embodiments of the disclosed method for accurately detecting lens dirt, in step S3, the image-specific description features used to screen the dirty target area are "center-to-whole" contrast, gray-level co-occurrence matrix contrast, and gray-level co-occurrence matrix dissimilarity; usually, after the image-specific description features are selected, the dirty target area can be further screened according to the features to filter out target areas that do not meet the conditions, thereby avoiding false detections while ensuring a high recall rate. For example, when the image-specific description features of the dirty target area are less than a certain threshold, the dirty target area can be filtered out;

[0046] In order to screen the dirty target area, the following specific conditions are set:

[0047] ① The “center-to-whole” contrast is less than 0.8, and the gray-level co-occurrence matrix contrast is less than 0.5;

[0048] ②The gray level co-occurrence matrix contrast is less than 0.02;

[0049] ③The gray level co-occurrence matrix dissimilarity is less than 0.05;

[0050] If the dirty target area meets one of the above conditions, the dirty target area will be ignored. Generally, the threshold of the image-specific description feature needs to be analyzed according to the specific task, and the specific empirical value of the specific task is determined after empirical summary as the threshold of the image-specific description feature; generally, a dirty lens is photographed in a specific production environment to obtain specific image features, so the specific range of the threshold can be determined through actual testing, and the threshold is further determined based on the empirical value obtained from the actual test, which can well detect dirty lenses;

[0051] In some embodiments of the disclosed lens dirt accurate detection method, step S4 specifically includes traversing the dirty target area list, determining the grading score of each dirty target area, and accurately grading the dirty target area according to the grading score; wherein the grading score S is expressed as:

[0052] S=S1×α+S2×β

[0053] Among them, S1 represents the score of the area part of the dirty target area, and α is its corresponding weight; S2 represents the score of the "center-overall" contrast part of the dirty target area, and β is its corresponding weight, α+β=1.

[0054] In some embodiments, α is 0.4 and β is 0.6. Generally, the weight of the area of ​​the dirty target area and the weight of the "center-to-whole" contrast depend on the experience of the specific task; if practical experience shows that the "center-to-whole" contrast has a greater impact, then the corresponding weight β should be increased. For example, if there is a very black dirty spot on the image, which means the contrast is very high, but its area is very small, if this defect is to be treated with higher priority, the weight of the contrast should be increased.

[0055] Some embodiments disclose a method for accurately detecting lens dirt, and based on the area a of the dirty target area, calculate a score S1 of the area of ​​the dirty target area. The calculation method includes:

[0056] If a is in the range [0,625], then:

[0057] If a is in the range (625,1600], then:

[0058] If a is in the range (1600,3025], then:

[0059] If a is in the range (3025,4000], then:

[0060] If a is in the range (4000,5000], then:

[0061] If a is in the range (5000,6500], then:

[0062] If a is in the range (6500,∞), then:

[0063] If the calculation result of S1 is greater than 100, it is assigned a value of 100; wherein, in the above S1 calculation formula, the calculation result is expressed as an integer;

[0064] Based on the “center-to-whole” contrast c of the dirty target area, the scoring score S2 of the “center-to-whole” contrast part of the dirty target area is calculated, and the specific method includes:

[0065] If c is in the range [0,4], then:

[0066] If c is in the range (4,9], then:

[0067] If c is in the range (9,13], then:

[0068] If c is in the range (13,17], then:

[0069] If c is in the range (17,21], then:

[0070] If c is in the range (21,25], then:

[0071] If c is in the range (25,∞), then:

[0072] If the calculation result of S2 is greater than 100, it is assigned a value of 100. In the above S2 calculation formula, the calculation result is expressed as an integer.

[0073] The terminal device disclosed in some embodiments includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the terminal device executes a method for accurately detecting lens dirt.

[0074] Some embodiments disclose a computer-readable storage medium, which stores a computer program. When the computer program is executed, a method for accurately detecting lens dirt is performed.

[0075] The technical details are further illustrated below in conjunction with embodiments.

[0076] Example 1

[0077] In Example 1, Figure 1 As shown, the method for accurately detecting lens dirtiness includes the following steps:

[0078] S1, obtaining image data collected by a camera and detecting all potential dirty target areas in the image data;

[0079] S2, construct image-specific description features for the dirty target area;

[0080] S3, screening the dirty target area according to the image-specific description features;

[0081] S4. Grade and score the screened dirty target areas, and accurately grade the dirty target areas.

[0082] Example 2

[0083] In Example 2, Figure 2 As shown, the method of detecting all potential dirty target areas in image data using the water immersion method includes the steps of:

[0084] S101, converting the YUV image data collected by the camera into a grayscale image and performing filtering processing;

[0085] S102, sorting the pixel values ​​of the grayscale image in ascending order and putting them into a pixel list;

[0086] S103, traverse the pixel list, perform threshold segmentation on the grayscale image with the current pixel value as the threshold, binarize it, generate a black and white binary image, and store it in a binary image sequence;

[0087] S104, traversing the binary graph sequence, performing connected domain analysis, and extracting connected domains that meet set conditions; usually, the set conditions for extracting connected domains can be determined as the area value range of the connected domains;

[0088] S105, calculating the region information of the extracted connected domain, storing it in the dirty target region sequence, and obtaining all potential dirty target regions; the region information includes the area of ​​the connected domain, the width and height of the circumscribed rectangle, and the position coordinates.

[0089] Example 3

[0090] In Example 3, Figure 3 As shown, the method for constructing image-specific description features for a dirty target area includes the following steps:

[0091] S201, traversing a dirty target region sequence, and intercepting image data of a corresponding region on the original grayscale image according to region information of the dirty target region; usually, the region information of the dirty target region includes coordinate position, target size, etc.;

[0092] S202, calculating the gray level co-occurrence matrix of the image data;

[0093] S203, extracting texture features of the image according to the gray level co-occurrence matrix to obtain image-specific description features.

[0094] The method for accurately detecting lens dirt disclosed in the embodiment of the present invention uses the water immersion method to detect all potential dirty target areas in image data, which greatly improves the detection accuracy and recall rate of lens dirt, further constructs exclusive description features for the detected dirty target areas, determines a grading score based on the exclusive description features, and then accurately grades the dirty target areas according to the grading score, which provides a basis and method support for the classification and processing of defective products and has good application prospects in the field of image processing technology.

[0095] The technical solutions disclosed in the embodiments of the present invention and the technical details disclosed in the embodiments are merely illustrative of the inventive concept of the present invention and do not constitute a limitation on the technical solutions of the embodiments of the present invention. Any conventional changes, replacements or combinations of the technical details disclosed in the embodiments of the present invention have the same inventive concept as the present invention and are within the protection scope of the claims of the present invention.

Claims

1. A method for accurately detecting lens dirtiness, characterized in that: Includes steps: S1. Obtain image data collected by the camera and use the water immersion method to detect all potential dirty target areas in the image data; specifically including: S101, converting the YUV image data collected by the camera into a grayscale image and performing filtering processing; S102, sorting the pixel values ​​of the grayscale image in ascending order and putting them into a pixel list; S103, traverse the pixel list, perform threshold segmentation on the grayscale image with the current pixel value as the threshold, binarize it, generate a black and white binary image, and store it in a binary image sequence; S104, traversing the binary graph sequence, performing connected domain analysis, and extracting connected domains that meet set conditions; S105, calculating the region information of the extracted connected domain, storing it in the dirty target region sequence, and obtaining all potential dirty target regions; the region information includes the area of ​​the connected domain, the width and height of the circumscribed rectangle, and the position coordinates; S2, constructing image-specific description features for the dirty target area; including: S201, traversing a dirty target region sequence, and intercepting image data of a corresponding region on the original grayscale image according to region information of the dirty target region; S202, calculating the gray level co-occurrence matrix of the image data; S203, extracting texture features of the image according to the gray level co-occurrence matrix to obtain image-specific descriptive features; the texture features include energy, homogeneity, contrast, correlation, and dissimilarity of the gray level co-occurrence matrix; the image-specific descriptive features are "center-to-whole" contrast, gray level co-occurrence matrix contrast, and gray level co-occurrence matrix dissimilarity; the calculation formula of the "center-to-whole" contrast is: C=|M a -M c | Among them, M a is the overall pixel mean of the target area, M c is the mean pixel value in the center of the target area, C is the "center-overall" contrast; H a is the height of the target area's bounding rectangle, W a is the width of the target area's bounding rectangle, H c is the height of the center of the target area's circumscribed rectangle, W c is the center width of the circumscribed rectangle of the target area, I(i,j) is the pixel value of the i-th row and j-th column of the image in the target area; the center width and height are set to half of the overall width and height respectively; S3, screening the dirty target area according to the image-specific description features; wherein the specific conditions of "center-to-whole" contrast, gray level co-occurrence matrix contrast, and gray level co-occurrence matrix dissimilarity are: ① The "center-to-whole" contrast is less than 0.8, and the gray-level co-occurrence matrix contrast is less than 0.5; ②The gray level co-occurrence matrix contrast is less than 0.02; ③The gray level co-occurrence matrix dissimilarity is less than 0.05; If the dirty target area meets one of the above conditions, the dirty target area is ignored; S4. The screened dirty target areas are graded and scored, and the dirty target areas are accurately graded according to the graded scores.

2. The method for accurately detecting lens dirtiness according to claim 1, characterized in that: In step S104, the condition set is the area value range of the connected domain.

3. The method for accurately detecting lens dirtiness according to claim 1, characterized in that: Step S4 specifically includes traversing the dirty target area list, determining the grading score of each dirty target area, and accurately grading the dirty target area according to the grading score; wherein the grading score S is expressed as: S=S1×α+S2×β Among them, S1 represents the score of the area of ​​the dirty target area, and α is its corresponding weight; S2 represents the score of the "center-overall" contrast part of the dirty target area, and β is its corresponding weight, α+β=1.

4. The method for accurately detecting lens dirtiness according to claim 3, characterized in that: Based on the area a of the dirty target area, the scoring score S1 of the dirty target area area is calculated, and the calculation method includes: If a is in the range [0,625], then: If a is in the range (625,1600], then: If a is in the range (1600,3025], then: If a is in the range (3025,4000], then: If a is in the range (4000,5000], then: If a is in the range (5000,∞), then: If the calculated result of S1 is greater than 100, it is assigned a value of 100; Based on the "center-to-whole" contrast c of the dirty target area, the scoring score S2 of the "center-to-whole" contrast part of the dirty target area is calculated, and the specific method includes: If c is in the range [0,4], then: S2 = 10c; If c is in the range of (4,9], then: S2 = 32 + 2c; If c is in the range of (9,25], then: S2=28+3c; If c is in the range of (25,∞), then: S1=40+2c; If the calculated result of S2 is greater than 100, it is assigned the value 100.

5. A terminal device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the processor executes the computer program, the terminal device executes the method for accurately detecting lens dirt as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method for accurately detecting lens dirt as claimed in any one of claims 1 to 4 is executed.

Images