Lens dirt identification method and device, computer device and storage medium

Abstract

Embodiments of the present application disclose a lens dirt identification method and device, computer equipment and a storage medium. The method comprises: acquiring a source image captured by a lens, the source image comprising at least a plurality of color channel values corresponding to each pixel point in the source image; performing brightness conversion processing according to each color channel value to obtain a first brightness image corresponding to the source image; performing fitting processing on the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; performing dark corner removal processing according to the first brightness image and the second brightness image to obtain a target brightness image; and performing dirt detection on the target brightness image to identify whether the lens is dirty. The above method realizes the dark corner removal target brightness image using the first brightness image after brightness conversion and the second brightness image after fitting, and identifies whether the lens is dirty on the premise of removing the dark corner, so the result is more accurate and effective.

Description

Technical Field

[0001] This invention relates to the field of camera manufacturing technology, and in particular to a method, apparatus, computer device, and storage medium for identifying lens dirt. Background Technology

[0002] In the actual production process of camera modules, the workshop is not an ideal dust-free environment. Before the lens is installed, the imaging sensor and filter come into contact with air. Dust, dander, and other foreign objects floating in the air can easily adhere to the imaging sensor or filter, resulting in dirt on the final camera module during imaging, which seriously affects image quality. The current detection method is to compare the brightness of the current area with the surrounding area. When the contrast exceeds a set threshold, it is considered dirt. This method is easily affected by shading, which refers to the lens's vignetting. Due to issues in the mounting process, tilting of the lens or base, problems with incoming materials, etc., can all cause some vignetting, making it easy to misjudge or miss detections at the four corners or edges. Summary of the Invention

[0003] The main objective of this invention is to provide a method, apparatus, computer device, and storage medium for identifying lens dirt, which can solve the problem in the prior art where dark corners exist around the lens, leading to misjudgment or missed judgment of lens dirt.

[0004] To achieve the above objectives, the first aspect of the present invention provides a method for identifying lens dirt, the method comprising:

[0005] Acquire a source image captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image;

[0006] A brightness conversion process is performed based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image;

[0007] The first brightness image is fitted using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image;

[0008] The target brightness image is obtained by performing vignetting removal processing based on the first brightness image and the second brightness image;

[0009] Dirt detection is performed on the target brightness image to identify whether the lens is dirty.

[0010] In one feasible implementation, the step of performing luminance conversion processing based on each of the color channel values ​​to obtain a first luminance image corresponding to the source image includes:

[0011] The brightness value of each pixel is obtained by averaging the values ​​of each color channel corresponding to each pixel and the number of color channels.

[0012] The first brightness image is generated using the brightness values ​​of each pixel.

[0013] In one feasible implementation, the step of performing vignetting removal based on the first brightness image and the second brightness image to obtain the target brightness image includes:

[0014] Obtain the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image;

[0015] The brightness difference between each pixel is obtained by subtracting the first brightness value from the second brightness value of the same pixel.

[0016] By comparing the brightness difference of the same pixel with the first brightness value, the brightness ratio of each pixel is obtained. The brightness ratio is used as the third brightness value of each pixel to obtain the target brightness image generated from the third brightness values ​​of each pixel.

[0017] In one feasible implementation, the step of performing dirt detection on the target brightness image to identify whether the lens is dirty includes:

[0018] Obtain a first target pixel in the target brightness image, wherein the pixel value of the first target pixel is greater than a preset brightness threshold corresponding to the first target pixel;

[0019] If the first target pixel is obtained, the first target pixel is used as a dirt point to determine that the lens is dirty;

[0020] If the first target pixel is not obtained, it is determined that the lens is not dirty.

[0021] In one feasible implementation, the step of performing dirt detection on the target brightness image to identify whether the lens is dirty further includes:

[0022] A second target pixel in the target brightness image is obtained, wherein the gradient value of the second target pixel is greater than a preset brightness change rate threshold corresponding to the second target pixel, and the gradient value is the brightness change rate between adjacent pixels.

[0023] If a second target pixel is obtained, the second target pixel is used as a dirt spot to determine that the lens is dirty;

[0024] If the second target pixel is not obtained, it is determined that the lens is not dirty.

[0025] In one feasible implementation, the method further includes:

[0026] The aforementioned dirt spots are subjected to morphological processing to obtain candidate dirt regions;

[0027] If the candidate dirty area is larger than the preset dirty area threshold, then the candidate dirty area is taken as the target dirty area.

[0028] Output the dirt information of the target dirty area, the dirt information including at least the coordinates and size of the target dirty area.

[0029] In one feasible implementation, the step of performing brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image further includes:

[0030] The first brightness image is resized according to a preset size to obtain the resized first brightness image.

[0031] Gaussian filtering is applied to the transformed first brightness image to obtain the filtered first brightness image.

[0032] To achieve the above objectives, a second aspect of the present invention provides a lens dirt identification device, the device comprising:

[0033] Data acquisition module: used to acquire source images captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image;

[0034] Brightness conversion module: used to perform brightness conversion processing based on the values ​​of each color channel to obtain a first brightness image corresponding to the source image;

[0035] Fitting processing module: used to fit the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image;

[0036] Vignette removal module: used to perform vignetting removal processing based on the first brightness image and the second brightness image to obtain the target brightness image;

[0037] Dirt detection module: used to detect dirt in the target brightness image and identify whether the lens is dirty.

[0038] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps shown in the first aspect and any feasible implementation.

[0039] To achieve the above objectives, a fourth aspect of the present invention provides a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps shown in the first aspect and any feasible implementation.

[0040] The embodiments of the present invention have the following beneficial effects:

[0041] This invention provides a method for identifying lens dirt. The method includes: acquiring a source image captured by the lens, the source image including at least multiple color channel values ​​corresponding to each pixel in the source image; performing brightness conversion processing based on each color channel value to obtain a first brightness image corresponding to the source image; fitting the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; performing vignetting removal processing based on the first brightness image and the second brightness image to obtain a target brightness image; and performing dirt detection on the target brightness image to identify whether the lens is dirty. This method achieves the goal of obtaining a vignetted-removed target brightness image using the first brightness image after brightness conversion and the fitted second brightness image, and identifies whether the lens is dirty while removing vignetting, resulting in more accurate and effective results. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] in:

[0044] Figure 1 This is a flowchart of a lens dirt identification method according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a source image in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of a first brightness image according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of a pseudo-color display of a first brightness image and a second brightness image in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram illustrating a pseudo-color display of a target brightness image and a pseudo-color display of dirt in an embodiment of the present invention;

[0049] Figure 6This is another flowchart of a lens dirt identification method according to an embodiment of the present invention;

[0050] Figure 7 This is a structural block diagram of a lens dirt identification device according to an embodiment of the present invention;

[0051] Figure 8 This is a structural block diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Reference Figure 1 , Figure 1 This is a flowchart of a lens dirt identification method according to an embodiment of the present invention. The lens dirt identification method specifically includes the following steps:

[0054] 101. Obtain a source image captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image;

[0055] In this embodiment, images captured by the lens to be inspected can be used to identify whether the lens is dirty. Specifically, firstly, the source image captured by the lens is acquired. The source image is the unprocessed raw image captured by the lens, that is, an image with a data format of raw. The raw image has multiple color channels, such as four color channels (R, Gr, Gb, B), where R is a red channel, G is a green channel, B is a blue channel, Gr is a mixture of the red and green channels, and Gb is a mixture of the blue and green channels. Please refer to [link to previous document]. Figure 2 , Figure 2 This is a schematic diagram of a source image according to an embodiment of the present invention. Wherein, Figure 2 Source image 201 is the original image captured by the camera for detecting dirt on the lens.

[0056] 102. Perform brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image;

[0057] Furthermore, dirt in the source image typically appears as a faint black patch. Therefore, it can be determined by the brightness values ​​of the source image. Brightness values ​​are generally represented by grayscale values ​​between 0 and 255, where 0 represents black and 255 represents white. Thus, by performing brightness conversion processing on the color channel values ​​corresponding to each pixel in the source image, a first brightness image corresponding to the source image is obtained. This first brightness image is generated from the brightness values ​​of each pixel in the source image obtained after brightness conversion processing. It can be generated directly based on the aforementioned brightness values, or it can be obtained by replacing the color channel values ​​of the same pixel with the brightness value of that pixel. This example does not impose limitations. See also... Figure 3 , Figure 3 This is a schematic diagram of a first brightness image according to an embodiment of the present invention. Wherein, Figure 3 The first brightness image 301 is a brightness image obtained after brightness conversion processing of the source image captured by the camera. The pixel value of each pixel in the first brightness image 301 is the brightness value of each pixel in the source image after brightness conversion.

[0058] 103. Use a fitting algorithm to fit the first brightness image to obtain the second brightness image corresponding to the first brightness image;

[0059] This embodiment requires further processing of the first brightness image. This can be achieved by fitting the first brightness image to obtain a second brightness image, which is the fitted brightness image. The specific fitting method can be using a B-spline fitting surface to process the first brightness image, or other polynomial fitting methods; this example is not limited to any particular method. By fitting the first brightness image and removing redundant pixels, a clearer, fitted second brightness image is obtained. Specifically, if a B-spline fitting surface is used, the data in each column and row of the first brightness image can be fitted to obtain the corresponding B-spline fitting surface, generating the second brightness image (i.e., the B-spline fitting surface imgBspline). For example, refer to... Figure 4 , Figure 4 This is a schematic diagram illustrating the pseudo-color display of a first brightness image and a second brightness image according to an embodiment of the present invention. 401 is a pseudo-color display effect diagram of the first brightness image, and 402 is a pseudo-color display effect diagram of the second brightness image. Figure 4 As can be seen, compared with the pseudo-color display of the first brightness image 401, the pseudo-color display of the second brightness image 402 has a smoother color distribution. Specifically, the first region 4010 in the first brightness image 401 and the second region 4020 in the second brightness image 402 are compared and contrasted, with the second region 4020 being smoother.

[0060] 104. Perform vignetting removal based on the first brightness image and the second brightness image to obtain the target brightness image;

[0061] In this embodiment, to ensure the accuracy of dirt detection, the influence of vignetting around the lens is eliminated. Vignetting refers to the phenomenon of shadows, specifically the dark corners of the image caused by insufficient incident light. Due to issues in the mounting process, lens or base tilt, material problems, etc., can all lead to vignetting in the lens image. Shading is broadly divided into two types: Lens Shading: Due to the optical characteristics of the lens, the light intensity received at the edges of the image area of ​​the imaging sensor is less than that at the center, resulting in inconsistent brightness between the center and the corners, i.e., vignetting; Color Shading: Due to insufficient incident angles at the periphery of the lens, color deviation occurs, generally manifested as inconsistencies in color between the center and the surrounding areas.

[0062] Furthermore, vignetting can be performed on the obtained first and second brightness images to obtain a target brightness image with vignetting removed. This vignetting process refers to removing vignetting features from the image.

[0063] 105. Perform dirt detection on the target brightness image to identify whether the lens is dirty.

[0064] Furthermore, dirt detection is performed on the target brightness image after removing vignetting to identify whether the lens is dirty, thus improving the accuracy of lens dirt detection. For details, please refer to... Figure 5 This is a schematic diagram illustrating the pseudo-color display of dirt in a target brightness image according to an embodiment of the present invention; wherein, Figure 5 In the image, 501, 502, and 503 represent pseudo-color displays of the target brightness image, where 5010 represents the dirty area, 5020 represents the non-dirty area, and 5030 represents the dirty boundary.

[0065] This invention provides a method for identifying lens dirt. The method includes: acquiring a source image captured by the lens, the source image including at least multiple color channel values ​​corresponding to each pixel in the source image; performing brightness conversion processing based on each color channel value to obtain a first brightness image corresponding to the source image; fitting the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; performing vignetting removal processing based on the first brightness image and the second brightness image to obtain a target brightness image; and performing dirt detection on the target brightness image to identify whether the lens is dirty. This method achieves the goal of obtaining a vignetted-removed target brightness image using the first brightness image after brightness conversion and the fitted second brightness image, and identifies lens dirt while removing vignetting, resulting in more accurate and effective results.

[0066] See Figure 6 , Figure 6 This is another flowchart of a lens dirt identification method according to an embodiment of the present invention. The lens dirt identification method specifically includes the following steps:

[0067] 601. Obtain a source image captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image;

[0068] 602. Perform brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image;

[0069] It should be noted that steps 601 and 602 are related to... Figure 1 Steps 101 and 102 are similar and will not be repeated here to avoid repetition. Please refer to the aforementioned steps 101 and 102 for details.

[0070] In one feasible implementation, step 602 may include steps A and B:

[0071] A. Calculate the average value of each pixel by using the color channel values ​​corresponding to each pixel and the number of color channels.

[0072] B. Generate the first brightness image using the brightness values ​​of each pixel.

[0073] For example, the first brightness image can be obtained through the brightness conversion process shown in steps A and B. Specifically, by obtaining the average value of each color channel corresponding to each pixel in the source image and the number of color channels, the average value is used as the brightness value of each pixel, and the brightness value of each pixel can be obtained; then the first brightness image is generated using the brightness values ​​of each pixel. Taking a source image where each pixel has four color channels (R, Gr, Gb, B) as an example, where pixel P1 is (125, 126, 127, 128), pixel P2 is (127, 126, 129, 128)... and pixel Pn is (135, 126, 127, 128); therefore, the brightness value L1 corresponding to pixel P1 is (125+126+127+128) / 4 = 126.5; the brightness value L2 corresponding to pixel P2 is (127+126+129+128) / 4 = 127.5;...; the brightness value Ln corresponding to pixel Pn is (135+126+127+128) / ... 4 = 129; Then, the brightness values ​​of each pixel are used to generate the first brightness image. The method of generating the first brightness image is not limited. It can be that the brightness value is used as the pixel value for each pixel to generate a new image of the same size, and this new image is used as the first brightness image; or the four color channel values ​​of the same pixel in the source image are replaced with the brightness value to obtain the first brightness image. For example, if pixel P1 is (125, 126, 127, 128), the brightness value L1 corresponding to pixel P1 is 126.5; therefore, in the first brightness image, pixel P1 is (126.5, 126.5, 126.5, 126.5). This example is not limited.

[0074] Furthermore, in order to reduce the amount of computation and ensure the accuracy of the calculation, the first brightness image can be preprocessed to facilitate subsequent calculations. For example, after step 602, the following steps can be performed: image size transformation processing of the first brightness image according to a preset size to obtain the transformed first brightness image; and Gaussian filtering of the transformed first brightness image to obtain the filtered first brightness image.

[0075] It should be noted that the computational load of the first brightness image can be reduced by dimensionality reduction. Specifically, this dimensionality reduction can be achieved through image resizing. The first brightness image can be reduced in size using the `resize()` or `CVresize()` functions in open-source computer vision (OPENCV), compressing it to a preset size. Further Gaussian filtering is then applied to the transformed first brightness image to obtain the filtered first brightness image, which is the final first brightness image. Using this processed first brightness image for subsequent calculations reduces computational load and improves processing efficiency. The `resize()` or `cvResize()` functions are used to resize the first brightness image to the preset size. If a region of interest (ROI) is defined, the function will support ROI as usual. Dimensionality reduction and compression of the first brightness image reduce resolution, thereby reducing computational load and improving detection efficiency.

[0076] 603. Use a fitting algorithm to fit the first brightness image to obtain a second brightness image corresponding to the first brightness image;

[0077] It should be noted that step 603 and Figure 1 The content of step 103 shown is similar, and will not be repeated here to avoid repetition. For details, please refer to the content of step 103 mentioned above.

[0078] In one feasible implementation, vignetting is performed based on the first brightness image and the second brightness image to obtain the target brightness image, which may specifically include the following steps 604-606:

[0079] 604. Obtain the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image;

[0080] 605. The difference between the first brightness value and the second brightness value of the same pixel is used to obtain the brightness difference value of each pixel;

[0081] Specifically, by obtaining the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image, a subtraction operation is performed on the first brightness value and the second brightness value of the same pixel to obtain the brightness difference value corresponding to that pixel. The above subtraction operation is performed on all pixels to obtain the brightness difference value of each pixel.

[0082] 606. By comparing the brightness difference of the same pixel with the first brightness value, the brightness ratio of each pixel is obtained, and the brightness ratio is used as the third brightness value of each pixel to obtain the target brightness image generated by the third brightness value of each pixel.

[0083] Furthermore, the brightness difference of the same pixel is compared with the first brightness value to obtain the brightness ratio of each pixel. This brightness ratio is recorded as the third brightness value of each pixel, and then a target brightness image with vignetting removed is generated from the third brightness value. The target brightness image has the same size as the first brightness image and the second brightness image. The specific generation method can also refer to the aforementioned generation method of the second brightness image, which will not be elaborated here.

[0084] For example, the third luminance value of the target luminance image can be obtained by referring to the following formula:

[0085]

[0086] In the formula, Pn represents the nth pixel, n is a positive integer, imgSrc(Pn) is the first brightness value of pixel Pn, and imgBspline(Pn) is the second brightness value of pixel Pn, wherein the second brightness value is obtained by fitting a surface to the B-spline of the first brightness image.

[0087] Furthermore, a method coefficient β can be set for the obtained brightness difference to make the brightness difference more obvious and easier to calculate.

[0088] 607. Perform dirt detection on the target brightness image to identify whether the lens is dirty.

[0089] It should be noted that step 607 and Figure 1 The content of step 104 shown is similar, so it will not be repeated here to avoid repetition. For details, please refer to the content of step 607 mentioned above.

[0090] In one feasible implementation, step 607 may include steps a, b, and c:

[0091] a. Obtain a first target pixel in the target brightness image, wherein the pixel value of the first target pixel is greater than a preset brightness threshold corresponding to the first target pixel;

[0092] b. If the first target pixel is obtained, the first target pixel is used as a dirt point to determine that the lens is dirty;

[0093] c. If the first target pixel is not obtained, then the lens is determined to be clean.

[0094] It should be noted that the dirt spot is a faint black patch. The pixel value of this black patch differs from that of ordinary pixel areas. Furthermore, due to the characteristics of lens imaging, the center is brighter, while the brightness decreases towards the edges. To obtain the first target pixel corresponding to this black patch in the target brightness image, a preset brightness threshold can be set for each pixel. The pixel value of each pixel in the target brightness image is compared with the preset brightness threshold, and pixels with values ​​greater than the threshold are identified as the first target pixel. When this first target pixel is found, it is considered a dirt spot, and the lens is determined to be dirty if it is present. Conversely, if no first target pixel is found, the lens is considered clean. Setting different brightness thresholds for different pixel areas can improve the accuracy of dirt identification. The brightness threshold is related to the lens area and the number of pixels. The specific threshold setting can be adjusted and determined through batch image testing. For example, the preset brightness threshold can be 0; when the pixel value is greater than 0, the pixel is identified as the first target pixel. The first target pixel is the pixel corresponding to the area where the dirt is located. For details, please refer to... Figure 5 The pseudo-color display effect of the target brightness image in Figure 501 is shown in Figure 5010. In Figure 5010, a dirty area is formed by multiple first target pixels. It can be understood that dirt can be a single point or a region. Therefore, if the lens is dirty, at least one of the first target pixels will exist.

[0095] In one feasible implementation, step 607 may further include steps a1, b1, and c1:

[0096] a1. Obtain a second target pixel in the target brightness image, wherein the gradient value of the second target pixel is greater than a preset brightness change rate threshold corresponding to the second target pixel, and the gradient value is the brightness change rate between adjacent pixels;

[0097] b1. If a second target pixel is obtained, the second target pixel is used as a dirt point to determine that the lens is dirty;

[0098] c1. If the second target pixel is not obtained, it is determined that the lens is not dirty.

[0099] It should be noted that the gradient value of each pixel in the target brightness image can also be calculated. The gradient value refers to the rate of change of a pixel in the x and y directions, specifically obtained by comparing it with neighboring pixels. The gradient value is a two-dimensional vector composed of two components: the change along the x-axis and the change along the y-axis. Since dirt is a black block, it has a boundary with other pixel areas. By calculating the rate of change of each pixel value, the point where the pixel value changes abruptly can be identified. This pixel can then be marked as the boundary point of the dirt, i.e., the second target pixel. Furthermore, by setting a preset brightness change rate threshold, pixels in the target brightness image with a value greater than this threshold are designated as second target pixels, thus confirming the lens dirt. If no second target pixel exists, the lens is not dirty. It is understood that different pixel areas have different color change characteristics; therefore, the preset brightness change rate threshold can be configured for different pixel areas or different pixels. For more information, refer to... Figure 5 Figure 503 shows the pseudo-color display effect of the target brightness image. Figure 5 5030 in the image represents the dirty boundary formed by multiple second target pixels; therefore, there is more than one second target pixel.

[0100] Optionally, steps a, b, and c, as well as a1, b1, and c1, can all determine whether the lens is dirty. Steps a, b, and c obtain dirt points based on pixel values, and these dirt points can constitute a dirty area. Steps a1, b1, and c1 obtain dirt points based on the brightness change rate of pixels, and these dirt points can constitute dirt boundaries. Therefore, steps a, b, and c, as well as a1, b1, and c1, can be executed separately, or steps a, b, and c can be executed first, and then steps a1, b1, and c1 can be used to determine the boundaries of the dirt points identified in steps a, b, and c, thus improving processing efficiency. This example does not limit the scope of the analysis.

[0101] Furthermore, the dirt can be further processed by obtaining the coordinates and size of each dirt point, thereby distinguishing different dirty areas. See steps i, ii, and iii below for details:

[0102] i. Perform morphological processing on the aforementioned dirt spots to obtain candidate dirt regions;

[0103] If both a first target pixel and a second target pixel exist, it indicates that the lens is dirty. This dirty point can be further processed to obtain its shape and size. Morphological processing of the dirty point yields candidate dirty regions. This morphological processing includes, but is not limited to, edge extraction morphological algorithms, connected component morphological algorithms, convex hull morphological algorithms, and region skeleton morphological algorithms. The candidate dirty region is a connected region obtained after morphological processing, which can be used as a candidate region for determining the size and type of dirty area.

[0104] ii. If the candidate dirty area is larger than the preset dirty area threshold, then the candidate dirty area is taken as the target dirty area;

[0105] iii. Output the dirt information of the target dirty area, the dirt information including at least the coordinates and size of the target dirty area.

[0106] Furthermore, if the candidate dirty area is larger than a preset dirty area threshold, the candidate dirty area is taken as the target dirty area, and the dirty information of the target dirty area is output. The dirty information includes the coordinates of the target dirty area and the size of the target dirty area. The coordinates of the target dirty area are obtained from the coordinates of each pixel in the connected region, and the size of the target dirty area is the size of the target dirty area, which can be obtained from the area of ​​the connected region.

[0107] This invention provides a method for identifying lens dirt. The method includes: acquiring a source image captured by the lens, the source image including at least multiple color channel values ​​corresponding to each pixel in the source image; performing brightness conversion processing based on each color channel value to obtain a first brightness image corresponding to the source image; fitting the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; acquiring the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image; subtracting the first brightness value from the second brightness value of the same pixel to obtain a brightness difference value for each pixel; comparing the brightness difference value of the same pixel with the first brightness value to obtain a brightness ratio value for each pixel, using the brightness ratio value as a third brightness value for each pixel to obtain a target brightness image generated from the third brightness values ​​of each pixel; and performing dirt detection on the target brightness image to identify whether the lens is dirty. This method uses the brightness ratio value obtained by comparing the brightness difference value with the first brightness value to remove vignetting, obtaining a target brightness image generated from the brightness ratio value. By removing vignetting, it identifies whether the lens is dirty, resulting in a more accurate and effective dirt identification. Furthermore, it improves the accuracy of dirt detection by employing multiple methods for identifying dirt spots.

[0108] Please refer to Figure 7 , Figure 7 A structural block diagram of a lens dirt detection device is provided. The lens dirt detection device specifically includes:

[0109] Data acquisition module 701: used to acquire the source image captured by the lens, the source image including at least multiple color channel values ​​corresponding to each pixel in the source image;

[0110] Brightness conversion module 702: used to perform brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image;

[0111] Fitting processing module 703: used to fit the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image;

[0112] Vignette removal module 704: used to perform vignetting removal processing based on the first brightness image and the second brightness image to obtain the target brightness image;

[0113] Dirt detection module 705: used to detect dirt in the target brightness image and identify whether the lens is dirty.

[0114] It should be noted that, Figure 7 The function of each module in the device shown is as follows: Figure 1 The steps in the method shown are similar, and will not be repeated here to avoid repetition. Please refer to the previous sections for details. Figure 1 The content of each step in the method shown.

[0115] This invention provides a lens dirt identification device, comprising: a data acquisition module for acquiring a source image captured by the lens, the source image including at least multiple color channel values ​​corresponding to each pixel in the source image; a brightness conversion module for performing brightness conversion processing based on each color channel value to obtain a first brightness image corresponding to the source image; a fitting processing module for fitting the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; a vignetting removal module for performing vignetting removal processing based on the first brightness image and the second brightness image to obtain a target brightness image; and a dirt identification module for detecting dirt in the target brightness image to identify whether the lens is dirty. This method achieves the goal of obtaining a vignetted-free target brightness image using the first brightness image after brightness conversion and the fitted second brightness image, and identifies whether the lens is dirty while removing the vignetting, resulting in more accurate and effective results.

[0116] Figure 8 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 8As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program, which, when executed by the processor, causes the processor to perform the aforementioned methods. The internal memory may also store a computer program, which, when executed by the processor, causes the processor to perform the aforementioned methods. Those skilled in the art will understand that… Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform actions such as... Figure 1 or Figure 6 The steps shown in either method.

[0118] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following actions: Figure 1 or Figure 6 The steps shown in either method.

[0119] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for identifying lens dirt, characterized in that, The method includes: Acquire a source image captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image; A brightness conversion process is performed based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image; The first brightness image is fitted using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; The target brightness image is obtained by performing vignetting removal processing based on the first brightness image and the second brightness image; Dirt detection is performed on the target brightness image to identify whether the lens is dirty; The step of performing vignetting removal based on the first brightness image and the second brightness image to obtain the target brightness image includes: Obtain the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image; The brightness difference between each pixel is obtained by subtracting the first brightness value from the second brightness value of the same pixel. By comparing the brightness difference of the same pixel with the first brightness value, the brightness ratio of each pixel is obtained, and the brightness ratio is used as the third brightness value of each pixel to obtain the target brightness image generated by the third brightness value of each pixel. Wherein, the third brightness value of the target brightness image Refer to the following formula: In the formula, This represents the nth pixel, where n is a positive integer. For pixels The first brightness value, For pixels The second brightness value is obtained by fitting a surface to the B-spline of the first brightness image.

2. The method according to claim 1, characterized in that, The step of performing brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image includes: The brightness value of each pixel is obtained by averaging the values ​​of each color channel corresponding to each pixel and the number of color channels. The first brightness image is generated using the brightness values ​​of each pixel.

3. The method according to claim 1, characterized in that, The step of performing dirt detection on the target brightness image to identify whether the lens is dirty includes: Obtain a first target pixel in the target brightness image, wherein the pixel value of the first target pixel is greater than a preset brightness threshold corresponding to the first target pixel; If the first target pixel is obtained, the first target pixel is used as a dirt point to determine that the lens is dirty; If the first target pixel is not obtained, it is determined that the lens is not dirty.

4. The method according to claim 1, characterized in that, The step of performing dirt detection on the target brightness image to identify whether the lens is dirty also includes: A second target pixel in the target brightness image is obtained, wherein the gradient value of the second target pixel is greater than a preset brightness change rate threshold corresponding to the second target pixel, and the gradient value is the brightness change rate between adjacent pixels. If a second target pixel is obtained, the second target pixel is used as a dirt spot to determine that the lens is dirty; If the second target pixel is not obtained, it is determined that the lens is not dirty.

5. The method according to claim 3 or 4, characterized in that, The method further includes: The aforementioned dirt spots are subjected to morphological processing to obtain candidate dirt regions; If the candidate dirty area is larger than the preset dirty area threshold, then the candidate dirty area is taken as the target dirty area. Output the dirt information of the target dirty area, the dirt information including at least the coordinates and size of the target dirty area.

6. The method according to claim 1, characterized in that, The step of performing brightness conversion processing based on each of the color channel values ​​to obtain a first brightness image corresponding to the source image further includes: The first brightness image is resized according to a preset size to obtain the resized first brightness image. Gaussian filtering is applied to the transformed first brightness image to obtain the filtered first brightness image.

7. A lens dirt identification device, characterized in that, The device includes: Data acquisition module: used to acquire source images captured by the lens, wherein the source image includes at least multiple color channel values ​​corresponding to each pixel in the source image; Brightness conversion module: used to perform brightness conversion processing based on the values ​​of each color channel to obtain a first brightness image corresponding to the source image; Fitting processing module: used to fit the first brightness image using a fitting algorithm to obtain a second brightness image corresponding to the first brightness image; Vignette removal module: used to perform vignetting removal processing based on the first brightness image and the second brightness image to obtain the target brightness image; Dirt detection module: used to detect dirt in the target brightness image and identify whether the lens is dirty; The vignetting removal module is specifically used to: obtain the first brightness value of each pixel in the first brightness image and the second brightness value of each pixel in the second brightness image; obtain the brightness difference value of each pixel by subtracting the first brightness value and the second brightness value of the same pixel; obtain the brightness ratio value of each pixel by comparing the brightness difference value of the same pixel with the first brightness value; and use the brightness ratio value as the third brightness value of each pixel to obtain the target brightness image generated by the third brightness value of each pixel. Wherein, the third brightness value of the target brightness image Refer to the following formula: In the formula, This represents the nth pixel, where n is a positive integer. For pixels The first brightness value, For pixels The second brightness value is obtained by fitting a surface to the B-spline of the first brightness image.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor performs the steps of the method as described in any one of claims 1 to 6.

9. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images