Lens module dirty identification method based on auto-encoder

Abstract

This invention relates to the field of camera inspection technology, and more particularly to a method for identifying dirt in lens modules based on an autoencoder. It solves the problem that manual intervention is required when acquiring lens images, making it unsuitable for automated production lines. The technical solution includes the following steps: S1: Collect several sample images of clean lenses; S2: Construct an autoencoder neural network, the neural network structure consisting of an encoder and a decoder; S3: Preprocess the input image, extracting the lens region from the image; S4: Train the autoencoder network; S5: Train the lens dirt classifier. The beneficial effects of this invention are: This invention can automatically identify whether there is dirt in the lenses of the finished module, without the need for manual intervention.

Description

Technical Field

[0001] This invention relates to the field of camera inspection technology, and in particular to a method for identifying dirt in a lens module based on a self-encoder. Background Technology

[0002] For example, patent CN103245676B, "Method for Detecting Dirt on Optical Lenses," authorized on May 20, 2015, states: "This invention discloses a method for detecting whether an optical lens in a solution has dirt adhering to it, comprising the following steps: defining an optical lens, which is divided into a peripheral area and an optical area; using an image capturing unit to focus on the peripheral area of ​​the optical lens to generate a first original image; focusing the image capturing unit on the optical area of ​​the optical lens to generate a second original image; performing image equalization on the first original image and the second original image respectively to obtain a first equalized image and a second equalized image, and then obtaining first image data and second image data through image processing; and respectively setting the grayscale value of each pixel of the first image data and the second image data to a first threshold value." In comparison, it is used to determine whether there is dirt in the edge detection area and the central detection area corresponding to the optical lens. This is described in patent CN113141462A, a method for treating dirt on camera lenses, a mobile terminal, and a computer storage medium, published on July 20, 2021. The patent states: "This application relates to a method for treating dirt on camera lenses, a mobile terminal, and a computer storage medium. The method for treating dirt on camera lenses includes: capturing a first image using a target camera according to the current shooting mode; determining whether the lens of the target camera is dirty based on the first image; if so, performing a dirt treatment operation according to the current shooting mode and / or shooting state. Through the above method, dirt on the camera lens can be detected and treated in a timely manner, improving shooting effect and efficiency."

[0003] Currently, lens modules may become contaminated with dirt during production and assembly, affecting the quality of the finished product. Conventional lens contamination detection only inspects individual lenses, making it difficult to assess contamination on finished products. Existing technologies primarily rely on traditional image processing techniques to determine lens contamination, such as CN103245676B, which analyzes contamination by calculating pixel averages; however, this method is easily affected by environmental factors. For example, CN113141462A requires manual intervention when acquiring lens images, making it unsuitable for automated production lines. Summary of the Invention

[0004] The purpose of this invention is to provide a method for identifying dirt in a lens module based on an autoencoder.

[0005] To achieve the above-mentioned objectives, the present invention adopts a technical solution that includes the following steps:

[0006] S1: Collect several sample images of clean lenses;

[0007] S2: Construct an autoencoder neural network, whose structure consists of an encoder and a decoder;

[0008] S3: Preprocess the input image and extract the lens area from the image;

[0009] S4: Autoencoder network training;

[0010] S5: Lens dirt classifier training.

[0011] Step S2 is as follows:

[0012] The input x is processed by encoder E to obtain f = E(x), and then processed by decoder D to obtain output x^' = D(f), where the encoder and decoder are symmetric structures;

[0013] Where: x is the image input to encoder E, and x^' is the image output by decoder D.

[0014] Step S3 is as follows:

[0015] S3.1: Perform an edge detection algorithm on an input image;

[0016] S3.2: Apply the Hough transform to the result of step S3.1 to find the circular region in the image. Select the region with the smallest distance from the predefined circle (x0, y0, r0) as the lens region image, and obtain the circle containing the lens as (x0, y0, r0). d ,y d ,r d );

[0017] Where (x) o y o () represents the coordinates of the center of the circle containing the lens, r o Indicates radius;

[0018] After using the circle fitting algorithm, a set S of circles is obtained, with a size of C. The set is then traversed, and for each element (x...) in the set... c ,y c ,r c ), where (x c ,y c () represents the coordinates of the center of the circle, r c Indicates radius,

[0019] (1) Initialize Δ = +∞, D = +∞

[0020] Where: Δ is used to measure the radius difference, and D is used to measure the center difference; traverse the set of fitted circles to find the circle with the smallest radius difference and center difference, that is, the circle that is closest to the predefined circle.

[0021] (2) Calculate the absolute value Δ of r0 - r c c Calculate If Δ c < Δ and D c < D, update Δ = Δ c , D c = D

[0022] D C represents the distance between the center of a circle in the set and the predefined center; traverse each circle in the set, calculate the radius difference and the distance between the centers with the known circle, and find the circle in the set with the smallest difference from the known circle;

[0023] (3) Traverse the set S, and the parameters corresponding to the minimum value calculated in step (2) are the circle where the lens is located, and the circle where the lens is located is obtained as (x d , y d , r d );

[0024] where (x<​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0034] S4.4: Repeat steps S4.1-S4.3 until the model converges and the encoder parameters E_θ are obtained.

[0035] Step S5 specifically includes:

[0036] S5.1: Randomly select M images from the sample set, input the encoder parameters E_θ obtained in step S4, and obtain the feature set {f_1,f_2,…,f_M};

[0037] Where: M is the number of images in the sample set, and f_i represents the feature of the i-th sample image;

[0038] S5.2: Use the feature set from step S5.1 as input to the one-class SVM to train the classifier OSVM. φ .

[0039] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention can automatically identify whether lenses in the finished module are dirty, without manual intervention. S1: In the initial sample collection stage, only clean sample images need to be collected, eliminating the need for dirty sample images, thus reducing the difficulty of sample collection. S3: It can automatically locate the local image of the lens in the module without manual intervention. Analyzing the local image of the lens also reduces interference from other components in the module on the recognition results. S4: It utilizes an autoencoder to extract image features, avoiding the disadvantage of poor feature descriptor performance caused by manual feature extraction. The encoder trained through data iteration can more accurately describe image features. S5: It uses a one-class SVM, requiring only one class of positive samples to train the classifier, avoiding the difficulty of collecting negative samples in practical applications. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0041] Figure 1 This is a schematic diagram of a clean lens sample of the present invention.

[0042] Figure 2 This is a schematic diagram of the neural network structure of the present invention.

[0043] Figure 3 This is a schematic diagram illustrating the deployment of the present invention.

[0044] Figure 4 This is a schematic diagram of the input image read by the present invention.

[0045] Figure 5 This is a schematic diagram of the Canny edge result of the present invention.

[0046] Figure 6 This is a schematic diagram of the lens area calculation results of the present invention.

[0047] Figure 7 This is a partial schematic diagram of the lens captured in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] Example 1

[0050] A method for dirt detection in a lens module based on an autoencoder, the technical solution of which includes the following steps:

[0051] S1: Collect several sample images of clean lenses; such as Figure 1 As shown;

[0052] S2: Construct an autoencoder neural network, the neural network structure consisting of an encoder and a decoder; the neural network structure is as follows: Figure 2 As shown;

[0053] S3: Preprocess the input image and extract the lens area from the image;

[0054] S4: Autoencoder network training;

[0055] S5: Lens dirt classifier training.

[0056] Step S2 is as follows:

[0057] The input x is processed by encoder E to obtain f = E(x), and then processed by decoder D to obtain output x^' = D(f), where the encoder and decoder are symmetric structures;

[0058] Where: x is the image input to encoder E, and x^' is the image output by decoder D.

[0059] Step S3 is as follows:

[0060] S3.1: For an input image, perform an edge detection algorithm; specifically, the Canny edge detection method.

[0061] S3.2: Apply the Hough transform to the result of step S3.1 to find the circular region in the image. Select the region with the smallest distance from the predefined circle (x0, y0, r0) as the lens region image, and obtain the circle containing the lens as (x0, y0, r0). d y d r d );

[0062] where (x o , y o ) represents the center coordinates of the circle where the lens is located, and r o represents the radius;

[0063] Through the circular fitting algorithm, a set S of circles is obtained, with the number being C. Traverse the set. For an element (x c , y c , r c ) in the set, where (x c , y c ) represents the center coordinates of the circle, and r c represents the radius.

[0064] (1) Initialize Δ = +∞, D = +∞

[0065] where: Δ is used to measure the radius difference, and D is used to measure the center difference; Traverse the set of fitted circles to find the circle with the smallest radius difference and center difference, that is, the circle closest to the predefined circle in position.

[0066] (2) Calculate the absolute value Δ of r0 - r c , and calculate c . If Δ < Δ and D c < D, update Δ = Δ c , D c = D c ; D

[0067] D C represents the distance between the center of a circle in the set and the predefined center; Traverse each circle in the set, calculate the radius difference and center distance with the known circle, and find the circle in the set with the smallest difference from the known circle.

[0068] (3) Traverse the set S. The parameters corresponding to the minimum value calculated in step (2) are the circle where the lens is located, and the circle where the lens is located is obtained as (x d , y d , r d );

[0069] where (x d , y d ) represents the center coordinates of the circle where the lens is located, and r d represents the radius;

[0070] S3.3: According to the result of step S3.2, intercept the lens area picture from the graph. The width and height of the picture are r d + d′, where d′ is a predefined parameter. Set the pixels outside the circular area in the graph to 0, and scale the width and height to W × H, where W = H

[0071] Where W is the image width and H is the image height.

[0072] Step S4 specifically includes:

[0073] S4.1: Read an input image from the sample set, and after processing in step S3, crop the image x and input it into the network to obtain the output x^';

[0074] S4.2: Calculate the reconstruction error, Loss = ||xx^'||

[0075] ‖xx^'‖ refers to the L1 loss function, and its calculation formula is:

[0076]

[0077] N is the number of training samples, x_i and x_i′ represent the value of the i-th pixel in x and x^' respectively, and P represents the pixel set;

[0078] S4.3: Use gradient descent to update the parameters of the encoder and decoder;

[0079] S4.4: Repeat steps S4.1-S4.3 until the model converges and the encoder parameters E_θ are obtained.

[0080] Step S5 specifically includes:

[0081] S5.1: Randomly select M images from the sample set, input the encoder parameters E_θ obtained in step S4, and obtain the feature set {f_1,f_2,…,f_M};

[0082] Where: M is the number of images in the sample set, and f_i represents the feature of the i-th sample image;

[0083] S5.2: Use the feature set from step S5.1 as input to the one-class SVM to train the classifier OSVM. φ .

[0084] Example 2

[0085] Based on Example 1,

[0086] Step S2: The encoder first compresses the input image from dimension W×H to 512 using a fully connected layer L1, where W and H are the width and height of the image, respectively. Then, a ReLU activation function is added to increase the non-linearity of the model and improve its expressive power. Next, a fully connected layer L2 is added to further compress the dimension to 256, and a ReLU activation function is cascaded. Then, a fully connected layer L3 is added, maintaining the output dimension at 256, and a ReLU activation function is cascaded. Next, a fully connected layer L4 is added, reducing the output dimension to 128, and a ReLU activation function is cascaded. Next, a fully connected layer L5 is added, reducing the output dimension to 128, and a ReLU activation function is cascaded. Finally, a fully connected layer L6 is added, reducing the output dimension to 64. The network model is shown in the table below.

[0087]

[0088]

[0089] Similar to the encoder, the decoder uses several fully connected layers and a ReLU activation function to restore the features output by the encoder to the image dimension. The network model is shown in the table below:

[0090] serial number operate Input Dimensions Output Dimension 1 Fully connected layer, ReLU 64 128 2 Fully connected layer, ReLU 128 128 3 Fully connected layer, ReLU 128 256 4 Fully connected layer, ReLU 256 256 5 Fully connected layer, ReLU 256 512 6 Fully connected layer 512 W×H

[0091] Deploying algorithms to devices, such as Figure 3 The memory is used to store the parameters of the trained autoencoder network model and the classifier OSVM. φ The parameters are: input is used to receive video images from the camera, the computing center performs calculations and classifications on the input images, and outputs the results. The computing center is a computing device running a Linux or Windows operating system.

[0092] Read the input image, such as Figure 4 As shown, step 3 of the training phase is called to obtain the lens region image I. roi Step S3.1 calculates the Canny edge result as follows: Figure 5 The result of calculating the circle containing the lens is as follows: Figure 6 As shown, the result marked with a white circle is the calculated lens, and the cropped partial image is as follows. Figure 7 As shown, encoder E is called. θ , and extract the partial image of the lens I roi The feature description f is obtained as input to the encoder. roi Call the classifier OSVM φ , will f roi Input the classifier, get the classification result, and output the result indicating whether the lens is dirty.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying dirt in a lens module based on an autoencoder, characterized in that, Includes the following steps: S1: Collect several sample images of clean lenses; S2: Construct an autoencoder neural network, whose structure consists of an encoder and a decoder; S3: Preprocess the input image and extract the lens area from the image; S4: Autoencoder network training; S5: Lens dirt classifier training; Step S2 is as follows: The input x is encoded by the encoder E to obtain f=E(x), and then the decoder D is used to obtain the output x^'=D(f), where the encoder and decoder are symmetric structures; Where: x is the image input to encoder E, and x^' is the image output by decoder D; Step S3 is as follows: S3.1: Perform an edge detection algorithm on an input image; S3.2: Apply the Hough transform to the result of step S3.1 to find the circular region in the graph, selecting the region that matches the predefined circle. The image with the smallest distance is used as the lens region image, and the circle containing the lens is obtained. ; Where (x) o y o (r) represents the coordinates of the center of the circle containing the lens. o Indicates radius; After using the circle fitting algorithm, a set S of circles is obtained, with a size of C. The set is then traversed, and for each element in the set... ,in This represents the coordinates of the center of the circle. Indicates radius, (1) Initialization , ; in: D is used to measure the difference in radius, and D is used to measure the difference in center. (2) Calculation absolute value ,calculate ,if and ,renew , ; D C This represents the distance between the center of a circle in the set and the center of a predefined circle. (3) Traverse set S. The parameter corresponding to the minimum value calculated in step (2) is the circle where the lens is located. The circle where the lens is located is obtained as follows: ; Where (x) d y d (r) represents the coordinates of the center of the circle containing the lens. d Indicates radius; S3.3: Based on the result of step S3.2, extract the lens area image from the image. The image width and height are... ,in As a predefined parameter, set the pixels outside the circular area in the image to 0, and scale the width and height to [the specified values]. ,in ; Where W is the image width and H is the image height; Step S4 specifically includes: S4.1: Read an input image from the sample set, and after processing in step S3, crop the image x and input it into the network to obtain the output x^'; S4.2: Calculate the reconstruction error, Loss = ||xx^'||; ‖xx^' ‖ refers to the L1 loss function, which is calculated using the following formula: ； N is the number of training samples. and Let x and x' represent the values ​​of the i-th pixels in x and x' respectively, and let P represent the set of pixels; S4.3: Use gradient descent to update the parameters of the encoder and decoder; S4.4: Repeat steps S4.1-S4.3 until the model converges and the encoder parameters E_θ are obtained; Step S5 specifically includes: S5.1: Randomly select M images from the sample set, input the encoder parameters E_θ obtained in step S4, and obtain the feature set {f_1,f_2,…,f_M}; Where: M is the number of images in the sample set, and f_i represents the feature of the i-th sample image; S5.2: Use the feature set from step S5.1 as input to a one-class SVM to train the classifier. .

Images