Lens module appearance detection method and system based on SSD algorithm

Abstract

The present invention belongs to the field of target detection technology, and specifically relates to a lens module appearance detection method and system based on an SSD algorithm, the method comprising: step one, obtaining a sample image of the lens module, and setting feature information and category labels of the image to obtain a training set; step two, building an original SSD algorithm model; step three, optimizing the original SSD algorithm model to obtain a detection algorithm model; step four, training the detection algorithm model with the training set; step five, using the trained detection algorithm model to identify and detect a packaged lens module. The present application can realize automatic detection of lens module damage while retaining the advantages of the SSD algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a lens module appearance detection method and system based on an SSD algorithm. Background Art

[0002] Against the backdrop of economic development and rising consumer spending, smart products are becoming more and more popular, and smartphones are developing rapidly, which has led to the large-scale production of mobile phone lens modules. The lens module industry is showing a good development trend. The camera of a smartphone is mainly composed of components such as an optical lens, a voice coil motor, an infrared cutoff filter, and an image sensor. During the packaging process, the lens module is often damaged due to bumps during transmission or human factors. At present, after the lens module is assembled, the staff will screen for damage by naked eye observation. However, in large-scale production, it is easy for staff to suffer from visual fatigue when conducting high-intensity screening, and it is easy to miss or over-screen, and the efficiency is low.

[0003] On the other hand, in recent years, the feature extraction method of input images by deep learning neural networks has developed rapidly, greatly improving the development of detection technologies such as target positioning, classification, and recognition. Many computer vision scholars have begun to study various target detection algorithms and have created many representative target detection algorithms. At present, there are two main mainstream frameworks for target detection, namely Two Stage represented by Faster RCNN and Single Shot represented by YOLO / SSD. Among them, Faster RCNN has a higher detection mAp and a lower miss detection rate recall, but a slower detection speed; while YOLO is the opposite, with a faster speed but lower accuracy and miss detection rate. SSD balances the advantages and disadvantages of YOLO and Faster RCNN. SSD inherited the idea of ​​converting detection into regression from YOLO, completing the location and classification of the target at one time, saving a lot of training time; based on the Anchor idea in Faster RCNN, it proposed the Prior box method to improve the accuracy of target detection; YOLO directly connects the fully connected layer after the convolution layer, that is, only the highest-level feature maps are used when detecting targets. In addition, SSD adds a detection method based on feature pyramids, that is, to detect targets on feature maps with different receptive fields. SSD uses feature maps with small receptive fields to detect small targets, and uses feature maps with large receptive fields to detect larger targets. Based on the above characteristics, the SSD algorithm has both detection speed and detection accuracy.

[0004] The SSD algorithm is undoubtedly an ideal target detection algorithm model due to its high detection speed and high detection accuracy. However, the SSD algorithm can only be used to identify relatively large targets, while the size of the lens module is very small, so the SSD algorithm cannot be used to detect damaged points on the lens module. In the currently commonly used detection method based on the SSD algorithm, a lens module can only be regarded as one or more detection points in the detection method, and it is impossible to accurately detect all positions on the outer surface of the lens module. Other algorithm models that can detect small-sized targets are either too expensive or too inaccurate, and are not very suitable for lens module damage detection. Therefore, the current damage detection of mobile phone lens modules is still basically performed manually. Summary of the invention

[0005] The purpose of the present invention is to provide a lens module appearance detection method based on the SSD algorithm, which can realize automatic detection of lens module damage while retaining the advantages of the SSD algorithm.

[0006] The basic scheme provided by the present invention is:

[0007] The lens module appearance detection method based on the SSD algorithm includes:

[0008] Step 1: Obtain sample images of the lens module, and set feature information and category labels of the images to obtain a training set;

[0009] Step 2: Build the original SSD algorithm model;

[0010] Step 3: Optimize the original SSD algorithm model to obtain a detection algorithm model, wherein the optimization includes:

[0011] S3.1, optimize and enlarge the scale of the original SSD algorithm model and the size of the prior box;

[0012] S3.2, change the FC7 and FC6 layers in the VGG16 backbone network to convolutional layers and use them for subsampling parameters, change Pool5 from 2×2-S2 to 3×3-S1, where S1 and S2 represent step sizes of 1 and 2 respectively; use dilated convolution to fill the missing parts and all dropout layers, and remove the FC8 layer;

[0013] S3.3, extract the third convolution features of the fourth convolution layer and combine them with the features extracted by the FC7 layer, and then replace the 19×19×1024, 10×10×512, and 5×5×256 convolution layers of the optimized and enlarged SSD algorithm model with the RFB module;

[0014] Step 4: Use the training set to train the detection algorithm model;

[0015] Step 5: Use the trained detection algorithm model to identify and detect the packaged lens module.

[0016] Basic solution working principle and beneficial effects:

[0017] In the field of machine vision, when using the SSD algorithm for detection, a lens module is considered as one or more detection points, and it is impossible to accurately detect all positions on the outer surface of the lens module. Therefore, when those skilled in the art study the recognition algorithm of lens module damage, they automatically filter out the SSD algorithm and turn to study how to reduce the cost of algorithm models that can accurately identify small-sized targets. However, the actual effect of such technical ideas is not particularly ideal.

[0018] The applicant has broken away from this technical thinking and improved the SSD algorithm model that the technicians in the field assumed could not detect small-sized targets, so that it can identify damage on the lens module. This not only overcomes the technical bias of the technicians in the field, but also requires very little cost and retains the characteristics of the SSD algorithm in terms of both detection speed and detection accuracy.

[0019] In this application, the detection algorithm model is obtained by optimizing the original SSD algorithm model. Specifically, the scale of the original SSD algorithm model and the size of the prior frame are optimized and enlarged, the input image size is improved on the backbone network of the original SSD algorithm model, and the original input image is optimized and enlarged, which can improve the detection efficiency and accuracy of the SSD algorithm model for small target objects to a certain extent; and the third convolution feature of the fourth convolution layer is combined with the feature extracted by the FC7 layer, and the accuracy of extracting the features of small targets can be improved by merging the matrices of these two features; in addition, the RFB module (convolution receptive field module) is added to improve the SSD algorithm, improve the feature extraction capability of the SSD algorithm backbone network VGG16, and improve the detection efficiency and detection speed of the original SSD target detection algorithm, so that the detection algorithm model is more accurate and faster. The detection algorithm model obtained after the above optimization has the ability to identify damage on the lens module.

[0020] After that, the detection algorithm model is trained with the pre-prepared training set before being put into use. In this way, when the appearance of the lens module is detected, the captured lens module image is directly identified with the detection algorithm model to understand whether the corresponding lens module is qualified. It is not only efficient, but also greatly reduces labor costs and error rates, and is very suitable for appearance inspection in factories that mass-produce lens modules. Moreover, since this application only improves the existing SSD algorithm, the model cost is low and easy to implement, and the detection efficiency and accuracy of the SSD algorithm are retained, and the effect of use is also excellent.

[0021] In summary, this method can realize automatic detection of lens module damage while retaining the advantages of the SSD algorithm.

[0022] Further, step one includes:

[0023] S1.1, acquiring an image of the lens module;

[0024] S1.2, processing the image of the lens module to expand the image data set;

[0025] S1.3, marking the damaged parts of the lens module on the images in the dataset as feature information;

[0026] S1.4, converting the dataset into a unified dataset format;

[0027] S1.5, classify the images in a unified format and set type labels to obtain a training set, where the type labels include pass and error.

[0028] Beneficial effect: This training set preparation method is simple and easy, and a training set suitable for the detection algorithm model can be quickly obtained.

[0029] Furthermore, in S1.2, the image processing methods include scaling, cropping, stretching and selective interception.

[0030] Beneficial effects: In this way, the acquired image material can be utilized in a variety of ways to expand the number of images in the data set.

[0031] Further, step two includes:

[0032] S2.1, build a convolutional neural network model;

[0033] S2.2, construct an SSD300×300×3 algorithm model as the original SSD algorithm model.

[0034] Beneficial effect: SSD300×300×3 is the most basic SSD algorithm model. Compared with building other SSD algorithm models, it can reduce the basic model construction time before algorithm optimization.

[0035] Further, step five includes:

[0036] S5.1, calling the camera unit to obtain the detection image of the lens module, and obtaining each frame of the detection image as the image to be tested;

[0037] S5.2, putting the image to be tested into the trained detection algorithm model for recognition detection;

[0038] S5.3, obtaining the results of the module appearance inspection, and marking the modules with damaged modules as detected.

[0039] Beneficial effect: In this way, the detection image obtained by the camera unit is directly processed, and the lens module is detected, with a high degree of automation.

[0040] Furthermore, in S5.1, the camera unit is called using opencv.

[0041] Beneficial effect: It can further improve the automation of the overall process.

[0042] Furthermore, in S3.1, the scale of the original SSD algorithm model and the size of the prior box are adjusted to 512×512×3.

[0043] Beneficial effect: Such optimized size can achieve the best effect of damage detection on the lens module.

[0044] The present invention also provides a lens module appearance detection system based on the SSD algorithm, which is applied to the above-mentioned lens module appearance detection method based on the SSD algorithm, and includes a camera unit, a storage unit and a processing unit;

[0045] The storage unit stores a detection algorithm model, which is obtained by optimizing the basic SSD algorithm model. The processing unit is used to call the camera unit to obtain the detection image of the lens module, and obtain each frame of the detection image as the image to be tested. The processing unit is also used to call the detection algorithm model to detect the image to be tested, obtain the detection result, and mark the module with a detection result showing damage.

[0046] Beneficial effects: By using this system, the damage of the lens module can be automatically detected in an easy-to-implement way. When the appearance of the lens module is detected, the captured image can be directly identified by the detection algorithm model to understand whether the corresponding lens module is qualified. In this way, it is not only efficient, but also greatly reduces labor costs and error rates. It is very suitable for appearance detection in factories that mass-produce lens modules.

[0047] Furthermore, the optimization process of the detection algorithm model includes:

[0048] Optimize and enlarge the scale of the original SSD algorithm model and the size of the prior box;

[0049] The FC7 and FC6 layers in the VGG16 backbone network are changed to convolutional layers and used for subsampling parameters. Pool5 is changed from 2×2-S2 to 3×3-S1, where S1 and S2 represent step sizes of 1 and 2 respectively. Dilated convolution is used to fill the missing parts and all dropout layers, and the FC8 layer is removed.

[0050] The third convolution features of the fourth convolutional layer are extracted and combined with the features extracted from the FC7 layer, and then the 19×19×1024, 10×10×512, and 5×5×256 convolutional layers of the optimized and enlarged SSD algorithm model are replaced with the RFB module.

[0051] Beneficial effects: The scale of the original SSD algorithm model and the size of the prior frame are optimized and enlarged, the input image size is improved on the backbone network of the original SSD algorithm model, and the original input image is optimized and enlarged, which improves the detection efficiency and accuracy of the SSD algorithm model for small target objects to a certain extent; and the third convolution feature of the fourth convolution layer is combined with the feature extracted by the FC7 layer. By merging the matrix of these two features, the accuracy of extracting the features of small targets can be improved; in addition, the RFB module (convolution receptive field module) is added to improve the SSD algorithm, improve the feature extraction ability of the SSD algorithm backbone network VGG16, and improve the detection efficiency and detection speed of the original SSD target detection algorithm, so that the detection algorithm model is more accurate and faster. The detection algorithm model obtained after the above optimization has the ability to identify damage on the lens module. In addition, since this application only improves the existing SSD algorithm, the model cost is low and easy to implement; in addition, the improvement method of this application retains the detection efficiency and accuracy of the SSD algorithm, and the effect of use is also very good.

[0052] Furthermore, it also includes a display unit for displaying the detection results.

[0053] Beneficial effect: It is convenient for staff to understand the test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of Embodiment 1 of the present invention;

[0055] Figure 2 This is a logic block diagram of the first embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following is a further detailed description through specific implementation methods:

[0057] Embodiment 1

[0058] In the field of machine vision, when using the SSD algorithm for detection, a lens module is considered as one or more detection points, and it is impossible to accurately detect all positions on the outer surface of the lens module. Therefore, when technical personnel in this field study the recognition algorithm of lens module damage, they automatically filter out the SSD algorithm and turn to study how to reduce the cost of algorithm models that can accurately identify small-sized targets. However, the progress of such technical ideas is not particularly ideal.

[0059] The applicant has jumped out of such a technical idea, and improved the SSD algorithm model that the technicians in the field assumed could not detect small-sized targets. By improving the input image size on the backbone network of the original SSD algorithm model, the original input image is optimized and enlarged, which improves the detection efficiency and accuracy of the SSD algorithm model for small target objects to a certain extent; and the third convolution feature of the fourth convolution layer is combined with the feature extracted by the FC7 layer. By merging the matrix of these two features, the accuracy of extracting the features of small targets can be improved; in addition, the RFB module (convolution receptive field module) is added to improve the SSD algorithm, which improves the feature extraction capability of the SSD algorithm backbone network VGG16, and improves the detection efficiency and detection speed of the original SSD target detection algorithm, thereby making the detection algorithm model more accurate and faster. The detection algorithm model obtained in this way has the ability to identify damage on the lens module.

[0060] This solution not only overcomes the technical bias of technicians in this field that "the SSD algorithm is automatically ignored when identifying small-sized targets", but also requires very little cost and retains the detection efficiency and accuracy of the SSD algorithm. The details are as follows:

[0061] like Figure 1 As shown, the lens module appearance detection method based on the SSD algorithm includes:

[0062] Step 1: Obtain sample images of the lens module and set the feature information and category labels of the images to obtain a training set. Specifically, it includes:

[0063] S1.1, acquiring an image of the lens module;

[0064] S1.2, processing the image of the lens module to expand the image data set; wherein the image processing method includes scaling, cropping, stretching and selective interception. In this way, the acquired image material can be used in a variety of ways to expand the number of images in the data set.

[0065] S1.3, marking the damaged parts of the lens module on the images in the dataset as feature information;

[0066] S1.4, converting the data set into a unified data set format; in this embodiment, the converted unified format is VOC2007.

[0067] S1.5, classify the images in a unified format and set type labels to obtain a training set, wherein the type labels include pass and error. That is, the images are divided into two types: passed detection and failed detection.

[0068] This method of preparing a training set is simple and easy, and a training set suitable for the detection algorithm model can be quickly obtained.

[0069] Step 2: Build the original SSD algorithm model. Specifically include:

[0070] S2.1, build a convolutional neural network model;

[0071] S2.2, construct an SSD300×300×3 algorithm model as the original SSD algorithm model.

[0072] SSD300×300×3 is the most basic SSD algorithm model. Compared with building other SSD algorithm models, it can reduce the basic model construction time before algorithm optimization.

[0073] Step 3: Optimize the original SSD algorithm model to obtain a detection algorithm model, including:

[0074] S3.1, optimize and enlarge the scale of the original SSD algorithm model and the size of the prior box;

[0075] S3.2, change the FC7 and FC6 layers in the VGG16 backbone network to convolutional layers and use them for subsampling parameters, change Pool5 from 2×2-S2 to 3×3-S1, where S1 and S2 represent step sizes of 1 and 2 respectively; use dilated convolution to fill the missing parts and all dropout layers, and remove the FC8 layer;

[0076] S3.3, extract the third convolution features of the fourth convolution layer and combine them with the features extracted by the FC7 layer, and then replace the 19×19×1024, 10×10×512, and 5×5×256 convolution layers of the optimized and enlarged SSD algorithm model with the RFB module;

[0077] Step 4: Use the training set to train the detection algorithm model.

[0078] Step 5: Use the trained detection algorithm model to identify and detect the packaged lens module. Specifically include:

[0079] S5.1, using opencv to call the camera unit to obtain the detection image of the lens module, and obtain each frame of the detection image as the image to be tested;

[0080] S5.2, putting the image to be tested into the trained detection algorithm model for recognition detection;

[0081] S5.3, obtaining the results of the module appearance inspection, and marking the modules with damaged modules as detected.

[0082] In this way, the detection image obtained by the camera unit is directly processed and the lens module is detected, with a high degree of automation.

[0083] The present application also provides a lens module appearance detection system based on the SSD algorithm, which is applied to the lens module appearance detection method based on the SSD algorithm, such as Figure 2 As shown, it includes a camera unit, a storage unit, a processing unit and a display unit. In this embodiment, the camera unit is a camera, and the storage unit, the processing unit and the display unit are integrated on an industrial PC. The installation prevention and installation position of the camera unit can be specifically set by those skilled in the art according to the transportation route during the lens module detection.

[0084] The storage unit stores the detection algorithm model, which is obtained by optimizing the basic SSD algorithm model. Specifically, the optimization process of the detection algorithm model includes: optimizing and enlarging the scale of the original SSD algorithm model and the size of the prior box; changing the FC7 layer and FC6 layer in the VGG16 backbone network to convolutional layers and using them for subsampling parameters, changing Pool5 from 2×2-S2 to 3×3-S1, where S1 and S2 represent step sizes of 1 and 2 respectively; using dilated convolution to fill the missing parts and all dropout layers, and removing the FC8 layer; extracting the third convolution feature of the fourth convolutional layer and combining it with the feature extracted from the FC7 layer, and then replacing the 19×19×1024, 10×10×512, and 5×5×256 convolutional layers of the optimized and enlarged SSD algorithm model with the RFB module.

[0085] The processing unit is used to call the camera unit to obtain the detection image of the lens module, and obtain each frame of the detection image as the image to be tested; the processing unit is also used to call the detection algorithm model to detect the image to be tested, obtain the detection result, and mark the module with a detection result showing that it is damaged; the display unit is used to display the detection result.

[0086] The specific implementation process is as follows:

[0087] Using this solution, you can first collect images of the lens module through the Internet and your own photography; then, process the images of the lens module through scaling, cropping, stretching, and selective interception to expand the image data set. Then, use the Labelimg annotation tool to annotate the images in the data set and mark the damaged areas; and standardize the data set, convert all of them into the VOC 2007 data set format, and classify them, dividing the images in the data set into pass and error.

[0088] On the other hand, when building the detection algorithm model, the convolutional neural network model is first built, and the SSD 300×300×3 algorithm model is constructed on the basis of the convolutional neural network model as the original SSD algorithm model. Compared with building other SSD algorithm models, SSD300×300×3 is the most basic SSD algorithm model, which can reduce the basic model construction time before algorithm optimization. The SSD algorithm is a regression-based target detection algorithm, and the detection and classification of targets in the regression-based target detection algorithm are performed simultaneously. The main principle is to use the convolutional neural network to extract features from the input image, and then uniformly and randomly perform dense sampling at different positions of the input image. At the same time, different sizes are used for sampling. The prediction box and classification operations of the target object are performed together, so only one step is needed in this process, so its speed is very fast. However, the existing SSD algorithm models can only detect targets that are not very small in size, while the size of the lens module is very small, which does not meet the use conditions of the SSD algorithm.

[0089] In this application, the original SSD algorithm will be optimized. Specifically, by optimizing and enlarging the scale of the original SSD algorithm model and the size of the prior frame, the input image size is improved on the backbone network of the original SSD algorithm model, and the original input image is optimized and enlarged, which improves the detection efficiency and accuracy of the SSD algorithm model for small target objects to a certain extent. In this embodiment, the scale of the original SSD algorithm model and the size of the prior frame are adjusted to 512×512×3. Such an optimized size can achieve the best effect of damage detection on the lens module. And the third convolution feature of the fourth convolution layer is combined with the feature extracted by the FC7 layer. By merging the matrices of these two features, the accuracy of extracting the features of small targets can be improved; in addition, the RFB module (convolution receptive field module) is added to improve the SSD algorithm, improve the feature extraction capability of the SSD algorithm backbone network VGG16, and improve the detection efficiency and detection speed of the original SSD target detection algorithm, so that the detection algorithm model is more accurate and fast. The detection algorithm model obtained after the above optimization has the ability to identify damage on the lens module.

[0090] Then, the detection algorithm model is trained with the training set. The training process can use the existing training method, which will not be described here. Then, the trained detection algorithm model can be used to detect the damage of the lens module. Specifically, after using opencv to call the camera unit to obtain the detection image of the lens module to be detected, each frame of the detection image is used as the image to be tested; and the image to be tested is put into the trained detection algorithm model for recognition and detection; after that, the result of the module appearance detection can be obtained, and the module with the detection result of being damaged can be marked.

[0091] By using this solution, by improving the basic SSD algorithm model, it is possible to achieve automated detection of the lens module, with low implementation cost and easy implementation; in addition, the improved method of this application retains the detection efficiency and accuracy of the SSD algorithm, and the effect of use is also excellent.

[0092] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A lens module appearance detection method based on the SSD algorithm, characterized in that: include: Step 1: Obtain sample images of the lens module, and set feature information and category labels of the images to obtain a training set; Step 2: Build the original SSD algorithm model; Step 2 includes: S2.1, build a convolutional neural network model; S2.2, construct an SSD300×300×3 algorithm model as the original SSD algorithm model; Step 3: Optimize the original SSD algorithm model to obtain a detection algorithm model, wherein the optimization includes: S3.1, optimize and enlarge the scale of the original SSD algorithm model and the size of the prior box; S3.2, change the FC7 and FC6 layers in the VGG16 backbone network to convolutional layers and use them for subsampling parameters, change Pool5 from 2×2-S2 to 3×3-S1, where S1 and S2 represent step sizes of 1 and 2 respectively; use dilated convolution to fill the missing parts and all dropout layers, and remove the FC8 layer; S3.3, extract the third convolution features of the fourth convolution layer and combine them with the features extracted by the FC7 layer, and then replace the 19×19×1024, 10×10×512, and 5×5×256 convolution layers of the optimized and enlarged SSD algorithm model with the RFB module; Step 4: Use the training set to train the detection algorithm model; Step 5: Use the trained detection algorithm model to identify and detect the packaged lens module; Step five includes: S5.1, calling the camera unit to obtain the detection image of the lens module, and obtaining each frame of the detection image as the image to be tested; S5.2, putting the image to be tested into the trained detection algorithm model for recognition detection; S5.3, obtaining the results of the module appearance inspection, and marking the modules with damaged modules as detected.

2. The lens module appearance detection method based on the SSD algorithm according to claim 1, characterized in that: Step one includes: S1.1, acquiring an image of the lens module; S1.2, processing the image of the lens module to expand the image data set; S1.3, marking the damaged parts of the lens module on the images in the dataset as feature information; S1.4, converting the dataset into a unified dataset format; S1.5, classify the images in a unified format and set type labels to obtain a training set, where the type labels include pass and error.

3. The lens module appearance detection method based on the SSD algorithm according to claim 2, characterized in that: In S1.2, the image processing methods include scaling, cropping, stretching and selective interception.

4. The lens module appearance detection method based on the SSD algorithm according to claim 3, characterized in that: In S5.1, opencv is used to call the camera unit.

5. The lens module appearance detection method based on the SSD algorithm according to claim 1, characterized in that: In S3.1, the scale of the original SSD algorithm model and the size of the prior box are adjusted to 512×512×3.

6. The lens module appearance detection system based on the SSD algorithm is characterized by: The lens module appearance detection method based on the SSD algorithm applied to any one of claims 1 to 5 comprises a camera unit, a storage unit and a processing unit; The storage unit stores a detection algorithm model, which is obtained by optimizing the basic SSD algorithm model. The processing unit is used to call the camera unit to obtain the detection image of the lens module, and obtain each frame of the detection image as the image to be tested. The processing unit is also used to call the detection algorithm model to detect the image to be tested, obtain the detection result, and mark the module with a detection result showing damage.

7. The lens module appearance detection system based on the SSD algorithm according to claim 6, characterized in that: The device also includes a display unit for displaying the detection results.

Images