AI visual inspection method based on deep learning

Abstract

The invention discloses an AI visual inspection method based on deep learning. The AI visual inspection method comprises the following steps: acquiring images of different surfaces of a defective product; preprocessing the image, and constructing a product defect data set; establishing a deep learning model, and training by utilizing the defect data set; image acquisition is carried out, and visual detection is carried out by using the trained deep learning model; the image acquisition comprises the following steps: placing a product at a set position, and illuminating the product by using a light source; placing a reflector on the side surface of the product to enable the side surface of the product to be imaged on the camera; and shooting the product by using the camera to obtain an image containing the top surface and the side surface of the product at the same time. According to the invention, a rotating device is not needed to rotate the product, on one hand, the production cost and the operation cost of detection equipment are saved, the influence of the precision of the rotating device on visual detection is avoided, and the stability and the precision of the visual detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to an AI visual inspection method based on deep learning. Background Art

[0002] Visual inspection refers to the use of machines to measure and judge instead of human eyes. The product is photographed by a camera with the assistance of a lens and a light source. The captured image is then stored in a computer or server hard disk through an acquisition card. The image is segmented and calculated using image processing software to obtain certain results, which are then used for product or equipment inspection, sorting, and control.

[0003] However, in order to capture images of multiple sides of a single product, existing visual inspection methods often require a rotating device to be installed in the inspection equipment to rotate the product. On the one hand, this increases the production and operating costs of the inspection equipment and affects the stability and accuracy of visual inspection. On the other hand, the image capture time for a single product is long, which makes it difficult to meet the rhythm requirements of the production line.

[0004] Based on the above technical problems, this application proposes an AI visual detection method based on deep learning. Summary of the invention

[0005] The purpose of the present invention is to provide an AI visual inspection method based on deep learning to solve the technical problems mentioned in the background technology. The purpose of the present invention is achieved through the following technical solutions: An AI visual inspection method based on deep learning includes the following steps: Step S1, acquiring images of different surfaces of a defective product; Step S2: preprocess the image to construct a product defect dataset; Step S3: Establish a deep learning model and use the defect data set for training; Step S4: perform image acquisition and use the trained deep learning model for visual inspection; The image acquisition in step S4 includes: Step S41, placing the product at a set position and using a light source to illuminate the product; Step S42, placing a reflector on the side of the product so that the side of the product is imaged on the camera; Step S43: photograph the product using the camera to obtain an image that includes both the top and side surfaces of the product.

[0006] Furthermore, step S2 specifically includes: Step S21, extracting ROI and denoising the extracted image; Step S22, marking defects in the image obtained in step S21; Step S23: perform image amplification on the annotated image to obtain a defect data set.

[0007] Furthermore, the image amplification methods in step S23 include HSV color conversion, rotation, mirroring, scaling, blurring, cropping / filling and migration.

[0008] Further, step S3 includes: Step S31, dividing the defect data set into a training set and a test set in proportion; Step S32, weighting the training set data; Step S33, inputting the weighted training set into the deep learning model for training; Step S34: input the test set into the deep learning model for testing and save the optimal deep learning model.

[0009] Furthermore, the weighted processing includes: Step S321, reading training set samples; Step S322, calculating the number of different defect categories; Step S323, assigning category weights according to the number of defect categories; Step S324, selecting the training set according to the category weight; The more categories there are, the smaller the assigned weights are.

[0010] Furthermore, the deep learning model includes a backbone network, a neck network and an output network, the backbone network is used to extract features, the neck network is used to enhance features, and the output network is used for detection and classification.

[0011] Furthermore, the backbone network includes five convolution modules, four CSPLayer modules and one SPPF module, and the convolution module includes a convolution layer, a normalization layer and a SiLU activation function.

[0012] Furthermore, the neck network is a path aggregation network with a feature pyramid network.

[0013] Furthermore, the output network includes a detection head and a classification head.

[0014] Furthermore, the light source is a stepped annular light source, and the reflector is arranged at an angle of 45°.

[0015] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages: 1. By arranging reflectors around the product to obtain images that include both the top and side surfaces of the product, there is no need to use a rotating device to rotate the product. On the one hand, this saves the production and operating costs of the inspection equipment, avoids the impact of the precision of the rotating device on visual inspection, and improves the stability and accuracy of visual inspection; on the other hand, it shortens the image acquisition time of a single product and improves the inspection efficiency of the product; 2. By weighting the training set data, the balance of test samples is ensured and the accuracy of training results is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings in the specification, which constitute a part of this application, are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0017] Figure 1 This is a flow chart of the principle of the embodiment of this application; Figure 2 This is a flowchart of image preprocessing in an embodiment of the present application; Figure 3 This is a flow chart of model training for an embodiment of the present application; Figure 4 This is a weighted processing flow chart of an embodiment of the present application; Figure 5 This is a flowchart of image acquisition according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the drawings and specific implementation methods of the specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] like Figure 1 The AI ​​visual inspection method based on deep learning shown in the figure comprises the following steps: Step S1, acquiring images of different surfaces of a defective product.

[0020] The defective product is illuminated by a light source, and the image of the fixed surface and four sides of the defective product is obtained by a camera. The focal plane of the camera is parallel to the corresponding surface of the product to avoid deformation of the defective image and affect subsequent detection and identification.

[0021] Step S2: pre-process the image and construct a product defect dataset.

[0022] Due to interference factors such as environmental conditions, the original images collected by the camera may contain various noises, which will interfere with the subsequent learning of the visual detection model. Therefore, it is necessary to preprocess the collected original images. Figure 2 As shown, image preprocessing specifically includes: Step S21: extract ROI and denoise the extracted image.

[0023] ROI refers to the region of interest, which is an area in an image or video that is designated as requiring special attention or processing. By extracting ROI, redundant information can be removed and the image processing efficiency can be improved. During image preprocessing, the ROI is extracted and the extracted ROI image is filtered using a median filter algorithm to improve the image quality and facilitate the visual detection model to identify the target.

[0024] Step S22: marking defects in the image obtained in step S21.

[0025] The pre-processed images are manually annotated using annotation software and the minimum bounding rectangle of each defect is used as the standard, and the images are saved as annotation files.

[0026] Step S23: perform image amplification on the annotated image to obtain a defect data set.

[0027] Since the defective rate is low in the actual production process, it is impossible to provide enough defect samples. In order to provide enough defect images, it is necessary to increase the number of defect images through image amplification. Image amplification methods include HSV color conversion, rotation, mirroring, scaling, blurring, cropping / filling and migration.

[0028] Among them, HSV stands for hue, saturation, and value. HSV color conversion can improve the intuitiveness of color recognition and facilitate the extraction and processing of color features. Rotation is to rotate all pixels on the image by the same angle with the center of the image as the origin. Mirroring can be divided into vertical mirroring and horizontal mirroring. Horizontal mirroring is to swap the left and right halves of the image with the vertical center axis of the image; vertical mirroring is to swap the upper and lower halves of the image with the horizontal center axis of the image. Scaling refers to adjusting the size of the image. Blurring refers to reducing the clarity of the image through mosaic or Gaussian blur; cropping / filling refers to reducing the size of the image by trimming or increasing the size of the image by adding edges. Migration refers to copying the defects marked in the image and moving them to another position in the image.

[0029] Step S3: Establish a deep learning model and use the defect data set for training.

[0030] like Figure 3 As shown, model training includes: Step S31, dividing the defect data set into a training set and a test set in a ratio of 8:2; Step S32, weighting the training set data; like Figure 4 As shown, the weighting process includes: Step S321, reading training set samples; Step S322, calculating the number of different defect categories; Step S323: assign category weights according to the number of defect categories. The more categories there are, the smaller the assigned weights are.

[0031] Step S324: Delete the training set according to the category weight.

[0032] During the training process of the deep learning model, only a portion of the data is used in each iteration to speed up the training. A weight is assigned to each defect image through weighted processing. When selecting data, the number of defect images of different types is selected according to the weight, so that the model is more balanced relative to the entire training set.

[0033] Step S33: input the weighted training set into the deep learning model for training.

[0034] The deep learning model includes a backbone network, a neck network and an output network. The backbone network is used to extract features, including five convolution modules, four CSPLayer modules and one SPPF module. The convolution module includes a convolution layer, a normalization layer and a SiLU activation function. The neck network is used to enhance features, and the neck network is a path aggregation network with a feature pyramid network. The output network is used for detection and classification, and the output network includes a detection head and a classification head. Preferably, the deep learning model is a YOLO algorithm, and its principle is a prior art and will not be repeated here.

[0035] Step S34: input the test set into the deep learning model for testing, save the optimal deep learning model, and evaluate the accuracy of the deep learning model through the confusion matrix.

[0036] Among them, the accuracy rate P calculation formula is: The calculation formula of recall rate R is: Where TP is the true positive class, FP is the false positive class, and FN is the false negative class.

[0037] Step S4: perform image acquisition and use the trained deep learning model for visual inspection.

[0038] like Figure 5 As shown, the image acquisition in step S4 includes: Step S41: Place the product at a set position and illuminate the product using a light source, wherein the light source is a stepped ring light source to illuminate the top and sides of the product.

[0039] Step S42: Place reflectors on the four sides of the product so that the sides of the product are imaged on the camera. The reflectors are arranged at a 45° angle, and the height of the reflectors is greater than or equal to the height of the product, so that the sides of the product can be imaged in the camera lens without deformation.

[0040] Step S43: Use a camera to shoot the product to obtain an image that includes both the top and side surfaces of the product. By taking a single shot to obtain images of multiple surfaces of the product, the frequency of image acquisition is reduced, time waste is reduced, and production rhythm is improved.

[0041] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages: 1. By arranging reflectors around the product to obtain images that include both the top and side surfaces of the product, there is no need to use a rotating device to rotate the product. On the one hand, this saves the production and operating costs of the inspection equipment, avoids the impact of the precision of the rotating device on visual inspection, and improves the stability and accuracy of visual inspection; on the other hand, it shortens the image acquisition time of a single product and improves the inspection efficiency of the product; 2. By weighting the training set data, the balance of test samples is ensured and the accuracy of training results is guaranteed.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An AI visual inspection method based on deep learning, characterized in that: The following steps are involved: Step S1, acquiring images of different surfaces of a defective product; Step S2: preprocess the image to construct a product defect dataset; Step S3: Establish a deep learning model and use the defect data set for training; Step S4: perform image acquisition and use the trained deep learning model for visual inspection; The image acquisition in step S4 includes: Step S41, placing the product at a set position and using a light source to illuminate the product; Step S42, placing a reflector on the side of the product so that the side of the product is imaged on the camera; Step S43: photograph the product using the camera to obtain an image that includes both the top and side surfaces of the product.

2. The AI ​​visual inspection method based on deep learning according to claim 1, characterized in that: Step S2 specifically includes: Step S21, extracting ROI and denoising the extracted image; Step S22, marking defects in the image obtained in step S21; Step S23: perform image amplification on the annotated image to obtain a defect data set.

3. The AI ​​visual inspection method based on deep learning according to claim 2, characterized in that: The image amplification methods in step S23 include HSV color conversion, rotation, mirroring, scaling, blurring, cropping / filling and migration.

4. The AI ​​visual inspection method based on deep learning according to claim 1, characterized in that: Step S3 includes: Step S31, dividing the defect data set into a training set and a test set in proportion; Step S32, weighting the training set data; Step S33, inputting the weighted training set into the deep learning model for training; Step S34: input the test set into the deep learning model for testing and save the optimal deep learning model.

5. The AI ​​visual inspection method based on deep learning according to claim 4 is characterized in that: The weighting process includes: Step S321, reading training set samples; Step S322, calculating the number of different defect categories; Step S323, assigning category weights according to the number of defect categories; Step S324, selecting the training set according to the category weight; The more categories there are, the smaller the assigned weights are.

6. The AI ​​visual inspection method based on deep learning according to claim 4, characterized in that: The deep learning model includes a backbone network, a neck network and an output network, wherein the backbone network is used to extract features, the neck network is used to enhance features, and the output network is used for detection and classification.

7. The AI ​​visual inspection method based on deep learning according to claim 6, characterized in that: The backbone network includes five convolution modules, four CSPLayer modules and one SPPF module, and the convolution module includes a convolution layer, a normalization layer and a SiLU activation function.

8. The AI ​​visual inspection method based on deep learning according to claim 6, characterized in that: The neck network is a path aggregation network with a characteristic pyramid network.

9. The AI ​​visual inspection method based on deep learning according to claim 6, characterized in that: The output network includes a detection head and a classification head.

10. The AI ​​visual inspection method based on deep learning according to claim 1, characterized in that: The light source is a stepped annular light source, and the reflector is arranged at an angle of 45°.

Images