Lamination flaw detecting and grading device and method

Abstract

The invention discloses a lamination flaw detection and grading method, which belongs to the field of motors and comprises the following steps: acquiring an image, manually marking lamination flaws, establishing a lamination flaw information identification model by using a deep learning algorithm, importing the image into the identification model for flaw detection, grading the detected flaws, generating a detection image and the like. A machine vision target detection technology is fused with a high-performance deep learning target detection algorithm YOLOV5, and lamination flaws are accurately classified and flaw positions are accurately positioned; a deep learning target detection technology is used for a lamination line scanning image, the disadvantage of deep learning in dimension measurement is solved, an image of a position where a defect is located is reserved, threshold segmentation is used for the image of the position where the defect is located, a large amount of complex backgrounds are removed, and a large amount of calculation amount is reduced. Therefore, the flaws of each area of the lamination can be accurately graded. The invention further comprises a lamination flaw detecting and grading device for implementing the lamination flaw detecting and grading method.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a lamination defect detection and grading device and method. Background Art

[0002] As various industries have higher and higher requirements for product quality and appearance, machine vision inspection technology plays an important role, but also faces huge challenges. At present, the detection of lamination defects in the motor industry mainly uses sensors, CCD cameras and other technologies to extract the information of the entire image of the lamination, and then extracts the defect information from the entire image, and then processes the extracted information data through a computer to determine the defect type. However, this discrimination method has high requirements for background complexity, and the detection accuracy of laminations with complex backgrounds is not high. It cannot accurately grade defects. Some tiny defects that can be ignored will be inspected, and some slightly larger defects will be inspected, which greatly causes a waste of resources. Therefore, a high-precision defect type classification and defect grade classification method is particularly important. Summary of the invention

[0003] In order to overcome the deficiencies of the prior art, one of the objectives of the present invention is to provide a high-precision lamination defect detection and grading method.

[0004] In order to overcome the deficiencies of the prior art, a second object of the present invention is to provide a high-precision lamination defect detection and grading device.

[0005] One of the purposes of the present invention is achieved by the following technical solution:

[0006] A lamination defect detection and grading method comprises the following steps:

[0007] Image acquisition: Take a picture with a line scan camera to obtain a grayscale image of the stack;

[0008] Manually label the defects of the laminate: crop the grayscale image, crop the original image into several valid images, and use the LabelImage labeling tool to manually label the defects in the laminate, with the labeling types of Chip_off and White_dot;

[0009] Use deep learning algorithms to establish a lamination defect information recognition model: divide the annotated image data into training set, test set and validation set, build a YOLOV5 target detection network, input the image data of the training set and test set after manual annotation into the YOLOV5 network for training, save the parameters of the YOLOV5 network after training several times and export the pt file;

[0010] Import the image into the recognition model for defect detection and classify the detected defects: Use the pt file of the YOLOV5 network to determine the defect type and locate the defect position on the stack, generate the ROI area of ​​the defect type and position, obtain the complete defect area, use the algorithm to crop the area of ​​interest, and obtain the defect image; use the closing operation on the image, and after the closing operation, use the threshold segmentation to analyze the defects to be detected. Through analysis, the defects and the background are divided according to the threshold, and the pixel value occupied by the defect threshold is counted; the area occupied by the defect is calculated by the pixel value occupied by the defect threshold, and the defects are graded according to the area size;

[0011] Generate detection map: return the classified grade information to the original image, and generate a detection map to complete the type and grade classification.

[0012] Furthermore, in the image acquisition step, a stroboscopic light source is used for lighting.

[0013] Furthermore, in the step of establishing a stacking defect information recognition model using a deep learning algorithm, the YOLOV5 target detection network is specifically built as follows: the Focus structure and the CSP structure are used as the bacbone of the network to automatically extract the features of the stacking defect type, the FPN+PAN structure is used as the Neck network of the network, and GIOU_Loss is used as the loss function of the output layer to output the defect type and position information of the stack to be detected.

[0014] Furthermore, building the YOLOV5 target detection network also includes introducing the CSP2 structure to enhance the network feature fusion capability.

[0015] Furthermore, the image is imported into the recognition model for defect detection and the detected defects are graded. The pt file of the YOLOV5 network is used to determine the defect type and locate the defect position of the laminate. Specifically, the specific positions of the Chip_off and White_dot defects in the image are identified. When the probability of the defect Chip_off type is greater than 0.2, the position of the Chip_off in the image is returned; when the probability of the defect White_dot type is greater than 0.2, the position of the White_dot in the image is returned.

[0016] Furthermore, the image is imported into the recognition model for defect detection and the detected defects are graded to obtain the defect complete area: the defect complete area is obtained by locating the coordinate information of the upper left corner and the lower right corner of the rectangular box where the ROI area is located.

[0017] Furthermore, in the step of importing the image into the recognition model for defect detection and classifying the detected defects, it is necessary to use Gaussian filtering on the defect image for noise reduction before using the closing operation on the image.

[0018] The second object of the present invention is achieved by adopting the following technical solution:

[0019] A stack defect detection and grading device, used to implement any one of the above-mentioned stack defect detection and grading methods, characterized in that: the stack defect detection and grading device includes a moving structure, an image acquisition structure, a controller and a calculator, the controller is electrically connected to the moving structure and the image acquisition structure, the controller controls the moving structure to drive the stack to move in two different directions, the image acquisition structure includes a line scan camera, the line scan camera takes a picture of the stack on the moving structure, collects a grayscale image of the stack and transmits the grayscale image to the calculator, the calculator performs defect detection on the grayscale image and grades the detected defects to generate a detection image.

[0020] Furthermore, the stack defect detection and grading device also includes a light source, which is located at the upper end of the stack to illuminate the stack, and the light source is a stroboscopic light source, and the controller is electrically connected to the light source to control the operation of the light source.

[0021] Furthermore, the image acquisition structure also includes a magnifying lens, which is installed at the lower end of the line scan camera and located between the line scan camera and the laminate, and the line scan camera takes a picture of the laminate through the magnifying lens.

[0022] Compared with the prior art, the stacking defect detection and grading method of the present invention adopts the target detection technology in machine vision technology and integrates the high-performance deep learning target detection algorithm YOLOV5, which can accurately classify stacking defects and locate the defect positions; deep learning target detection technology is used for stacking line scan images and combined with traditional image processing algorithms to solve the weakness of deep learning in size measurement, retain the image of the defect location, use threshold segmentation for the image of the defect location, remove a large amount of complex background, reduce a large amount of calculation, so that the defects in each area of ​​the stack can be accurately graded; high precision and high intelligence can realize online detection and grading of stacking defects, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the lamination defect detection and grading method of the present invention;

[0024] Figure 2 The original image of the lamination is collected in the lamination defect detection and grading method of the present invention;

[0025] Figure 3 A diagram for identifying and locating lamination defects in the lamination defect detection and grading method of the present invention;

[0026] Figure 4a This is the original image of the Chip_off defect;

[0027] Figure 4b It is the closed operation graph of Chip_off defect;

[0028] Figure 4c It is the threshold segmentation map of Chip_off defect;

[0029] Figure 5a This is the original image of the White_dot defect;

[0030] Figure 5b is the closed operation graph of White_dot defect;

[0031] Figure 5c It is the threshold segmentation map of White_dot defect;

[0032] Figure 6 Schematic diagram of the lamination defect detection and grading device of the present invention.

[0033] In the figure: 10, moving structure; 11, first guide rail; 12, second guide rail; 13, first driving member; 14, second driving member; 20, image acquisition structure; 21, bracket; 22, positioning member; 23, line scan camera; 24, magnifying lens; 30, light source; 40, mounting structure; 50, controller; 60, calculator; 200, lamination. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may be another intermediate component through which it is fixed. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may be another intermediate component at the same time. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be another intermediate component at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0037] Figure 1 This is a flow chart of the lamination defect detection and grading method of the present invention, which comprises the following steps:

[0038] Acquiring an image: acquiring a grayscale image of the stack 200 by taking a picture with the line scan camera 23;

[0039] Manually label the defects of the laminate 200: crop the grayscale image, crop the original image into several valid images, and use the LabelImage labeling tool to manually label the defects in the laminate 200, with the labeling types of Chip_off and White_dot;

[0040] Use deep learning algorithm to establish a laminate 200 defect information recognition model: divide the annotated image data into training set, test set and validation set, build a YOLOV5 target detection network, input the image data of the training set and test set after manual annotation into the YOLOV5 network for training, save the parameters of the YOLOV5 network after training several times and export the pt file;

[0041] Import the image into the recognition model for defect detection and classify the detected defects: Use the pt file of the YOLOV5 network to determine the defect type and locate the defect position of the stack 200, generate the ROI area of ​​the defect type and position, obtain the complete defect area, use the algorithm to crop the area of ​​interest, and obtain the defect image; use the closing operation on the image, and after the closing operation, use the threshold segmentation to analyze the defects to be detected. Through the analysis, the defects and the background are divided according to the threshold, and the pixel value occupied by the defect threshold is counted; the area occupied by the defect is calculated by the pixel value occupied by the defect threshold, and the defects are graded according to the area size;

[0042] Generate detection map: return the classified grade information to the original image, and generate a detection map to complete the type and grade classification.

[0043] The image acquisition step is as follows: the stack 200 is mounted on the mobile structure 10 using the mounting structure 40, and the first drive member 13 drives the first guide rail 11 to work, and the first guide rail 11 is a crawler guide rail. When the stack 200 reaches below the line scan camera 23, the second drive member 14 is operated, and the line scan camera 23 takes a photo to collect a grayscale image of the stack 200 (such as Figure 2 As shown), at the same time, the controller 50 (PLC) controls the light source 30 to illuminate, and controls the light source 30 to be a stroboscopic light source.

[0044] The specific steps for manually labeling defects in the stack 200 are as follows: crop the grayscale image, and the original image size obtained is 8186×6000, which is cropped into 30 effective images of 800×800; use the LabelImage labeling tool to manually label the defects in the stack 200, and the labeling types are Chip_off and White_dot.

[0045] The deep learning algorithm is used to establish a stack 200 defect information recognition model as follows: the annotated image data is divided into a training set, a test set, and a validation set according to the principle of 7:2:1; the Focus structure and the CSP structure are used as the bacbone of the network to automatically extract the features of the stack defect type; the FPN+PAN structure is used as the Neck network of the network, and the CSP2 structure is introduced to enhance the network feature fusion capability; GIOU_Loss is used as the loss function of the output layer to output the defect type and location information of the stack 200 to be detected; after the above three steps, a complete YOLOV5 target detection network is built, and the image data after manual annotation of the training set and the test set are input into the YOLOV5 network for training; after training 300 times, the parameters of the YOLOV5 network are saved and the pt file is exported, and the file size is about 169MB.

[0046] The steps of importing the image into the recognition model for defect detection and classifying the detected defects are as follows: using the pt file of the YOLOV5 network to determine the defect type and position of the laminate 200 (such as Figure 3 As shown), it is possible to identify chip_off in the image (such as Figure 4a As shown), White_dot (as shown Figure 5a When the probability of the defect chip_off type is greater than 0.2, the position of chip_off in the image is returned; when the probability of the defect White_dot type is greater than 0.2, the position of White_dot in the image is returned; the ROI area of ​​the defect type and position is generated, and the coordinate information of the upper left corner and the lower right corner of the rectangular box where the ROI area is located is located to obtain the complete defect area, and the algorithm is used to crop the area of ​​interest to obtain the defect image; the obtained defect image is subjected to Gaussian filtering for noise reduction, and then a 3×3 rectangular convolution kernel is used to close the image (such as Figure 4b as well as Figure 5b As shown), after the closing operation, the threshold segmentation is used to analyze the defects to be detected. Through analysis, the defects and the background are divided according to the threshold (as shown Figure 4cas well as Figure 5c As shown), the pixel value occupied by the defect threshold is counted; the area occupied by the defect is calculated according to the pixel value occupied by the defect threshold, and the defects are graded according to the area size.

[0047] The stacking defect detection and grading method of the present invention adopts the target detection technology in machine vision technology and integrates the high-performance deep learning target detection algorithm YOLOV5, which can accurately classify the stacking 200 defects and locate the defect positions; the deep learning target detection technology is used for the stacking 200 line scan image and combined with the traditional image processing algorithm to solve the weakness of deep learning in size measurement. The image of the defect location is retained, and the image of the defect location is segmented by threshold value to remove a large amount of complex background, which reduces a large amount of calculation, so that the defects in each area of ​​the stack can be accurately graded; high precision and high intelligence can realize online detection and grading of stacking defects, and has broad application prospects.

[0048] Please continue reading Figure 6 The present invention also relates to a lamination defect detection and grading device, which is used to implement the lamination defect detection and grading method. The lamination defect detection and grading device can detect defects on the lamination 200 and classify and grade the defects. The lamination defect detection and grading device includes a moving structure 10, an image acquisition structure 20, a light source 30, a mounting structure 40, a controller 50 and a calculator 60.

[0049] The mobile structure 10 includes a first guide rail 11, a second guide rail 12, a first driving member 13 and a second driving member 14. The first guide rail 11 provides movement in a first direction. The second guide rail 12 provides movement in a second direction perpendicular to the first direction. Specifically, the first guide rail 11 and the second guide rail 12 are crawler guide rails. The first guide rail 11 is driven by the first driving member 13, and the second guide rail 12 is driven by the second driving member 14. The first driving member 13 and the second driving member 14 are both motors.

[0050] The image acquisition structure 20 includes a bracket 21, a positioning member 22, a line scan camera 23, and a magnifying lens 24. The bracket 21 is fixed to one side of the mobile structure 10, the positioning member 22 is installed on the top of the bracket 21, and the line scan camera 23 is installed on the bracket 21 through the positioning member 22. At this time, the line scan camera 23 is located above the mobile structure 10. The magnifying lens 24 is installed below the line scan camera 23.

[0051] The light source 30 is located above the mobile structure 10 and on one side of the line scan camera 23. The light source 30 is a stroboscopic light source.

[0052] The mounting structure 40 is mounted on the second guide rail 12 of the moving structure 10. The mounting structure 40 can clamp the laminate 200 and position the laminate 200.

[0053] The controller 50 is electrically connected to the first driving member 13, the second driving member 14, the line scan camera 23 and the light source 30 to control the movement of the stack 200 and the operation of the line scan camera 23 and the light source 30. In this embodiment, the controller 50 is a PLC.

[0054] The calculator 60 is electrically connected to the controller 50 and the line scan camera 23. The calculator 60 acquires the grayscale image captured by the line scan camera 23 and performs defect detection and defect classification on the grayscale image.

[0055] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which are equivalent modifications and improvements made to the above embodiments based on the essential technology of the present invention, and all of them belong to the protection scope of the present invention.

Claims

1. A lamination defect detection and grading method, characterized in that: The following steps are involved: Image acquisition: Take a picture with a line scan camera to obtain a grayscale image of the stack; Manually label the defects of the laminate: crop the grayscale image, crop the original image into several valid images, and use the LabelImage labeling tool to manually label the defects in the laminate, with the labeling types of Chip_off and White_dot; Use deep learning algorithms to establish a lamination defect information recognition model: divide the annotated image data into training set, test set and validation set, build a YOLOV5 target detection network, input the image data of the training set and test set after manual annotation into the YOLOV5 network for training, save the parameters of the YOLOV5 network after training several times and export the pt file; Import the image into the recognition model for defect detection and grade the detected defects: Use the pt file of the YOLOV5 network to determine the defect type and locate the defect position of the laminate, generate the ROI area of ​​the defect type and position, obtain the complete defect area, use the algorithm to crop the area of ​​interest, and obtain the defect image; Use a closing operation on the image, and then use threshold segmentation to analyze the defects to be detected. Through analysis, the defects and background are divided according to the threshold, and the pixel value occupied by the defect threshold is counted; The area occupied by the defect is calculated by the pixel value occupied by the defect threshold, and the defects are graded according to the area size; Generate detection map: return the classified grade information to the original image, and generate a detection map to complete the type and grade classification.

2. The lamination defect detection and grading method according to claim 1, characterized in that: In the step of acquiring an image, a stroboscopic light source is used for lighting.

3. The lamination defect detection and grading method according to claim 1, characterized in that: In the step of establishing a stacking defect information recognition model using a deep learning algorithm, the YOLOV5 target detection network is specifically built as follows: the Focus structure and the CSP structure are used as the bacbone of the network to automatically extract the features of the stacking defect type, the FPN+PAN structure is used as the Neck network of the network, and GIOU_Loss is used as the loss function of the output layer to output the defect type and position information of the stack to be detected.

4. The lamination defect detection and grading method according to claim 3, characterized in that: Building the YOLOV5 target detection network also includes introducing the CSP2 structure to enhance the ability of network feature fusion.

5. The lamination defect detection and grading method according to claim 1, characterized in that: In the step of importing the image into the recognition model for defect detection and grading the detected defects, the pt file of the YOLOV5 network is used to determine the defect type and locate the defect position of the laminate. Specifically, the specific positions of the Chip_off and White_dot defects in the image are identified. When the probability of the defect Chip_off type is greater than 0.2, the position of Chip_off in the image is returned; when the probability of the defect White_dot type is greater than 0.2, the position of White_dot in the image is returned.

6. The lamination defect detection and grading method according to claim 1, characterized in that: In the step of importing the image into the recognition model for defect detection and classifying the detected defects, the defect complete area is obtained by locating the coordinate information of the upper left corner and the lower right corner of the rectangular box where the ROI area is located to obtain the defect complete area.

7. The lamination defect detection and grading method according to claim 1, characterized in that: In the step of importing the image into the recognition model for defect detection and classifying the detected defects, it is necessary to use Gaussian filtering on the defect image for noise reduction before using the closing operation on the image.

8. A lamination defect detection and grading device, used to implement the lamination defect detection and grading method according to any one of claims 1 to 7, characterized in that: The stack defect detection and grading device includes a moving structure, an image acquisition structure, a controller and a calculator. The controller is electrically connected to the moving structure and the image acquisition structure. The controller controls the moving structure to drive the stack to move in two different directions. The image acquisition structure includes a line scan camera. The line scan camera takes pictures of the stack on the moving structure, collects a grayscale image of the stack and transmits the grayscale image to the calculator. The calculator performs defect detection on the grayscale image and grades the detected defects to generate a detection image.

9. The lamination defect detection and grading device according to claim 8, characterized in that: The stack defect detection and grading device also includes a light source, which is located at the upper end of the stack to illuminate the stack. The light source is a stroboscopic light source, and the controller is electrically connected to the light source to control the operation of the light source.

10. The lamination defect detection and grading device according to claim 8, characterized in that: The image acquisition structure also includes a magnifying lens, which is installed at the lower end of the line scan camera and located between the line scan camera and the laminate. The line scan camera takes a picture of the laminate through the magnifying lens.

Images