A method, system and device for detecting defects of special-shaped small parts trays

Abstract

The present invention relates to a method, system and device for detecting defects of irregular-shaped small-piece trays, which obtains detection images of the upper and lower parts of the surface of the tray to be detected; divides the region of interest in the detection image to form a regional image, and uses a detection model established by a deep learning algorithm to detect the regional image to obtain the detection result of the regional image; calculates the detection result of the regional image back to the detection image to obtain the detection result of the irregular-shaped small-piece tray; and discloses a system and device using this method. The partitioned acquisition detection image of the invention can ensure that we obtain more detailed image data, and detect through the detection model, thereby improving the speed and accuracy of detection.

Description

Technical Field

[0001] The present invention relates to the field of production line quality inspection, and in particular to a method, system and equipment for detecting defects of special-shaped small-piece trays. Background Art

[0002] The quality of electronic products has become a key factor that more and more consumers are concerned about. In the production process, there are often some smaller, irregular-shaped components, which are called irregular small parts. Irregular components are components with irregular geometric shapes. These components are usually placed in blister boxes. The correct placement of irregular small parts is crucial to the subsequent process. Irregular small parts may have position deviation or missing during the automated placement process, which will affect the packaging and assembly process and lead to defects in product appearance and function. Therefore, accurate detection of irregular small parts placement defects is the key to ensuring product quality. The detection system is required to have higher accuracy and intelligence, not only to identify obvious placement problems, but also to have strong learning ability to effectively identify defects of different types and forms. Irregular small parts placement defects usually refer to the failure of irregular small parts to be placed correctly during the production process, resulting in position offset or missing. Existing traditional visual inspection can detect obvious placement problems, but it is often difficult to achieve good results for minor deviations, and it is easy to miss or misdetect. Summary of the invention

[0003] In view of the existing deficiencies, the present invention provides a method, system and equipment for detecting defects of special-shaped small-piece trays.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for detecting defects of special-shaped small-piece trays,

[0005] Acquire detection images of the upper and lower parts of the surface of the wobble plate to be detected;

[0006] A region of interest is divided in the detection image to form a regional image, and the regional image is detected using a detection model established by a deep learning algorithm to obtain a detection result of the regional image;

[0007] The detection result of the regional image is back-calculated to the detection image to obtain the detection result of the special-shaped small-piece display tray.

[0008] Preferably, the detection image is divided into regional images after binarization processing.

[0009] Preferably, the detection model is obtained by training the image database of special-shaped small parts on plates as a training set of a convolutional neural network, wherein the image database of special-shaped small parts on plates includes a standard image database of special-shaped small parts on plates and a defective image database of special-shaped small parts on plates.

[0010] Preferably, the establishment of the detection model includes the following steps: obtaining a target image from a database of images of irregular small-piece platter plates; extracting features of the target image; establishing a feature sequence based on the features of the target image, and sampling the feature sequence according to a preset sampling ratio to obtain sampling data; collecting the sampling data as a platter defect training set for a convolutional neural network, and using the platter defect training set to train the convolutional neural network to determine a detection model, wherein the detection model is used to determine the coordinate information of a defect target boundary box in a regional image.

[0011] A system for detecting defects in irregular-shaped small-piece platter, comprising: an image acquisition module and an image detection module, wherein the image acquisition module is used to acquire images of the upper and lower halves of the platter surface; the image detection module divides an area of ​​interest in the image acquired by the image acquisition module to form a regional image, and uses a detection model established by a deep learning algorithm to detect the regional image to obtain a detection result of the regional image, and then reversely calculates the detection result of the regional image back to the detection image to obtain a detection result of the irregular-shaped small-piece platter.

[0012] Preferably, the image acquisition module comprises an upper station line scan camera arranged corresponding to the upper half of the wobble plate surface and a lower station line scan camera arranged corresponding to the lower half of the wobble plate surface.

[0013] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to execute the computer program stored in the memory, so as to implement the method for detecting defects of irregular-shaped small-piece trays as described in any of the preceding items.

[0014] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting defects of a special-shaped small-piece tray as described in any of the preceding items is implemented.

[0015] The beneficial effects of the present invention are as follows: the invention acquires the detection image of the special-shaped small-piece platter in two parts, namely, an upper and lower part, so that the detection image of the special-shaped small-piece platter can be acquired more comprehensively and accurately, and the acquired image data is more detailed, thereby improving the comprehensiveness and accuracy of the detection; then, a specific region of interest (ROI) is defined for the detection image to form a regional image, thereby acquiring the key area image of the special-shaped small-piece platter, and concentrating the detection resources on those areas where defects are most likely to occur, and excluding those irrelevant areas, which not only reduces the storage space of the image, but also greatly shortens the training time, improves the recognition ability of slight deviations and missing small parts, improves the detection speed and accuracy, improves the detection efficiency, and optimizes the use of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of a detection method according to an embodiment of the present invention; DETAILED DESCRIPTION

[0017] In order to more clearly illustrate the purpose, technical solutions and advantages of the embodiments of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments for a clear and complete description. Obviously, the embodiments described are partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0018] Embodiments of the present invention Figure 1 As shown in, a method and system for detecting defects of a dish with irregular small parts are provided. The detection system includes an image acquisition module and an image detection module. During detection, the image acquisition module is used to obtain detection images of the upper and lower parts of the dish surface to be detected. That is, after the irregular small parts are placed in a blister box, images of the upper and lower parts of the dish surface of the blister box are obtained. At this time, the image acquisition module includes an upper station line scan camera corresponding to the upper part of the dish surface and a lower station line scan camera corresponding to the lower part of the dish surface. That is to say, two station line scan cameras are arranged in parallel at intervals on the production line. When the blister box is filled with small parts and flows through the two stations, one line scan camera The scanning camera is responsible for scanning and photographing the upper part of the surface of the blister box, and the other line scanning camera is responsible for scanning and photographing the lower part of the surface of the blister box. In this way, two local detection images of the special-shaped small parts after the plate is placed are obtained, which narrows the shooting range, obtains more detailed image data, and improves the recognition ability of slight deviations and missing small parts. At this time, multiple cameras with different angles can be set at each workstation for shooting, which further improves the comprehensiveness and accuracy of the detection, and the line scanning camera can also be replaced by structured light or laser scanning device to obtain three-dimensional information of the special-shaped surface, which can more accurately detect whether the position of the special-shaped small parts plate is deviated or missing;

[0019] The region of interest is divided out in the detection image to form a regional image, and the detection model established by the deep learning algorithm is used to detect the regional image to obtain the detection result of the regional image; the image detection module in the detection system is used to divide the region of interest (ROI) in the detection image obtained by the image acquisition module to form a regional image, and the detection model established by the deep learning algorithm is used to detect the regional image to obtain the detection result of the regional image. After the region of interest is delineated in the detection image, the key area image of the special-shaped small-piece platter can be obtained, especially the surface area of ​​the special-shaped platter, so that the detection resources can be concentrated on those areas where defects are most likely to occur, thereby excluding irrelevant areas and improving the accuracy and efficiency of detection. The division of ROI can also ensure that only the most important part of the image is analyzed during detection, effectively reducing the consumption of computing resources, speeding up the overall detection process, and performing target detection on the divided regional image. The detection model can mark the position deviation and missing small pieces in the regional image, ensuring rapid identification of unqualified items in the regional image, thereby providing detailed regional image detection results.

[0020] The detection results of the regional image are back-calculated back to the detection image to obtain the detection results of the irregular small parts platter; because the detection model is based on the ROI image, the detection results of the regional image are back-calculated back to the detection image, and this conversion is achieved by adding the coordinates of the detection results of the regional image with the coordinates of the upper left corner of the ROI. The number of missing small parts and position deviations can be calculated to evaluate whether the irregular small parts platter is qualified.

[0021] As a further improvement, the detection image is divided into regional images after binarization. The detection image is subjected to image smoothing and denoising operations and image contrast enhancement operations; wherein, in the image smoothing and denoising operation, the interference information in the image is eliminated, and the key information in the detection image is retained, and the key information includes contour information and edge information; in the image contrast enhancement operation, the detection image is enhanced by using a global linear grayscale transformation algorithm to improve the clarity and contrast of the image and strengthen the characteristics of the target area of ​​the image. In the image contrast enhancement, the image is enhanced by using a global linear grayscale transformation algorithm to improve the clarity and contrast of the image and strengthen the characteristics of the target area of ​​the image. Image smoothing and denoising and global linear grayscale transformation are implemented by any existing method. When dividing the regional image, the image processing software is used to separate the surface area of ​​the wedge plate from the background area, and finally a defect segmentation image represented by a binary value is obtained.

[0022] A further improvement is that the detection model is obtained by training the image database of the special-shaped small parts plate as a training set of the convolutional neural network. The image database of the special-shaped small parts plate includes a standard image database of the special-shaped small parts plate and a database of the image database of the special-shaped small parts plate with defects. The image database of the special-shaped small parts plate can be obtained based on the historical image data of the special-shaped small parts plate detection. For example, the images of the surface of the special-shaped small parts plate obtained in the past multiple special-shaped small parts plate detections are marked, and the images without defects on the special-shaped small parts plate are marked as standard image data of the special-shaped small parts plate, and the images with defects on the special-shaped small parts plate are marked as image data of the special-shaped small parts plate with defects, so that the image database of the special-shaped small parts plate can be obtained; the establishment of the detection model includes the following steps:

[0023] Obtain a target image from the image database of special-shaped small-piece trays. The target image is an image used for training and can be any image in the image database of special-shaped small-piece trays.

[0024] Extract the features of the target image, such as geometric features, including the length of the boundary, the area of ​​the region, the aspect ratio of the circumscribed rectangle, etc.;

[0025] A feature sequence is established according to the features of the target image, and the feature sequence is sampled according to the preset sampling ratio to obtain sampling data, that is, the image features are represented by a vector set or a matrix, and the feature points representing the image features are represented by vectors or numbers; the preset sampling ratio can be selected as 8:2, and 80% of the data in the feature sequence is used to constitute the sampling data;

[0026] The sampled data are collected as a training set of the convolutional neural network for plate defects, and the convolutional neural network is trained using the plate defect training set to determine the detection model, which is used to determine the coordinate information of the defect target boundary box in the regional image; the plate defect training set contains a data set for multiple trainings, and each data set contains multiple data as multiple training samples. At this time, the coordinate information of the boundary box of the defect target in the acquisition image can be determined by constructing and optimizing the defect target detection model based on the preset loss function. The preset loss function relies on the coordinate information of the actual boundary box and the predicted boundary box of the defect target, as well as the actual area and multiple predicted areas of the defect target; the loss function is an operation function used to measure the difference between the predicted value of the preset convolutional neural network and the true value, that is, the loss function is an operation function used to measure the difference between the training result of the preset convolutional neural network and the training label. It is a non-negative real value function. The smaller the loss function, the better the robustness of the model, such as the mean square error loss function, the Euclidean distance loss function, the Manhattan distance loss function, the huber loss function, the relative entropy, etc. In this way, when using the detection model for detection, a bounding box and the corresponding category confidence will be output for each detected defect target. By analyzing these bounding boxes, the location of the defect can be accurately marked on the image. At the same time, by comparing the maximum value of the confidence of each category, it can be determined whether the bounding box represents a position deviation or a missing part, so that the position deviation and missing small parts can be accurately marked.

[0027] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to execute the computer program stored in the memory, so as to implement the method for detecting defects of irregular-shaped small-piece trays as described in any of the preceding items.

[0028] A computer-readable storage medium stores a computer program. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory and software distribution medium, etc. that can carry the computer program code. The computer program code can be in any existing code form. When the computer program is executed by a processor, the method for detecting defects of irregular small pieces on a tray as described in any of the preceding items is implemented.

[0029] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made on the basis of the present invention. Therefore, these modifications or improvements made on the basis of not departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for detecting defects in a small shaped metal plate, characterized in that: Acquire detection images of the upper and lower parts of the surface of the wobble plate to be detected; A region of interest is divided in the detection image to form a regional image, and the regional image is detected using a detection model established by a deep learning algorithm to obtain a detection result of the regional image; The detection result of the regional image is back-calculated to the detection image to obtain the detection result of the special-shaped small-piece display tray.

2. The method for detecting defects of irregular-shaped small-piece wobble trays according to claim 1, characterized in that : The detection image is divided into regional images after binarization processing.

3. The method for detecting defects of odd-shaped small-piece wobble trays according to claim 1, characterized in that :The detection model is obtained by training the image database of special-shaped small parts on plates as the training set of the convolutional neural network, and the image database of special-shaped small parts on plates includes a standard image database of special-shaped small parts on plates and a defective image database of special-shaped small parts on plates.

4. The method for detecting defects of odd-shaped small-piece wobble trays according to claim 3 is characterized in that The establishment of the detection model includes the following steps: obtaining a target image from a database of images of irregular small-piece platter plates; extracting features of the target image; establishing a feature sequence based on the features of the target image, and sampling the feature sequence according to a preset sampling ratio to obtain sampling data; collecting the sampling data as a platter defect training set for a convolutional neural network, and using the platter defect training set to train the convolutional neural network to determine a detection model, wherein the detection model is used to determine the coordinate information of a defect target boundary box in a regional image.

5. A detection system for defects of special-shaped small-piece trays, characterized in that: include: An image acquisition module and an image detection module, wherein the image acquisition module is used to acquire images of the upper and lower parts of the surface of the wedge plate; The image detection module divides the area of ​​interest in the image obtained by the image acquisition module to form a regional image, and uses the detection model established by the deep learning algorithm to detect the regional image to obtain the detection result of the regional image, and then back-calculates the detection result of the regional image to obtain the detection result of the special-shaped small-piece platter.

6. The detection system for defects of irregular shaped small parts wobble trays according to claim 5, characterized in that: The image acquisition module comprises an upper station line scan camera arranged corresponding to the upper half of the surface of the wobble plate and a lower station line scan camera arranged corresponding to the lower half of the surface of the wobble plate.

7. An electronic device, characterized in that : includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to execute the computer program stored in the memory, so as to implement the method for detecting defects of irregular-shaped small-piece trays as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that : The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for detecting defects of irregular-shaped small-piece trays as described in any one of claims 1-4.

Images