Method and equipment for detecting product appearance through machine vision and storage medium thereof

Abstract

The invention relates to the technical field of capacitor appearance detection, and particularly provides a method and equipment for detecting the appearance of a product through machine vision and a storage medium thereof.The method comprises the steps that S1, image collection is conducted, specifically, image collection is conducted on a to-be-detected product; s2, image preprocessing: preprocessing the acquired image information; s3, feature extraction: extracting feature information from the preprocessed image by taking a VGG deep convolutional neural network model as a feature extractor; s4, feature classification: classifying the extracted feature information by using a classification algorithm model; and S5, defect judgment: inputting the classified feature information into a processor, and performing defect judgment on the feature information by using the processor. The objective of the invention is to solve the technical problems of high misjudgment rate, low identification efficiency and high cost of manual external detection.

Description

Technical Field

[0001] The present invention relates to the technical field of capacitor appearance detection, and in particular to a method, device and storage medium thereof for detecting product appearance through machine vision. Background Art

[0002] With the rapid development of the domestic electronics industry, the market demand for capacitors is increasing. In order to ensure the quality and life of capacitors, the detection of their appearance defects is an indispensable part of the capacitor production process. As capacitors tend to be miniaturized and integrated, the requirements for detection are getting higher and higher.

[0003] On the one hand, the industry has long relied on manual inspections to control products, using the naked eye to judge whether the product appearance meets the requirements. This inspection method not only has high labor costs, but also has low efficiency. In addition, when judging the product appearance, it relies on the subjective judgment of the inspection workers, which may lead to deviations in the control of some standards and may cause misjudgment.

[0004] On the other hand, during the production process, capacitors are prone to more than 20 defects in different locations and of different natures, such as top notches, bulging tops, pinching, hose ruptures, hose holes, poor bottom shrinkage, missing solder joints, broken legs, exposed copper, and liquid leakage. Manual inspection and identification using the naked eye alone has low recognition accuracy and speed, and a high misjudgment rate, which greatly reduces the product's pass rate. Once defective capacitors are put into use, the safety factor and service life of the corresponding products will be affected, causing huge safety hazards and economic losses.

[0005] At present, some existing electrolytic capacitor defect detection methods are mainly aimed at capacitor internal defect detection, foil defect detection, pin defect detection, arc discharge defect detection, etc. There is no corresponding solution technology for all-round and no-dead-angle defect detection of the appearance of finished electrolytic capacitors of different sizes and specifications. Summary of the invention

[0006] The purpose of the present invention is to provide a method, device and storage medium for detecting product appearance through machine vision, so as to solve the technical problems in the prior art that capacitors rely on manual external inspection, resulting in high misjudgment rate, low recognition efficiency and high labor cost.

[0007] The embodiments of the present invention are implemented by the following technical solutions:

[0008] The embodiment of the present invention provides a method for detecting the appearance of a product by machine vision, which comprises the following steps:

[0009] S1. Image acquisition: image acquisition of the product to be inspected;

[0010] S2, image preprocessing: preprocessing the collected image information;

[0011] S3, feature extraction: The VGG deep convolutional neural network model is used as a feature extractor to extract feature information from the preprocessed image;

[0012] S4, feature classification: use the classification algorithm model to classify the extracted feature information;

[0013] S5. Defect determination: The classified feature information is input into the processor, and the processor is used to perform defect determination on it.

[0014] Preferably, the image preprocessing in S2 further includes the following sub-steps:

[0015] S21, performing illumination and lens distortion correction on the collected image;

[0016] S22, performing multi-scale decomposition on the collected image using wavelet transform, and then performing threshold processing on the high-frequency part to remove image noise;

[0017] S23. Use a Gaussian filter to smooth the image, then use the Sobel operator to detect edges, and finally refine the image edges through non-maximum suppression and double threshold detection.

[0018] Preferably, the feature extraction in S3 is used to extract appearance image feature information.

[0019] Preferably, the appearance image feature information includes shape, color and identification information.

[0020] Preferably, the classification algorithm model in S4 adopts support vector machine SVM.

[0021] Preferably, the defect determination in S5 includes the following sub-steps:

[0022] S51, inputting the classified feature information into a processor;

[0023] S51, the processor extracts feature points and uses a feature point description algorithm to describe local information of the feature points;

[0024] S52, matching the extracted feature points with known standard feature points to find corresponding points;

[0025] S53, judging whether there is a defect according to the position and number of the matching points, if so, classifying it as an unqualified class, otherwise, classifying it as a qualified class;

[0026] S54. Classify the detected defects using a classification algorithm model.

[0027] Preferably, the method further comprises the following steps:

[0028] S6. Information storage: Store product information in two storage locations according to defect determination results.

[0029] Preferably, the method further comprises the following steps:

[0030] S7. Report generation: sort and classify the test results, automatically generate a test report, and record the results of each test.

[0031] An embodiment of the present invention also relates to an electronic device, comprising: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor can perform the steps of the above-mentioned method for detecting the appearance of a product through machine vision.

[0032] An embodiment of the present invention also relates to a computer-readable storage medium, on which is stored a program for a method for detecting the appearance of a product through machine vision, and the program for the method for detecting the appearance of a product through machine vision is executed by a processor to implement the steps of the above-mentioned method for detecting the appearance of a product through machine vision.

[0033] The technical solution of the embodiment of the present invention has at least the following advantages and beneficial effects:

[0034] (1) Using machine vision instead of manual inspection can not only improve production efficiency, but also improve product quality while reducing labor costs;

[0035] (2) Preprocess the collected images by using illumination and lens distortion correction, and use wavelet transform to perform multi-scale decomposition on the collected images. Then, threshold processing is performed on the high-frequency part to remove image noise. Use Gaussian filter to smooth the image, then use Sobel operator to detect edges, and finally refine the image edges through non-maximum suppression and double threshold detection, which effectively improves the image quality and makes subsequent feature extraction and analysis more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic flow chart of a method for inspecting product appearance through machine vision provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0039] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0040] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0041] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0042] The capacitor industry has long relied on manual inspection to control product appearance, using the naked eye to determine whether the product appearance meets the requirements. This method not only has high labor costs, but also has low efficiency and relies on the subjective judgment of the inspection workers when judging the product appearance. By using visual inspection equipment, the judgment of product appearance can be standardized, which can effectively ensure product quality, improve efficiency and save labor costs.

[0043] In order to solve the above problems, the present invention proposes a method for detecting the appearance of a product by machine vision, and the appearance detection method is as follows:

[0044] ⑴ Image acquisition: The products sorted by the sorting machine run through the conveyor belt, driving the products to the bottom of the CCD industrial camera for image acquisition.

[0045] ⑵ Image preprocessing: After the CCD camera collects the image information of the product, it performs preprocessing operations such as denoising, smoothing, and edge detection on the collected image to improve the image quality and prepare for subsequent defect detection.

[0046] (3) Feature extraction: Extract defect-related features such as shape, color, and identification features from the preprocessed image.

[0047] (4) Classification and recognition: The extracted features are input into a processor, such as a support vector machine or neural network, for detection and judgment.

[0048] (5) Defect detection: Based on the judgment results, qualified and unqualified products are transported to two storage locations respectively.

[0049] To achieve the above purpose, please refer to Figure 1 , which is a method flow chart of an embodiment of a method for detecting product appearance by machine vision provided in a specific embodiment of the present invention. Figure 1 As shown, in some embodiments, the method for detecting the appearance of a product by machine vision includes the following steps:

[0050] S1. Image acquisition: image acquisition of the product to be inspected;

[0051] In this embodiment, the capacitor to be tested is placed on a conveyor belt, a high-resolution industrial camera is fixedly arranged above the conveyor belt by a bracket, the shutter speed of the high-resolution camera is set, the product to be tested is transported to the bottom of the lens of the high-resolution industrial camera by the conveyor belt, and the capacitor to be tested is photographed by the high-resolution industrial camera to collect image information of the capacitor to be tested;

[0052] Specifically, in this embodiment, the image information of the capacitor to be tested includes appearance information of six surfaces of the capacitor, namely, the front surface, the side surface, and the back surface.

[0053] S2, image preprocessing: preprocessing the collected image information;

[0054] In this embodiment, the preprocessing of the collected image information includes image correction, denoising, smoothing and edge detection. Specifically, in this embodiment, it is first determined whether there is uneven illumination in the image. If so, the area with uneven illumination is selected, and then the local histogram equalization technology is used to expand the brightness range of the selected area and enhance the contrast of the overall image to achieve the purpose of illumination correction. At the same time, the internal parameters of the camera and the lens distortion coefficient are obtained, and the mapping relationship from the original pixel to the ideal pixel is established according to the calibration result. Then, the image is geometrically transformed according to the mapping function to remove the lens distortion, and finally, it is checked whether the corrected image eliminates the obvious distortion.

[0055] Specifically, in this embodiment, the collected image is decomposed into multiple scales using wavelet transform, and then the high-frequency part is thresholded to remove image noise. First, the collected image information is saved in digital format, and then the required wavelet base is prepared, and the image is subjected to two-dimensional wavelet transform. Here we use discrete wavelet transform to perform wavelet decomposition on the image, determine the threshold according to the standard part information, perform subtraction operation on the absolute value above the threshold, and use the wavelet coefficients after thresholding to perform inverse wavelet transform to reconstruct the denoised image.

[0056] Specifically, in this embodiment, the gaussian_blu ur function is used to smooth the input image to reduce noise; then the sobel_operator function is used to calculate the gradients in the horizontal and vertical directions, and the gradient amplitude and direction are obtained therefrom; then the non_max_suppression function is used to perform non-maximum suppression on the gradient amplitude, retaining only the local maximum value; finally, the hysteresis_thresholding function is used to apply double threshold detection to connect the weak edges around the strong edges and refine the image edges.

[0057] S3, feature extraction: The VGG deep convolutional neural network model is used as a feature extractor to extract feature information from the preprocessed image;

[0058] Specifically, the feature extractor extracts relevant information about the color, shape and logo of the image. In this embodiment, the trained VGG model is loaded from the Keras framework, and then the fully connected layer of the model is removed. The convolution layer of the VGG model is used as the feature extractor, and the preprocessed image information is input into the feature extractor to extract relevant information about the color, shape, logo and surface smoothness of each side of the image, and the extracted information is stored in correspondence with the number of each side of the product.

[0059] S4, feature classification: use the classification algorithm model to classify the extracted feature information;

[0060] In this embodiment, the classification algorithm model adopts a support vector machine SVM. Specifically, training set data and labels are collected, the training set data and labels are used to train the SVM model, the extracted image information is standardized, the image feature information is input into the trained SVM model, and the extracted feature information is classified using the SVM model, and the above feature information is divided into color information, shape information, facial flatness information, and identification information, etc.

[0061] S5, defect determination: input the classified feature information into a processor, and use the processor to perform defect determination on it;

[0062] In this embodiment, the classified feature information is input into the processor; then the processor extracts the feature points, and uses the feature point description algorithm to describe the local information of the feature points, such as the color of the image; then the extracted feature points are matched with the known standard feature points to find the corresponding points; then the presence of defects is determined based on the position and number of the matching points, such as: when measuring color information, when the position information of the color corresponding matching point is on the left side of the measured object, the quantity area is much larger than the quantity area of ​​the standard part corresponding color, and the standard part corresponding color is on the right side of the measured product, then the product is determined to be an unqualified product and is classified as an unqualified product; when measuring flatness, when it is detected that the height of the protrusion or depression in the area is greater than the preset value compared with the standard part, it is unqualified, and less than the preset value is qualified, where the preset value is the maximum height of the standard part + 3mm, and when the number of protrusions and depressions detected is greater than the preset value compared with the standard, it is unqualified, and less than the preset value, it is qualified, where the preset value is the sum of the number of protrusions and depressions of the standard part + 3.

[0063] Finally, the classification algorithm model is used to classify the detected defects, organize and annotate the training data set, ensure that the data set is divided into four categories: color, shape, logo, and flatness, use the training data set to train the classification algorithm model, input the detected defect information into the trained classification algorithm model, and classify the defect information into four categories: shape, color, logo, and flatness.

[0064] S6. Information storage: Store product information in two storage locations according to the defect determination result. For example, if the defect determination result in S5 is that the color of product No. 01 is abnormal and unqualified, then the relevant information of product 01 is stored in the storage location of unqualified products, and the stored information includes unqualified categories, colors, shapes, identification information, etc.; if the defect determination result shows that the product color, shape, identification and other parameter indicators are normal, then the detailed parameter information of the product is stored in the storage location of qualified products.

[0065] S7. Report generation: sort and classify the test results, automatically generate a test report, and record the results of each test.

[0066] Specifically, the VGG deep convolutional neural network consists of multiple convolutional layers, each of which uses a small 3x3 filter to keep the spatial dimensions of the input and output the same; the image passes through a series of convolutional layers, each of which uses a small 3x3 filter to extract local features of the image; each convolutional layer usually uses a ReLU activation function to introduce nonlinearity; after several convolutional layers, the spatial dimension of the feature map is reduced through a maximum pooling layer to retain the most important features; the combination of the above convolutional layers and pooling layers is repeated many times to gradually extract higher-level features; after multiple layers of convolution and pooling, a set of feature maps are finally obtained, which contain rich information of the image and ultimately achieve feature extraction.

[0067] Specifically, in this embodiment, the industrial CCD camera adopts a Hikvision CCD camera, model MV-CS200-10GC.

[0068] The camera's technical parameters are as follows:

[0069] ①Pixels: 20 million

[0070] ②Sensor type: CMOS, rolling shutter

[0071] ③Sensor model: SonyI MX183

[0072] ④ Resolution: 5472×3648

[0073] ⑤Maximum frame rate: 5.9fps@5472×3648

[0074] ⑥Dynamic range: 65.5dB

[0075] ⑦Signal-to-noise ratio: 41.5dB

[0076] ⑧Gain: 0~24dB

[0077] ⑨Exposure time: 46μs~2.5sec

[0078] ⑩Pixel format: Mono 8 / 10 / 12

[0079] Bayer GB 8 / 10 / 10Packed / 12 / 12Packed

[0080] YUV422Packed,YUV422_YUYV_Packed

[0081] RGB 8,BGR 8

[0082] The camera features are as follows:

[0083] It supports automatic exposure, manual exposure, one-key exposure mode, Rolling and Globa l Reset mode, the data interface supports Gi gabit Ethernet (1000Mbit / s) and is compatible with Fast Ethernet (100Mbit / s), 9-24VDC, and supports PoE power supply. The camera has built-in multiple complex algorithms and functions, such as lossless compression, 2D noise reduction, lens shading correction, CCM, etc., which can improve the camera image quality in all aspects.

[0084] An embodiment of the present invention also provides an electronic device, comprising: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor can perform the steps of the above-mentioned method for detecting the appearance of a product through machine vision.

[0085] An embodiment of the present invention also provides a computer-readable storage medium, on which is stored a program for a method for detecting the appearance of a product through machine vision. The program for the method for detecting the appearance of a product through machine vision is executed by a processor to implement the steps of the above-mentioned method for detecting the appearance of a product through machine vision.

[0086] The embodiment of the present invention uses machine vision to replace manual inspection, which can not only improve production efficiency, but also improve product quality while reducing labor costs.

[0087] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0088] At the same time, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, the alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for detecting product appearance by machine vision, characterized in that: The following steps are involved: S1. Image acquisition: image acquisition of the product to be inspected; S2, image preprocessing: preprocessing the collected image information; S3, feature extraction: The VGG deep convolutional neural network model is used as a feature extractor to extract feature information from the preprocessed image; S4, feature classification: use the classification algorithm model to classify the extracted feature information; S5. Defect determination: The classified feature information is input into the processor, and the processor is used to perform defect determination on it.

2. The method for detecting product appearance by machine vision according to claim 1, characterized in that: The image preprocessing in S2 further includes the following sub-steps: S21, performing illumination and lens distortion correction on the collected image; S22, performing multi-scale decomposition on the collected image using wavelet transform, and then performing threshold processing on the high-frequency part to remove image noise; S23. Use a Gaussian filter to smooth the image, then use the Sobel operator to detect edges, and finally refine the image edges through non-maximum suppression and double threshold detection.

3. The method for detecting product appearance by machine vision according to claim 1, characterized in that: The feature extractor in S3 is used to extract appearance image feature information.

4. The method for detecting product appearance by machine vision according to claim 3, characterized in that: The appearance image feature information includes shape, color and identification information.

5. The method for inspecting product appearance by machine vision according to claim 1, characterized in that: The classification algorithm model in S4 adopts support vector machine SVM.

6. The method for inspecting product appearance by machine vision according to claim 1, characterized in that: The defect determination in S5 includes the following sub-steps: S51, inputting the classified feature information into a processor; S51, the processor extracts feature points and uses a feature point description algorithm to describe local information of the feature points; S52, matching the extracted feature points with known standard feature points to find corresponding points; S53, judging whether there is a defect according to the position and number of the matching points, if so, classifying it as an unqualified class, otherwise, classifying it as a qualified class; S54. Classify the detected defects using a classification algorithm model.

7. The method for inspecting product appearance by machine vision according to claim 1, characterized in that: Also includes: S6. Information storage: Store product information in two storage locations according to defect determination results.

8. The method for inspecting product appearance by machine vision according to claim 1, characterized in that: The following steps are also included: S7. Report generation: sort and classify the test results, automatically generate a test report, and record the results of each test.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor can perform the steps of the method for detecting the appearance of a product by machine vision as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for a method for detecting the appearance of a product by machine vision, and the program for the method for detecting the appearance of a product by machine vision is executed by a processor to implement the steps of the method for detecting the appearance of a product by machine vision as described in any one of claims 1 to 8.

Images