Power pin product electroplating layer appearance defect detection method, device and related products

Abstract

The present application discloses a method, device and related products for detecting appearance defects of the electroplating layer of a power pin product. The method comprises: collecting an image of the electroplating layer to be detected of the power pin to be detected by combining a multi-spectral imaging device and a line scanning technology; preprocessing the image of the electroplating layer to be detected by using a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm; automatically extracting multi-level features in the image by using a deep convolutional neural network model; extracting physical features related to the physical properties of the electroplating layer by combining a feature engineering method; fusing the multi-level features with the physical features; compressing and screening the fused features by using feature dimensionality reduction technology to obtain a fused feature vector; and finally detecting the appearance defects of the electroplating layer by using a defect detection model. The present application can realize high-accuracy, high-precision and high-efficiency detection of appearance defects of the electroplating layer of power pin products.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for detecting appearance defects of electroplating layers of power pin products based on computer vision technology, and related products. Background Art

[0002] The existing electroplating layer appearance quality inspection usually adopts the method of comparing the appearance of the electroplating layer of the inspected product with the pre-set standard photo image of the qualified product to determine whether the electroplating layer of the inspected product is qualified. However, this method has many obvious disadvantages.

[0003] First, this method relies heavily on the accuracy and representativeness of the pre-set standard photo images. If the standard photo images themselves are biased or fail to cover all possible qualified situations, it is easy to cause misjudgment. For example, standard photos taken under different lighting conditions, shooting angles or equipment parameters may not truly reflect the actual qualified status of the product, making the test results unreliable. Secondly, this comparison method is difficult to accurately identify subtle defects and differences. Since it is only a simple image comparison, it is impossible to conduct in-depth analysis and quantitative evaluation of the characteristics of the defects. Some minor defects such as peeling, blistering, pitting, etc. may be ignored, thereby reducing the accuracy and sensitivity of the detection. In addition, this method is inefficient and not suitable for large-scale production testing. For a large number of products to be tested, it takes a lot of time and manpower to compare images one by one, which cannot meet the fast and efficient production rhythm of modern industry.

[0004] In summary, the existing electroplating layer appearance inspection technology based on standard photo image comparison has the disadvantages of low accuracy, poor precision and low efficiency, which makes it difficult to meet the increasingly high product quality inspection requirements. A more advanced, accurate and efficient inspection technology is urgently needed to solve these problems. Summary of the invention

[0005] This application provides a method for detecting the appearance defects of the electroplating layer of a power pin product based on computer vision technology, which can detect the appearance defects of the electroplating layer of the power pin product with high accuracy, high precision and high efficiency. It is specifically implemented through the following technical solutions:

[0006] In the first aspect, the present application provides a method for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology, comprising: using a combination of multispectral imaging equipment and line scanning technology to optimize acquisition parameters according to the size, shape and structural characteristics of the power pin to be detected, and collecting the electroplating layer image to be detected of the power pin to be detected; using a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm to pre-process the collected electroplating layer image to be detected to obtain a multispectral image to be detected; wherein the denoising algorithm learns the noise pattern through a deep convolutional neural network, and the illumination correction algorithm calculates and corrects the illumination intensity of the pixel points in the image according to the physical model of light propagation, and at the same time optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; using a deep convolutional neural network model to automatically extract multi-level features in the multispectral image to be detected; wherein the deep convolutional neural network The network model introduces an attention mechanism to focus on the feature area related to the preset common defects of the power pin; the physical features related to the physical characteristics of the electroplating layer of the power pin to be detected are extracted by combining the feature engineering method; the multi-level features of the extracted multi-spectral image to be detected and the physical features of the power pin to be detected are fused; the feature dimension reduction technology is used to compress and screen the fused features to obtain the fused feature vector of the multi-spectral image to be detected; the fused feature vector of the multi-spectral image to be detected is input into the pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be detected; wherein, the defect detection model is established using a support vector machine, and the defect detection model is trained using the fused feature vector of the multi-spectral sample image and the preset common defect category label of the power pin to learn the mapping relationship between the fused features of the multi-spectral sample image and the preset common defect category of the power pin.

[0007] In the second aspect, the present application provides a device for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology, including: an image acquisition module, which adopts a combination of multi-spectral imaging equipment and line scanning technology to optimize acquisition parameters according to the size, shape and structural characteristics of the power pin to be detected, and collect the electroplating layer image to be detected of the power pin to be detected; an image preprocessing module, which uses a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm to preprocess the collected electroplating layer image to be detected to obtain a multi-spectral image to be detected; wherein the denoising algorithm learns the noise pattern through a deep convolutional neural network, and the illumination correction algorithm calculates and corrects the illumination intensity of the pixel points in the image according to the physical model of light propagation, and at the same time optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; a feature extraction module, which uses a deep convolutional neural network model to automatically extract multi-level features in the multi-spectral image to be detected; wherein, The deep convolutional neural network model introduces an attention mechanism to focus on feature areas related to preset common defects of power pins; combines feature engineering methods to extract physical features related to the physical characteristics of the electroplating layer of the power pin to be detected; fuses the extracted multi-level features of the multi-spectral image to be detected and the physical features of the power pin to be detected; uses feature dimension reduction technology to compress and screen the fused features to obtain the fused feature vector of the multi-spectral image to be detected; a defect detection module is used to input the fused feature vector of the multi-spectral image to be detected into a pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be detected; wherein, the defect detection model is established using a support vector machine, and the defect detection model is trained using the fused feature vector of the multi-spectral sample image and the preset common defect category label of the power pin to learn the mapping relationship between the fused features of the multi-spectral sample image and the preset common defect category of the power pin.

[0008] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, it implements a method for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology as described in the first aspect above.

[0009] In a fourth aspect, the present application provides a computer storage medium, wherein the computer program contained in the computer storage medium is used, when executed by a computer device, to implement a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology as described in the first aspect above.

[0010] In a fifth aspect, the present application provides a computer program, which, when executed by a computer device, is used to implement a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology as described in the first aspect above.

[0011] From the above, it can be seen that this application has the following beneficial effects:

[0012] 1. The combination of multi-spectral imaging equipment and line scanning technology can optimize the acquisition parameters according to the size, shape and structural characteristics of the power pin to be detected, and can more accurately and comprehensively collect the image of the electroplating layer to be detected, thereby improving the image quality and detection accuracy.

[0013] 2. Use deep learning-driven denoising algorithms and physical model-based illumination correction algorithms for preprocessing to effectively remove noise and correct light intensity. Optimize correction parameters based on the shape of the power pin and the reflection characteristics of the electroplating layer to obtain clearer and more realistic multi-spectral images to be inspected, providing a reliable foundation for subsequent defect detection.

[0014] 3. The deep convolutional neural network model introduces an attention mechanism, which enables it to focus on the feature areas related to the common defects of the preset power pins, and can more efficiently extract key features and improve the accuracy and efficiency of defect detection.

[0015] 4. Combined with feature engineering methods, physical features related to the physical properties of the electroplating layer are extracted and fused, making full use of the multi-level features and physical features of the multispectral image, enriching the feature information and further improving the accuracy of defect detection.

[0016] 5. Use feature dimensionality reduction technology to compress and filter the fused features to obtain fused feature vectors, which can reduce computational complexity and improve detection speed while retaining key features to ensure detection accuracy.

[0017] 6. Use support vector machine to establish a defect detection model, and use the fused feature vector of multispectral sample images and preset common defect category labels of power pins for training. By learning the mapping relationship between the fused features of multispectral sample images and the preset common defect categories of power pins, accurate detection of appearance defects of the electroplating layer of the power pin can be achieved.

[0018] In summary, the present application achieves high-accuracy, high-precision and high-efficiency detection of appearance defects in the electroplating layer of power pin products; it overcomes the problems in the prior art that it is only a simple image comparison and is heavily dependent on the accuracy and representativeness of pre-set standard photo images, which can easily lead to misjudgment, and it is difficult to accurately identify subtle defects and differences, and it is impossible to conduct in-depth analysis and quantitative evaluation of the characteristics of the defects. Some tiny defects such as peeling, blistering, and pitting may be ignored, thereby reducing the accuracy and sensitivity of the detection, and it takes a lot of time and manpower to compare images one by one, resulting in low detection and recognition efficiency, and it is not suitable for large-scale production detection and cannot meet the fast and efficient production rhythm of modern industry. Technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the structure of a device for detecting appearance defects of electroplating layers of power pin products based on computer vision technology provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of a computer electronic device provided in an embodiment of the present application;

[0022] In the figure: 201, image acquisition module; 202, image preprocessing module; 203, feature extraction module; 204, defect detection module; 301, CPU; 302, ROM; 303, RAM; 304, bus; 305, I / O interface; 306, input part; 307, output part; 308, storage part; 309, communication part; 310, drive; 311, removable medium. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] See also Figure 1 The present application provides a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology, which specifically includes the following steps:

[0025] S1. A multi-spectral imaging device and line scanning technology are combined to optimize acquisition parameters according to the size, shape and structural characteristics of the power needle to be detected, and an image of the electroplating layer to be detected of the power needle to be detected is collected.

[0026] In an embodiment of the present invention, a high-speed line scan camera system can be used, which is composed of a high-speed moving line scan camera, a light source and a motion control device. First, use measuring tools such as calipers and microscopes to measure and analyze the size, shape and structure of the power needle to be detected. Then, determine the scanning path and parameter settings of the power needle to be detected based on the size, shape and structural characteristics of the power needle to be detected. Specifically, according to the structural characteristics of the power needle, select a suitable scanning path to ensure that the surface of the electroplating layer can be fully scanned. For example, for straight needle power needles, a straight scanning path can be used from one end to the other; for curved or special-shaped power needles, a curved or segmented scanning path is designed according to its shape to ensure that the entire surface of the electroplating layer is covered.

[0027] Choose a line scan camera with adjustable scanning speed and resolution. According to the length and diameter of the power pin to be inspected, adjust the scanning speed and resolution of the line scan camera to ensure that continuous images of the electroplating layer are obtained as quickly as possible without losing details. The length of the power pin is usually between 10㎜ and 2㎜, and the diameter is between 1㎜ and 2㎜. Generally speaking, for longer and thinner power pins, a lower scanning speed and a higher resolution can be selected to ensure that more details can be captured. For shorter and thicker power pins, a higher scanning speed and a lower resolution can be selected to improve the inspection efficiency. For example, if the length of the power pin is 15㎜ and the diameter is 1.5㎜, the scanning speed of the line scan camera can be set to 50 lines per second and the resolution can be set to 4096 pixels. This can increase the inspection speed while ensuring the inspection accuracy. If the length of the power pin is 10㎜ and the diameter is 1㎜, the scanning speed can be set to 60 lines per second and the resolution can be set to 4096 pixels. Because the power pin is short and thin, a higher resolution is required to detect surface defects. If the power needle is 20mm long and 2mm in diameter, the scanning speed can be set to 40 lines per second and the resolution can be set to 2048 pixels. Since the power needle is long and thick, a lower resolution and a slower scanning speed can meet the detection requirements and improve the detection efficiency.

[0028] When collecting images, place the power needle to be inspected on the above-mentioned motion control device, and start the above-mentioned high-speed line scanning camera system; the high-speed moving line scanning camera will scan the electroplating layer of the power needle to be inspected, and the continuous images of the electroplating layer to be inspected of the power needle to be inspected can be quickly obtained, and the continuous images of the electroplating layer to be inspected are transmitted to the computer for processing and analysis, which can ensure the dimensional accuracy and surface quality of the power needle. Among them, the motion control device is used to control the moving speed of the power needle to match the acquisition speed of the line scanning camera. According to the production speed of the power needle and the frame rate of the camera, set a suitable moving speed to ensure the clarity and continuous acquisition of the image. It should be noted that in an embodiment of the present invention, the executor of a method for detecting the appearance defects of the electroplating layer of a power needle product based on computer vision technology can be the processor of the computer.

[0029] In some feasible implementations, the light source in the above-mentioned high-speed line scan camera system can adopt a uniform white LED light source, and by adjusting the brightness and angle of the light source, the illumination on the surface of the power pin can be ensured to be uniform, and shadows and reflections can be reduced to ensure that the surface of the electroplating layer of the power pin can be clearly illuminated, so that the image of the electroplating layer can be clearly captured. For common electroplating materials of power pins, such as nickel, chromium, etc., the light source intensity can be set between 1000cd and 2000cd. Specifically, the image contrast and brightness can be adjusted according to the characteristics of the electroplating layer of the power pin to ensure that the defects and abnormalities of the electroplating layer can be clearly displayed; for example, the contrast is set to 50%~80%, and the brightness is adjusted to 50%~70% to highlight the defect characteristics of the electroplating layer.

[0030] At the same time, on the basis of the above, a high-resolution multi-spectral camera is used to collect visible light images to be detected and near-infrared images to be detected of the electroplating layer of the power pin to be detected in the visible light and near-infrared bands. The band parameters of the multi-spectral camera are adjusted according to the color and material of the electroplating layer of the power pin to be detected; specifically, the wavelength is determined according to the absorption characteristics of the electroplating layer material. For example, a high-resolution multi-spectral camera is selected to simultaneously collect images in the visible light and near-infrared bands. The camera has multiple filters of different wavelengths and can switch different bands for imaging. For nickel plating, a visible light wavelength of 550~650nm can be selected, including main color channels such as blue light, green light, and red light, to obtain the color and texture information of the power pin. For the near-infrared band, a wavelength of 800~1000nm can be selected to detect the thickness and uniformity of the electroplating layer, as well as whether there are potential defects. Therefore, the embodiment of the present invention collects the visible light image to be detected and the near-infrared image to be detected of the electroplating layer of the power pin to be detected in the visible light and near-infrared bands through a high-resolution multi-spectral camera, thereby ensuring the diversity of image data, and is conducive to using a deep convolutional neural network model to more accurately extract the texture, shape, color, electroplating layer thickness and conductivity characteristics in the image, so as to use the obtained fusion feature vector and defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be detected; the electroplating layer image to be detected can be collected more accurately and comprehensively, and the image quality and the accuracy of appearance defect detection can be improved.

[0031] In summary, the above-mentioned optimized acquisition parameters are used to acquire images of the power supply pin to be detected; the acquired continuous images of the electroplating layer to be detected, the visible light images to be detected, and the near-infrared images to be detected are used as the electroplating layer images to be detected of the power supply pin to be detected, wherein a suitable shooting position is set on the production line to ensure that each angle of the power supply pin can be clearly photographed, and for each power supply pin, multiple groups of images are acquired from different angles and under different lighting conditions to increase the diversity of data. For example, shooting from the front, side, top and other angles, while adjusting the light intensity and angle, to acquire images under different lighting conditions. In some feasible implementations, the acquired images can also be verified and analyzed to evaluate the optimization effect of the acquisition parameters. If necessary, the acquisition parameters can be further adjusted to improve the image quality and detection accuracy.

[0032] Through the above steps, it is possible to optimize the acquisition parameters according to the size, shape and structural characteristics of the power needle by combining multi-spectral imaging equipment and line scanning technology, and collect images of the electroplating layer of the power needle. In practical applications, adjustments and optimizations need to be made according to specific circumstances to ensure that high-quality images and accurate detection results are collected. The embodiment of the present invention uses a combination of multi-spectral imaging equipment and line scanning technology to collect images of the electroplating layer of the power needle in the visible light and near-infrared bands, and performs defect detection and analysis, which can effectively improve the quality and reliability of the product.

[0033] S2. Preprocess the collected images of the electroplating layer to be inspected using a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm to obtain a multispectral image to be inspected.

[0034] The denoising algorithm learns the noise pattern through a deep convolutional neural network. Specifically, an image denoising model based on a deep convolutional neural network architecture is constructed, such as DnCNN (Deep Convolutional Neural Network for Image Denoising); the original sample image of the electroplating layer of the sample power needle is obtained by the image acquisition method in the above step S1, that is, the acquisition parameters are optimized according to the size, shape and structural characteristics of the sample power needle by combining a multispectral imaging device and a line scanning technology, and the original sample image of the electroplating layer of the sample power needle is acquired. The specific implementation method can refer to the above step S1, which will not be repeated here. It can be understood that in order to better train the image denoising model, a large number of original sample images of the electroplating layer of sample power needles containing noise and noise-free / low noise, such as salt and pepper noise, Gaussian noise, etc., can be collected in a targeted manner, and then the noise area and noise pattern in the original sample image of the electroplating layer are annotated. For example, the noise pattern in the original sample image of the electroplating layer can be manually annotated, and the noise area and noise category are marked as a specific color or label. The annotated original sample image of the electroplating layer is used as the training set of the image denoising model. The training set is input into the image denoising model for training, and the weights and parameters of the network are continuously adjusted through the back propagation algorithm, so that the network learns the characteristics and denoising methods of different noise patterns; after multiple iterations of training, the image denoising model can identify the noise in the power needle image, and automatically remove the noise to restore clear image details; the image denoising model outputs a denoised sample image after the annotated noise area and noise pattern have been removed, so as to denoise the original sample image of the electroplating layer, and the image becomes clearer. Of course, it can be understood that the annotation of the original sample image of the electroplating layer can also include the normal area, and the annotated original sample image of the electroplating layer and the normal original sample image of the electroplating layer are divided into a training set, a verification set and a test set, and the image denoising model is trained, verified and tested using the training set, the verification set and the test set.

[0035] Then, the image denoising model trained above is used to denoise the collected image of the electroplating layer to be detected, and the image of the electroplating layer to be detected is input into the image denoising model, and the denoised image of the electroplating layer to be detected is output after the noise area and noise pattern of the image of the electroplating layer to be detected are removed. DnCNN has shown excellent performance in image denoising, and can effectively remove various types of noise while retaining the details and structural information of the image; after proper training, DnCNN can have good generalization ability for different types of images and noise, and is suitable for various practical application scenarios; without manually designing complex denoising algorithms, DnCNN can automatically learn denoising patterns and rules from training data through end-to-end learning; in addition, DnCNN has relatively high computational efficiency and can quickly process large-scale image data, which is conducive to improving the detection efficiency of appearance defects of the electroplating layer of power pin products.

[0036] The illumination correction algorithm calculates and corrects the illumination intensity of the pixels in the image based on the physical model of illumination propagation, and optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer. Specifically, the corrected illumination intensity of any pixel in the denoised image to be detected is calculated by the illumination intensity calculation formula, and the actual illumination intensity of any pixel is optimized and corrected by the calculated corrected illumination intensity until all pixels in the denoised image to be detected have completed the optimized correction of illumination intensity, and finally the multi-spectral image to be detected of the electroplating layer image to be detected after the above-mentioned denoising and illumination correction preprocessing is obtained; wherein, the calculation formula of the illumination intensity calculation formula is:

[0037] ;

[0038] in,

[0039] Indicates the corrected light intensity;

[0040] Indicates the light intensity of the incident light source;

[0041] Indicates the distance between the current pixel and the incident light source;

[0042] represents the attenuation factor;

[0043] Wherein, the attenuation factor is optimized according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; Represents the attenuation of the corrected light intensity of the current pixel as the distance from the incident light source increases; in addition, different types of light sources have different attenuation characteristics. For example, the light intensity distribution and attenuation methods of point light sources, line light sources, and surface light sources are all different. For point light sources, the light intensity attenuates evenly in all directions; while for line light sources or surface light sources, the attenuation of the light intensity is affected by the shape and size of the light source. When calculating the attenuation coefficient, it is necessary to consider the specific type of light source used. The intensity of the light source may not be evenly distributed, but may have different intensities in different directions. For example, some light sources may have higher intensity in the center area and lower intensity in the edge area. For example, LED light sources usually have a certain angle range within which the light intensity is relatively strong, and the light intensity decreases rapidly beyond this range. Attenuation coefficient It can be adjusted according to the intensity distribution of the light source to more accurately reflect the actual light intensity attenuation. It is a parameter that comprehensively considers multiple factors such as the ambient medium, distance, and light source characteristics, and is used to more accurately describe the attenuation of light intensity in the light intensity calculation formula. In practical applications, it is necessary to determine the specific attenuation coefficient value through experimental measurement and analysis to adapt to different detection environments and the characteristics of the power pin. Therefore, the embodiment of the present invention can adjust the attenuation coefficient in the model according to the special shape and reflection characteristics of the power pin. .

[0044] Indicates the incident angle of the incident light source from the air to the current pixel;

[0045] It indicates the refraction angle of the incident light source in the electroplating layer after it enters the current pixel from the air;

[0046] Represents the angle between the incident direction of the incident light source and the normal of the current pixel;

[0047] represents the angle-dependent function, which takes into account the changes in illumination response at different angles;

[0048] represents the area element at the current pixel, and Represents the normalization factor of the area element to ensure the consistency of the calculation of the corrected light intensity on different area elements.

[0049] In a feasible implementation, for a cylindrical power needle, considering the different angle responses of the cylindrical side and end faces, the following angle correlation function can be constructed. That is, if the shape of the power needle is cylindrical, the calculation formula of the angle correlation function is as follows:

[0050] ;

[0051] in,

[0052] It represents the angle correlation function corresponding to when the current pixel point is located on the side of the cylinder, that is, the angle correlation function when the current pixel point is a side point. The calculation formula is as follows:

[0053] ;

[0054] in, represents the radius of the cylinder; Indicates the vertical distance from the incident light source to the cylinder axis; It reflects the response of light shining on the side as the angle changes; is a normalization factor that ensures that the responses at different locations are relatively reasonable.

[0055] It represents the angle correlation function corresponding to when the current pixel point is located at the end face of the cylinder, that is, the angle correlation function when the current pixel point is an end face point; when considering the end face, the angle response is relatively simple, because the end face is usually perpendicular to the light source direction or close to perpendicular, so its calculation formula is as follows:

[0056] ;

[0057] in, It indicates the angle threshold between the incident direction of the incident light source and the normal of the current pixel. When the angle is large, it means that the light source has little effect on the illumination of the end face and can be ignored.

[0058] It should be noted that quality assessment indicators such as SNR (Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) can be defined to evaluate the quality of images after denoising and illumination correction (i.e., the image quality of multispectral images to be detected or multispectral sample images). For example, by calculating the SNR of the image before and after denoising, the higher the SNR, the better the denoising effect; and calculating the SSIM of the image before and after illumination correction, the closer the SSIM is to 1, the better the illumination correction effect. Therefore, those skilled in the art can optimize and adjust the denoising algorithm and the illumination correction algorithm according to the results of the quality assessment indicators; if it is found that there is still some residual noise in the denoised image, the depth of the DCNN can be increased or the training parameters can be adjusted; if the image after illumination correction still has the problem of uneven illumination in some areas, the correction parameters can be further optimized or the light source system can be adjusted.

[0059] The embodiment of the present invention calculates the corrected light intensity of any pixel in the denoised image to be detected by the above-mentioned light intensity calculation formula, and uses the calculated corrected light intensity to optimize and correct the actual light intensity of any pixel, until all pixels in the denoised image to be detected have completed the optimized correction of light intensity, and finally obtains the multi-spectral image to be detected after preprocessing of the electroplating layer image to be detected; through the light correction algorithm, the light intensity of the area in the image that appears darker due to uneven light can be increased to make it more consistent with the light in the surrounding area. By effectively using the deep learning-driven denoising algorithm and the physical model-based light correction algorithm to preprocess the collected power pin image, the image quality is improved, providing a better foundation for subsequent defect detection and analysis.

[0060] S3. Use a deep convolutional neural network model to automatically extract multi-level features in the multi-spectral image to be detected; and combine feature engineering methods to extract physical features related to the physical properties of the electroplating layer of the power pin to be detected.

[0061] First, we build a feature extraction model based on a deep convolutional neural network architecture, and select a deep convolutional neural network architecture suitable for multispectral image feature extraction, such as ResNet (Residual Network), DenseNet (Dense Connection Network), etc. These network architectures perform well in image classification and feature extraction, and can effectively extract multi-level features in multispectral images.

[0062] Similarly, after the original sample image of the electroplating layer is subjected to denoising and illumination correction preprocessing, the corresponding multispectral sample image is obtained; the specific implementation method can refer to the above step S2, which will not be repeated here. Then, the characteristic areas and defect categories related to the preset common defects of the power pin in the multispectral sample image are annotated, and the common defects of the power pin, such as peeling, pitting, flaking and cracking, as well as the normal area are annotated. The annotated multispectral sample image is input into the feature extraction model for training, and the feature extraction model is optimized using the random gradient descent algorithm; the feature extraction model automatically extracts the texture, shape and color features of multiple levels in the multispectral image to be detected; at the same time, the texture, shape and color features output by the feature extraction model are weighted by combining the spatial attention mechanism and the channel attention mechanism, and the importance weights of different areas in the image can be automatically learned, so that the feature extraction model can focus on the characteristic areas related to the preset common defects of the power pin, and improve the accuracy of defect detection. Among them, the texture feature can be extracted by the convolution layer of the feature extraction model, for example, extracting information such as edges and textures in the image. Shape features can be extracted through the pooling layer and the fully connected layer of the feature extraction model, for example, to extract information such as the shape and outline of the object in the image. Color features can be extracted through the input layer and the convolution layer of the feature extraction model, for example, to extract information such as the color distribution and color contrast in the image. During the model training process, the parameters of the attention mechanism are continuously adjusted through the back propagation algorithm so that it can better adapt to the characteristics and defect patterns of the power needle image.

[0063] At the same time, the plating thickness and conductivity of the power pin to be tested are extracted by feature engineering methods. Specifically, professional measuring equipment, such as a plating thickness meter, a conductivity tester, etc., are used to measure the plating thickness and conductivity of each power pin. It can be understood that if it is in the model training stage, the corresponding plating thickness and conductivity of the sample power pin are extracted by feature engineering methods; in order to make the model training effect better and more accurate, the measured physical property data is associated with its corresponding multi-spectral sample image to establish a one-to-one correspondence. For example, a unique identification code can be assigned to each power pin, and its multi-spectral sample image and its corresponding physical property data (plating thickness and conductivity) are associated with this identification code.

[0064] S4. Fusing the extracted multi-level features of the multi-spectral image to be detected and the physical features of the power pin to be detected.

[0065] The plating layer thickness is taken as a one-dimensional feature vector, namely, the plating layer thickness feature vector, and the conductivity is taken as a one-dimensional feature vector, namely, the conductivity feature vector. The plating layer thickness feature vector and the conductivity feature vector are also taken as inputs of the feature extraction model.

[0066] The last layer of the feature extraction model also includes a fully connected layer, which maps the extracted texture, shape and color features to a feature vector of a preset length to obtain a texture feature vector, a shape feature vector and a color feature vector; the texture feature vector, the shape feature vector, the color feature vector and the electroplating layer thickness feature vector are fused through the fully connected layer of the feature extraction model, and the texture feature vector, the shape feature vector, the color feature vector and the conductivity feature vector are fused; the fully connected layer of the feature extraction model outputs the fusion vector of the multi-spectral image to be detected.

[0067] S5. Use feature dimensionality reduction technology to compress and filter the fused features to obtain the fused feature vector of the multi-spectral image to be detected.

[0068] The principal component analysis (PCA) method is used to reduce the dimension of the fusion vector; then the mutual information-based feature screening method is used to calculate the mutual information between the fusion vector after dimension reduction and the preset common defect categories of power pins, and the qualified fusion vector is selected by the preset mutual information threshold, that is, all fusion vectors greater than or equal to the mutual information threshold are used as the fusion feature vector of the multispectral sample image. The feature dimension reduction technology is used to reduce the dimension of the feature, improve the quality and interpretability of the feature, and then the feature screening method based on mutual information is used to screen out the fusion vector that is closer to the common defect categories of power pins, which is conducive to further improving the accuracy of defect detection.

[0069] To summarize, the trained feature extraction model is finally used to extract the multi-level features in the multi-spectral image to be detected; and the physical features related to the physical properties of the electroplating layer of the power pin to be detected are extracted in combination with the feature engineering method; the extracted multi-level features of the multi-spectral image to be detected and the physical features of the power pin to be detected are fused; the feature dimensionality reduction technology is used to compress and screen the fused features to obtain the fused feature vector of the multi-spectral image to be detected.

[0070] S6. Input the fused feature vector of the multi-spectral image to be inspected into a pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be inspected.

[0071] Among them, the defect detection model is established using a support vector machine (SVM), and the defect detection model is trained using the fused feature vector of the multispectral sample image and the preset common defect category label of the power pin, so as to learn the mapping relationship between the fused features of the multispectral sample image and the preset common defect category of the power pin; thereby judging whether there are defects in the original sample image of the electroplating layer and the category of the defects.

[0072] The trained defect detection model is used to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer to be detected image. According to the mapping relationship between the fusion features of the multi-spectral sample image learned by the defect detection model and the preset common defect categories of the power pin, it is judged whether the fusion feature vector of the multi-spectral image to be detected satisfies the mapping relationship. If not, the defect detection model outputs a detection result that there are no appearance defects in the electroplating layer to be detected image. If so, the defect detection model outputs a detection result that there are appearance defects in the electroplating layer to be detected image and the defect category therein.

[0073] In a feasible implementation, the method described in the present application may also include the steps of: establishing a real-time feedback mechanism to promptly feed back the test results to the electroplating production equipment to achieve dynamic adjustment and optimization of the production process. According to the classification results, the power needles are quality evaluated and screened, and defective power needles are marked. The test results are fed back to the production line operators and quality management personnel so that timely measures can be taken for quality control. If serious defects are found, the production line can be stopped immediately for adjustments and repairs; for minor defects, records and statistics can be made for subsequent improvement of the production process. Regularly analyze and summarize the test results, evaluate the stability and quality level of the production process, and put forward improvement suggestions and measures.

[0074] In summary, the present application achieves high-accuracy, high-precision and high-efficiency detection of appearance defects in the electroplating layer of power pin products; it overcomes the problems in the prior art that it is only a simple image comparison and is heavily dependent on the accuracy and representativeness of pre-set standard photo images, which can easily lead to misjudgment, and it is difficult to accurately identify subtle defects and differences, and it is impossible to conduct in-depth analysis and quantitative evaluation of the characteristics of the defects. Some tiny defects such as peeling, blistering, and pitting may be ignored, thereby reducing the accuracy and sensitivity of the detection, and it takes a lot of time and manpower to compare images one by one, resulting in low detection and recognition efficiency, and it is not suitable for large-scale production detection and cannot meet the fast and efficient production rhythm of modern industry. Technical problems.

[0075] Combination of the above Figure 1The present invention provides a method for detecting the appearance defects of the electroplating layer of a power pin product based on computer vision technology in detail. Figure 2 A detailed description is given of a device for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology for executing a method for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology provided in an embodiment of the present application. Figure 2 A schematic diagram of a device for detecting appearance defects of electroplating layers of power pin products based on computer vision technology provided by an embodiment of the present invention; Figure 2 The device for detecting appearance defects of electroplating layer of power pin products based on computer vision technology includes:

[0076] The image acquisition module 201 is used to optimize acquisition parameters according to the size, shape and structural characteristics of the power pin to be detected by combining multi-spectral imaging equipment and line scanning technology, and to acquire the image of the electroplating layer to be detected of the power pin to be detected;

[0077] The image preprocessing module 202 is used to preprocess the collected image of the electroplating layer to be detected by using a denoising algorithm driven by deep learning and an illumination correction algorithm based on a physical model to obtain a multi-spectral image to be detected; wherein the denoising algorithm learns the noise pattern through a deep convolutional neural network, and the illumination correction algorithm calculates and corrects the illumination intensity of the pixel points in the image according to the physical model of light propagation, and optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer;

[0078] The feature extraction module 203 is used to automatically extract multi-level features from the multi-spectral image to be detected by using a deep convolutional neural network model; wherein the deep convolutional neural network model introduces an attention mechanism to focus on feature areas related to common defects of preset power pins; combines feature engineering methods to extract physical features related to the physical properties of the electroplating layer of the power pin to be detected; fuses the extracted multi-level features of the multi-spectral image to be detected and the physical features of the power pin to be detected; and uses feature dimensionality reduction technology to compress and screen the fused features to obtain a fused feature vector of the multi-spectral image to be detected;

[0079] The defect detection module 204 is used to input the fused feature vector of the multi-spectral image to be detected into a pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be detected; wherein, the defect detection model is established using a support vector machine, and the defect detection model is trained using the fused feature vector of the multi-spectral sample image and the preset common defect category label of the power pin, so as to learn the mapping relationship between the fused feature of the multi-spectral sample image and the preset common defect category of the power pin.

[0080] The specific implementation methods and technical effects of the device refer to the aforementioned method for detecting appearance defects of the electroplating layer of power pin products based on computer vision technology, which will not be repeated here.

[0081] An embodiment of the present invention further provides a computer electronic device, Figure 3 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 3 As shown, the computer electronic device includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other through the bus 304. The input / output (I / O) interface 305 is also connected to the bus 304. The following components are connected to the I / O interface 305: an input part 306 including a keyboard, a mouse, etc.; an output part 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 308 including a hard disk, etc.; and a communication part 309 including a network interface card such as a LAN card, a modem, etc. The communication part 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] As another aspect, an embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in a device for detecting appearance defects of electroplating layers of power pin products based on computer vision technology described in the above embodiment; or a computer-readable storage medium included separately and not assembled in an electronic device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology described in the present invention.

[0084] As another aspect, an embodiment of the present invention provides a computer program, which, when executed by a computer device, is used to implement a method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology as described in the above embodiment.

[0085] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but also should cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology, characterized in that: include: By combining multispectral imaging equipment and line scanning technology, the acquisition parameters are optimized according to the size, shape and structural characteristics of the power pin to be detected, and the image of the electroplating layer to be detected of the power pin to be detected is collected; The collected image of the electroplating layer to be detected is preprocessed using a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm to obtain a multispectral image to be detected; wherein the denoising algorithm learns the noise pattern through a deep convolutional neural network, and the illumination correction algorithm calculates and corrects the illumination intensity of the pixel points in the image according to the physical model of light propagation, and at the same time optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; Build a feature extraction model based on a deep convolutional neural network architecture; The denoised sample image is subjected to illumination correction processing by using an illumination intensity calculation formula to obtain a multispectral sample image; the characteristic regions and defect categories related to the preset common defects of the power pin in the multispectral sample image are annotated; the annotated multispectral sample image is input into the feature extraction model for training, and the feature extraction model is optimized by using a random gradient descent algorithm; the feature extraction model is used to automatically extract the texture, shape and color features of multiple levels in the multispectral image to be detected; at the same time, the texture, shape and color features output by the feature extraction model are weighted by combining the spatial attention mechanism and the channel attention mechanism; the plating layer thickness and conductivity of the power pin to be detected are extracted by a feature engineering method, the plating layer thickness is used as a one-dimensional feature vector, namely, the plating layer thickness feature vector, and the conductivity is used as a one-dimensional feature vector, namely, the conductivity feature vector; the plating layer thickness feature vector and the conductivity feature vector are weighted. The vector is also used as the input of the feature extraction model; the last layer of the feature extraction model also includes a fully connected layer, which maps the extracted texture, shape and color features to a feature vector of a preset length to obtain a texture feature vector, a shape feature vector and a color feature vector; the texture feature vector, the shape feature vector, the color feature vector and the electroplating layer thickness feature vector are fused through the fully connected layer of the feature extraction model, and the texture feature vector, the shape feature vector, the color feature vector and the conductivity feature vector are fused; the fully connected layer of the feature extraction model outputs the fusion vector of the multi-spectral image to be detected; the principal component analysis method is used to reduce the dimension of the fusion vector; and then the mutual information-based feature screening method is used to calculate the mutual information between the fusion vector after dimension reduction and the preset common defect category of the power pin, and the qualified fusion vector is selected through the preset mutual information threshold as the fusion feature vector of the multi-spectral image to be detected; The fused feature vector of the multi-spectral image to be inspected is input into a pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be inspected; wherein, the defect detection model is established using a support vector machine, and the defect detection model is trained using the fused feature vector of the multi-spectral sample image and preset common defect category labels of the power pin, so as to learn the mapping relationship between the fused features of the multi-spectral sample image and the preset common defect category of the power pin.

2. The method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology according to claim 1, characterized in that: By combining multispectral imaging equipment and line scanning technology, the acquisition parameters are optimized according to the size, shape and structural characteristics of the power pin to be detected, and the image of the electroplating layer to be detected of the power pin to be detected is collected, including: A high-speed moving line scanning camera is used to scan the electroplating layer of the power needle to be detected, and continuous images of the electroplating layer of the power needle to be detected are quickly obtained, wherein the scanning speed and resolution of the line scanning camera are adjusted according to the length and diameter of the power needle to be detected; and a high-resolution multispectral camera is used to simultaneously collect visible light images to be detected and near-infrared images to be detected of the electroplating layer of the power needle to be detected in the visible light and near-infrared bands, wherein the band parameters of the multispectral camera are adjusted according to the color and material of the electroplating layer of the power needle to be detected; the continuous images of the electroplating layer to be detected, the visible light images to be detected and the near-infrared images to be detected are used as the images of the electroplating layer of the power needle to be detected.

3. The method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology as claimed in claim 2, characterized in that: The collected images of the electroplating layer to be inspected are preprocessed using a deep learning driven denoising algorithm and a physical model based illumination correction algorithm to obtain a multispectral image to be inspected, including: Construct an image denoising model based on a deep convolutional neural network architecture; obtain the original sample image of the electroplating layer of the sample power pin through the above-mentioned image acquisition method; mark the noise area and noise pattern in the original sample image of the electroplating layer as a model training set; input the model training set into the image denoising model for training, and use the denoised sample image after removing the marked noise area and noise pattern as the output of the image denoising model to achieve denoising processing of the original sample image of the electroplating layer; then input the acquired image of the electroplating layer to be detected into the image denoising model, and output the denoised image of the electroplating layer to be detected after removing the noise area and noise pattern; The corrected light intensity of any pixel in the denoised image to be detected is calculated by the light intensity calculation formula, and the actual light intensity of any pixel is optimized and corrected by the calculated corrected light intensity until all pixels in the denoised image to be detected have completed the optimized correction of light intensity, and the multi-spectral image to be detected after denoising and light correction preprocessing of the electroplating layer to be detected is obtained; wherein the calculation formula of the light intensity calculation formula is: in, I represents the corrected light intensity; I0 represents the light intensity of the incident light source; d represents the distance between the current pixel and the incident light source; k represents an attenuation factor; wherein the attenuation factor is optimized according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; θ i Indicates the incident angle of the incident light source from the air to the current pixel; θ t It indicates the refraction angle of the incident light source in the electroplating layer after it enters the current pixel from the air; θ represents the angle between the incident direction of the incident light source and the normal of the current pixel; f(θ) represents the angle correlation function; A represents the area element at the current pixel.

4. The method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology as claimed in claim 3 is characterized in that: If the power pin is cylindrical, the angle-dependent function is calculated as follows: in, f side (θ) represents the angle correlation function corresponding to the current pixel point on the side of the cylinder, and its calculation formula is as follows: Where r represents the radius of the cylinder; d0 represents the vertical distance from the incident light source to the axis of the cylinder; f end (θ) represents the angle correlation function corresponding to the current pixel point located at the end face of the cylinder, and its calculation formula is as follows: Among them, θ threshold Indicates the angle threshold between the incident direction of the incident light source and the normal of the current pixel.

5. The method for detecting appearance defects of electroplating layers of power pin products based on computer vision technology according to any one of claims 1 to 4, characterized in that: Also includes: Input the fused feature vector of the multi-spectral image to be detected into the defect detection model, and judge whether the fused feature vector of the multi-spectral image to be detected satisfies the mapping relationship between the fused features of the multi-spectral sample image learned by the defect detection model and the preset common defect categories of the power pin, if not, the defect detection model outputs the detection result that there is no appearance defect in the electroplating layer image to be detected, if yes, the defect detection model outputs the detection result that there is appearance defect in the electroplating layer image to be detected and the defect category; A real-time feedback mechanism is established to promptly feed back the test results to the electroplating production equipment to achieve dynamic adjustment and optimization of the production process.

6. A device for detecting appearance defects of electroplating layer of power pin products based on computer vision technology, characterized in that: include: An image acquisition module is used to optimize acquisition parameters according to the size, shape and structural characteristics of the power pin to be detected by combining multi-spectral imaging equipment and line scanning technology, and to acquire an image of the electroplating layer of the power pin to be detected; An image preprocessing module, used to preprocess the collected image of the electroplating layer to be detected by using a deep learning-driven denoising algorithm and a physical model-based illumination correction algorithm to obtain a multispectral image to be detected; The denoising algorithm learns the noise pattern through a deep convolutional neural network, and the illumination correction algorithm calculates and corrects the illumination intensity of the pixels in the image based on the physical model of light propagation, and optimizes the correction parameters of the illumination correction algorithm according to the shape of the power pin to be detected and the reflection characteristics of the electroplating layer; Feature extraction module, used to build a feature extraction model based on deep convolutional neural network architecture; The denoised sample image is subjected to illumination correction processing by using an illumination intensity calculation formula to obtain a multispectral sample image; the characteristic regions and defect categories related to the preset common defects of the power pin in the multispectral sample image are annotated; the annotated multispectral sample image is input into the feature extraction model for training, and the feature extraction model is optimized by using a random gradient descent algorithm; the feature extraction model is used to automatically extract the texture, shape and color features of multiple levels in the multispectral image to be detected; at the same time, the texture, shape and color features output by the feature extraction model are weighted by combining the spatial attention mechanism and the channel attention mechanism; the plating layer thickness and conductivity of the power pin to be detected are extracted by a feature engineering method, the plating layer thickness is used as a one-dimensional feature vector, namely, the plating layer thickness feature vector, and the conductivity is used as a one-dimensional feature vector, namely, the conductivity feature vector; the plating layer thickness feature vector and the conductivity feature vector are weighted. The vector is also used as the input of the feature extraction model; the last layer of the feature extraction model also includes a fully connected layer, which maps the extracted texture, shape and color features to a feature vector of a preset length to obtain a texture feature vector, a shape feature vector and a color feature vector; the texture feature vector, the shape feature vector, the color feature vector and the electroplating layer thickness feature vector are fused through the fully connected layer of the feature extraction model, and the texture feature vector, the shape feature vector, the color feature vector and the conductivity feature vector are fused; the fully connected layer of the feature extraction model outputs the fusion vector of the multi-spectral image to be detected; the principal component analysis method is used to reduce the dimension of the fusion vector; and then the mutual information-based feature screening method is used to calculate the mutual information between the fusion vector after dimension reduction and the preset common defect category of the power pin, and the qualified fusion vector is selected through the preset mutual information threshold as the fusion feature vector of the multi-spectral image to be detected; A defect detection module is used to input the fused feature vector of the multi-spectral image to be detected into a pre-established defect detection model to detect the appearance defects of the electroplating layer of the power pin in the electroplating layer image to be detected; wherein, the defect detection model is established using a support vector machine, and the defect detection model is trained using the fused feature vector of the multi-spectral sample image and the preset common defect category label of the power pin, so as to learn the mapping relationship between the fused features of the multi-spectral sample image and the preset common defect category of the power pin.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology is implemented as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that: The computer program in the computer storage medium is used to implement the method for detecting appearance defects of the electroplating layer of a power pin product based on computer vision technology as described in any one of claims 1 to 5 when executed by a computer device.

Images