Visual detection method, system and edge computing box

Abstract

The application provides a visual detection method, system and edge computing box. The surface image of a target product is subjected to multi-scale edge detection to generate edge feature maps of different scales. A defect area candidate frame of the surface of the target product is determined according to all the edge feature maps, and the gradient variance of pixels in each edge feature map is determined. The positioning loss of the surface defects of the target product is determined according to the defect area candidate frame and all the gradient variances, and the contour shape gradient of each type of defect on the surface of the target product is determined based on the positioning loss and the fluctuation characteristics of the light intensity around the production line. The influence level of the surface defects of the target product on the target product is determined through the contour shape gradient of each type of defect, and a graded alarm signal of the surface defects of the target product is output based on the influence level. The application can realize dynamic grading alarm of products based on the geometric topological characteristics of product defects.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and more specifically, to a visual inspection method, system, and edge computing box. Background Technology

[0002] Visual inspection is an automated inspection technology based on machine vision technology that simulates the human visual system. It achieves functions such as detection, recognition, measurement, and positioning of target objects through image acquisition, processing, analysis, and understanding. It plays an important role in many fields such as industrial production, logistics, and medical care.

[0003] In the industrial production sector, surface defect detection is a crucial step in ensuring product quality. With the development of intelligent manufacturing, visual inspection systems, due to their advantages such as non-contact operation and high efficiency, have gradually become the mainstream technology for surface defect detection. Traditional visual inspection systems mostly rely on cloud computing centers for data processing and analysis. However, transmitting the acquired image data to a remote cloud computing center not only suffers from data transmission delays, making it difficult to meet the real-time inspection needs of the production line, but also faces data security risks during data transmission. Therefore, edge computing boxes can be deployed at the end of the production line to analyze product images captured by cameras in real time and detect product surface defects. However, when edge computing boxes perform visual inspection analysis, interference factors such as fluctuations in light intensity and vibrations on the production line lead to insufficient accuracy and stability in product surface defect detection, making it impossible to accurately locate and identify product surface defects, thus affecting product quality control and production efficiency. Given that existing detection methods cannot meet the needs of accurate detection in complex industrial environments, how to combine the geometric and topological characteristics of product defects themselves to construct a detection method that can adapt to dynamic interference environments and achieve dynamic graded alarms for products has become a key problem that the industry urgently needs to solve. Summary of the Invention

[0004] This application provides a visual inspection method, system, and edge computing box, which can realize dynamic hierarchical alarms for products based on the defect geometric topological features of the products.

[0005] In a first aspect, this application provides a visual inspection method for defect detection in a visual inspection system, the method comprising the following steps:

[0006] The surface images of the target product are captured by cameras deployed at the edge of the production line, and multi-scale edge detection is performed on the surface images to generate edge feature maps of different scales.

[0007] Feature fusion is performed on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product, and the gradient variance of pixels in each edge feature map is determined.

[0008] The surface defects of the target product are located based on the candidate boxes of the defect region and all gradient variances to obtain the location loss of the surface defects of the target product. Based on the location loss and the fluctuation characteristics of the light intensity around the production line, the contour morphology gradient of each type of defect on the surface of the target product is determined.

[0009] The impact level of the surface defects on the target product is determined by the contour morphology gradient of each type of defect and the vibration intensity of the production line, and a graded alarm signal for the surface defects of the target product is output based on the impact level.

[0010] In some embodiments, performing multi-scale edge detection on the surface image to generate edge feature maps of different scales specifically includes:

[0011] The surface image is preprocessed to obtain a preprocessed surface image;

[0012] The surface image is sampled at multiple scales based on multi-scale space to obtain surface sub-images at different scales;

[0013] Edge detection is performed on each surface sub-image to obtain edge feature maps at different scales.

[0014] In some embodiments, feature fusion of all edge feature maps to obtain candidate bounding boxes for defect regions on the surface of the target product specifically includes:

[0015] Feature fusion is performed on all edge feature maps to obtain a fused feature map;

[0016] The fused feature map is then enhanced to obtain an enhanced feature map;

[0017] Candidate bounding boxes for defect regions on the surface of the target product are extracted from the enhanced feature map.

[0018] In some embodiments, determining the gradient variance of pixels in each edge feature map specifically includes:

[0019] Select an edge feature map as the selected edge feature map, determine the gradient magnitude of each pixel in the selected edge feature map in the horizontal and vertical directions, and then determine the gradient value of each pixel in the selected edge feature map;

[0020] The gradient variance of pixels in the selected edge feature map is determined by the gradient values ​​of all pixels.

[0021] Continue to determine the gradient variance of pixels in the remaining edge feature map.

[0022] In some embodiments, the localization analysis of surface defects of the target product is performed based on the candidate defect region boxes and all gradient variances to obtain the localization loss of the surface defects of the target product, specifically including:

[0023] Obtain the various defect types of the products on the production line;

[0024] The surface defect deviation of the target product is determined based on the candidate defect region box and all defect types.

[0025] All gradient variances are fused to obtain the fused gradient variance of the surface image;

[0026] The location loss of the surface defect of the target product is determined by the surface defect deviation and the fusion gradient variance.

[0027] In some embodiments, determining the contour morphology gradient of each type of defect on the surface of the target product based on the positioning loss and the fluctuation characteristics of the light intensity around the production line specifically includes:

[0028] Determine the fluctuation characteristics of the light intensity around the production line;

[0029] The light compensation coefficient of the photographed products on the production line is determined based on the fluctuation characteristics.

[0030] The surface defects of the target product are morphologically compensated by the light compensation coefficient and the positioning loss, thereby obtaining the contour morphological gradient of each type of defect on the surface of the target product.

[0031] In some embodiments, determining the impact level of surface defects on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line specifically includes:

[0032] Select a type of defect as the selected defect, and normalize the contour morphology gradient of the selected defect to obtain the standardized defect contour morphology gradient.

[0033] Monitor the vibration intensity of the production line;

[0034] The vibration intensity and defect contour morphology gradient obtained from the monitoring are fused to obtain the fused feature vector of the selected defect.

[0035] The confidence score of the selected defects on the surface of the target product is determined based on the fused feature vector.

[0036] Continue to determine the confidence score of the remaining defects on the surface of the target product;

[0037] Determine the impact level range of the target product;

[0038] The impact level of the surface defects of the target product is obtained by judging all the confidence scores through the impact level range.

[0039] Secondly, this application provides a visual inspection system, which includes a surface defect detection unit, the surface defect detection unit comprising:

[0040] The acquisition module is used to acquire surface images of the target product through cameras deployed at the edge of the production line, perform multi-scale edge detection on the surface images, and generate edge feature maps of different scales.

[0041] The processing module is used to perform feature fusion on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product and determine the gradient variance of pixels in each edge feature map.

[0042] The processing module is also used to perform localization analysis on the surface defects of the target product based on the candidate boxes of the defect regions and all gradient variances, to obtain the localization loss of the surface defects of the target product, and to determine the contour morphology gradient of each type of defect on the surface of the target product based on the localization loss and the fluctuation characteristics of the light intensity around the production line.

[0043] The execution module is used to determine the impact level of the surface defects of the target product on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line, and output a graded alarm signal for the surface defects of the target product based on the impact level.

[0044] Thirdly, this application provides an edge computing box, including the aforementioned visual detection system.

[0045] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual detection method.

[0046] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0047] The visual inspection method, system, and edge computing box provided in this application first acquire surface images of the target product using cameras deployed at the edge of the production line. Multi-scale edge detection is then performed on the surface images to generate edge feature maps of different scales. Feature fusion is performed on all edge feature maps to obtain candidate bounding boxes for defect regions on the target product surface, and the gradient variance of pixels in each edge feature map is determined. Based on the candidate bounding boxes and all gradient variances, the surface defects of the target product are located and analyzed to obtain the location loss of the surface defects. Based on the location loss and the fluctuation characteristics of the light intensity around the production line, the contour morphology gradient of each type of defect on the target product surface is determined. The impact level of the surface defects on the target product is determined by the contour morphology gradient of each type of defect and the vibration intensity of the production line. Based on the impact level, a graded alarm signal for the surface defects of the target product is output.

[0048] Therefore, in the surface defect detection process of this application, firstly, feature fusion is performed on all edge feature maps to integrate multi-scale information, enhance the saliency of the defect area, and generate more accurate candidate boxes. The introduction of gradient variance can quantify the stability of edge features, providing a reliable basis for subsequent defect localization. Next, localization analysis is performed through defect area candidate boxes and gradient variance, making full use of defect geometric topology-related information, making the localization analysis more targeted and scientific. The calculation of localization loss can measure the accuracy of localization, providing a basis for subsequent optimization of localization. Furthermore, the contour morphology gradient of the defect is determined by combining the fluctuation characteristics of light intensity around the production line, taking into account the influence of external environmental factors on defect detection. Variations in light intensity can affect the appearance of defect edges, allowing for more accurate extraction of the defect's contour gradient. This further refines the geometric topological feature description of the defect, leading to more precise defect classification and analysis. Then, the influence level is determined using the contour gradient of each defect type and the vibration intensity of the production line. This considers both the defect's own geometric topological characteristics and the impact of the production line's operating environment on the product. The contour gradient reflects the defect's geometric properties, while the production line vibration intensity reduces the potential impact of external factors on product quality. Combining these two factors enables a comprehensive and dynamic assessment of the defect's impact on the product, achieving dynamic evaluation based on the defect's geometric topological features. Finally, a graded alarm signal for the target product's surface defects is output based on the influence level. Using this scheme, dynamic graded alarms for products based on their defect geometric topological features can be achieved. Attached Figure Description

[0049] Figure 1 This is an exemplary flowchart of a visual inspection method according to some embodiments of this application;

[0050] Figure 2This is an exemplary flowchart illustrating the determination of candidate boxes for defect regions according to some embodiments of this application;

[0051] Figure 3 These are example diagrams illustrating defect types according to some embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a surface defect detection unit according to some embodiments of this application;

[0053] Figure 5 This is a schematic diagram of the structure of a computer device implementing a visual inspection method according to some embodiments of this application. Detailed Implementation

[0054] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] refer to Figure 1 The figure is an exemplary flowchart of a visual inspection method according to some embodiments of this application. The visual inspection method 100 mainly includes the following steps:

[0056] In step 101, surface images of the target product are acquired by cameras deployed at the edge of the production line, and multi-scale edge detection is performed on the surface images to generate edge feature maps of different scales.

[0057] In a specific implementation, for example, a camera (such as an industrial-grade CCD or CMOS camera) can be deployed at the edge of the production line. After the production line is started, the camera is activated and a surface image of the target product is captured through the camera. Other methods can also be used to capture images in other embodiments, which are not limited here.

[0058] In some embodiments, multi-scale edge detection of the surface image to generate edge feature maps of different scales can be achieved by the following steps:

[0059] The surface image is preprocessed to obtain a preprocessed surface image;

[0060] The surface image is sampled at multiple scales based on multi-scale space to obtain surface sub-images at different scales;

[0061] Edge detection is performed on each surface sub-image to obtain edge feature maps at different scales.

[0062] In specific implementation, firstly, in the preprocessing stage, the surface image is converted to a grayscale image using the color conversion function of the open-source computer vision library (OpenCV library), and Gaussian filtering is performed using the Gaussian blur function in the open-source computer vision library to eliminate noise interference. The image after noise interference elimination is used as the preprocessed surface image. Secondly, in the multi-scale sampling stage, the Gaussian pyramid algorithm is used to construct a multi-scale space, and the preprocessed surface image is downsampled based on this multi-scale space to generate surface sub-images of different resolutions. That is, one resolution corresponds to one scale, thus obtaining surface sub-images of different scales. For example, the image is downsampled three times to obtain surface sub-images of 1 / 2, 1 / 4, and 1 / 8 resolutions. Finally, in the edge detection stage, the Canny edge detection algorithm is applied to each surface sub-image to extract edge feature maps of the surface sub-images at different scales. In other embodiments, other methods can also be used to determine the edge features, which are not limited here.

[0063] It should be noted that the edge feature map in this application represents the edge information of the target product in the surface image. The edge feature map includes abrupt changes in boundaries, colors, and textures, and is used to determine areas in the surface image where grayscale values ​​change significantly.

[0064] In step 102, feature fusion is performed on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product, and the gradient variance of pixels in each edge feature map is determined.

[0065] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart of determining candidate boxes for defect regions in some embodiments of this application. In this embodiment, feature fusion is performed on all edge feature maps to obtain candidate boxes for defect regions on the surface of the target product, which can be achieved by the following steps:

[0066] In step 1021, feature fusion is performed on all edge feature maps to obtain a fused feature map;

[0067] In step 1022, the fused feature map is enhanced to obtain an enhanced feature map;

[0068] In step 1023, candidate boxes for defect regions on the surface of the target product are extracted from the enhanced feature map.

[0069] In practice, feature fusion is performed on all edge feature maps to obtain the fused feature map. This can be achieved as follows: First, for each scale of edge feature map, a 1×1 convolutional layer is used to adjust the number of channels in each edge feature map, ensuring that all edge feature maps have the same channel dimension for easier subsequent fusion. Then, starting from the lowest-level feature map, bilinear interpolation is used to progressively upsample the low-resolution feature maps, making their size similar to that of adjacent high-resolution feature maps. Figure 1 Then, the upsampled feature map is added element-wise to the high-resolution feature map, and the resulting feature map is used as a fused feature map. The fused feature map represents a feature map that integrates edge feature maps at different levels. The fused feature map can more comprehensively express the edge features of the target product surface than a single edge feature map, which helps to more accurately identify and locate defect areas.

[0070] In specific implementation, feature enhancement is performed on the fused feature map to obtain the enhanced feature map. This can be achieved by combining attention mechanisms and morphological operations to enhance the fused feature map, using the enhanced fused feature map as the enhanced feature map. For example, the attention mechanism (such as the Convolutional Block Attention Module (CBAM)) includes two parts: channel attention and spatial attention. The channel attention part obtains the feature description of the channel dimension through global average pooling and global max pooling, then generates channel attention weights through a multilayer perceptron (MLP), and finally multiplies the weights element-wise with the fused feature map to enhance the feature response of important channels. Spatial attention... The process involves performing average pooling and max pooling on the fused feature map along the channel dimension, concatenating the results, and then passing them through a convolutional layer to generate spatial attention weights. These weights are then multiplied element-wise with the fused feature map to further highlight important spatial regions. In terms of morphological operations, dilation and erosion operations (cv2.morphologyEx in OpenCV) are used. Dilation can fill small holes in defective regions and connect broken edges, while erosion can remove some minor noise and burrs, making defective regions clearer. Through the above operations, an enhanced feature map is obtained. Other enhancement methods can be used in other embodiments, which are not limited here.

[0071] In specific implementation, the extraction of candidate bounding boxes for defect regions on the surface of the target product from the enhanced feature map can be achieved in the following way: based on the enhanced feature map, threshold segmentation and contour detection methods can be used to extract candidate bounding boxes for defect regions. First, the Otsu algorithm is used in conjunction with historical surface image data to automatically determine an optimal threshold for judging defects. Each pixel in the enhanced feature map is traversed, and the gray value of each pixel is compared with the optimal threshold. If the gray value of a pixel is greater than the optimal threshold, then the gray value of that pixel is set to a specified maximum value (usually 255, representing white), representing the foreground part in the enhanced feature map, i.e., the possible defect region. If the gray value of a pixel is less than or equal to the optimal threshold, then the gray value of that pixel is set to 0 (representing black), representing the background part in the enhanced feature map, thereby separating the defect region from the background. Then, the bounding rectangle of each contour of the foreground part is calculated (e.g., using cv2.boundingRect in OpenCV), and all the bounding rectangles are used as candidate bounding boxes for defect regions on the surface of the target product. Other extraction methods can be used in other embodiments, which are not limited here.

[0072] It should be noted that the defect region candidate box in this application refers to the candidate box containing the defect region of the target product in the surface image. It can provide a preliminary area range for defect identification and location of the target product, which is convenient for further analysis and processing of the defect region.

[0073] In some embodiments, determining the gradient variance of pixels in each edge feature map can be achieved using the following steps:

[0074] Select an edge feature map as the selected edge feature map, determine the gradient magnitude of each pixel in the selected edge feature map in the horizontal and vertical directions, and then determine the gradient value of each pixel in the selected edge feature map;

[0075] The gradient variance of pixels in the selected edge feature map is determined by the gradient values ​​of all pixels.

[0076] Continue to determine the gradient variance of pixels in the remaining edge feature map.

[0077] In specific implementation, firstly, the Sobel operator in the existing technology can be used to calculate the gradient magnitude of each pixel in the selected edge feature map in the horizontal and vertical directions. The average gradient magnitude of each pixel in the horizontal and vertical directions is then used as the gradient value of each pixel. Then, the gradient values ​​of all pixels can be substituted into the variance calculation formula, and the calculated variance is used as the gradient variance of the pixels in the selected edge feature map. The gradient variance of each pixel in the selected edge feature map can be obtained in the above way.

[0078] It should be noted that the gradient variance in this application is an indicator that measures the dispersion of pixel gradient values ​​in a selected edge feature map. The gradient variance can reflect the irregularity of edge changes in the selected edge feature map.

[0079] In step 103, the surface defects of the target product are located based on the candidate boxes of the defect regions and all gradient variances to obtain the location loss of the surface defects of the target product. Based on the location loss and the fluctuation characteristics of the light intensity around the production line, the contour morphology gradient of each type of defect on the surface of the target product is determined.

[0080] In some embodiments, the localization analysis of surface defects of the target product based on the candidate defect region boxes and all gradient variances, to obtain the localization loss of surface defects of the target product, can be achieved by the following steps:

[0081] Obtain the various defect types of the products on the production line;

[0082] The surface defect deviation of the target product is determined based on the candidate defect region box and all defect types.

[0083] All gradient variances are fused to obtain the fused gradient variance of the surface image;

[0084] The location loss of the surface defect of the target product is determined by the surface defect deviation and the fusion gradient variance.

[0085] It should be noted that the defect region candidate box gives the approximate range where the defect may exist, while the gradient variance provides detailed information about the features within that region. By taking both factors into account, the specific location and boundary of the defect can be determined more accurately. For example, within a large defect region candidate box, the sub-region with a higher gradient variance is more likely to be the true core defect region, thereby determining the location loss of the surface defect of the target product.

[0086] In this process, historical defect data of products on the production line is collected and organized in advance, and classification and statistical methods are used to determine the various types of defects that may occur on the production line, such as scratches, cracks, and holes. In some embodiments, reference is made to... Figure 3 As shown, this figure is an example diagram of defect types in some embodiments of this application. Black ellipses represent holes, thick black lines represent scratches, and thin black lines represent cracks.

[0087] In specific implementation, determining the surface defect deviation of the target product based on the defect region candidate box and all defect types can be achieved in the following way: For each circumscribed rectangle in the defect region candidate box, select one circumscribed rectangle in the defect region candidate box as the selected circumscribed rectangle. Use an image matching algorithm combined with a defect type database to judge the selected circumscribed rectangle, thereby determining the defect type corresponding to the selected circumscribed rectangle. Use a template matching algorithm in computer vision to calculate the difference between the position and size of the selected circumscribed rectangle and the standard defect feature corresponding to the defect type, thereby obtaining the defect difference of the selected circumscribed rectangle. Continue to determine the defect differences of the remaining circumscribed rectangles in the defect region candidate box, and use all defect differences as the surface defect deviation of the target product. For example, if the defect type is a standard circular hole, the deviation of the area defined by the candidate box from the standard circular hole in terms of position coordinates and radius can be calculated. Other methods can also be used to determine this in other embodiments, which are not limited here.

[0088] The purpose of fusing all gradient variances is to comprehensively utilize the gradient information of each feature map. A single gradient variance can only reflect the features of a feature map at a certain resolution. However, by fusing, gradient variance information at different locations and scales in the image can be integrated. This can more comprehensively describe the overall features of the image and avoid focusing only on local resolution while ignoring the overall resolution. In specific implementation, fusing all gradient variances to obtain the fused gradient variance of the surface image can be achieved in the following way: using Python's NumPy library to calculate a weighted average of all gradient variances, and using the calculated variance as the fused gradient variance of the surface image. The fused gradient variance represents a variance that represents the overall image features obtained after comprehensively processing multiple gradient variances, reflecting the degree of change in pixel grayscale values ​​in the image. Other methods can also be used to determine this in other embodiments, which are not limited here.

[0089] Among them, the surface defect bias reflects the difference between the predicted defect region candidate box and the standard defect features in terms of parameters such as position and size. The larger the value, the higher the deviation between the prediction and the real defect. The fusion gradient variance integrates the gradient information of each region of the image and can reflect the feature difference between the defect region and the normal region. A higher fusion gradient variance usually corresponds to a more obvious defect feature response. Considering the different contributions of these two indicators to defect localization, a loss function L = α × B + β × G is constructed by linear weighted summation, where B represents the surface defect bias, G represents the fusion gradient variance, and α and β are weight coefficients. Cross-validation is used to test on different production conditions and product samples. The weight coefficients are adjusted according to the accuracy indicators of localization analysis (such as recall and precision) to find a set of weight coefficients that can make the localization loss calculation more accurate in different scenarios. In specific implementation, the localization loss of the target product surface defect can be determined by the surface defect bias and the fusion gradient variance in the following way: input the surface defect bias and the fusion gradient variance into the above loss function, and then use the calculated result as the localization loss of the target product surface defect. Other methods can also be used in other embodiments, which are not limited here.

[0090] It should be noted that the positioning loss in this application represents a quantitative indicator of the degree of loss when locating the location of a surface defect of a target product. It can be used as a quantitative indicator to measure the degree of difference between the model prediction result and the actual defect location when locating a surface defect of a target product.

[0091] In some embodiments, determining the contour morphology gradient of each type of defect on the surface of the target product based on the positioning loss and the fluctuation characteristics of the light intensity around the production line can be achieved by the following steps:

[0092] Determine the fluctuation characteristics of the light intensity around the production line;

[0093] The light compensation coefficient of the photographed products on the production line is determined based on the fluctuation characteristics.

[0094] The surface defects of the target product are morphologically compensated by the light compensation coefficient and the positioning loss, thereby obtaining the contour morphological gradient of each type of defect on the surface of the target product.

[0095] It should be noted that light compensation mainly addresses the interference of the lighting environment on the defect morphology, while positioning loss guides the direction of defect morphology correction, so that the corrected defect morphology not only conforms to the image features after light compensation, but also minimizes the positioning error, thereby obtaining an accurate contour morphology gradient of each type of defect on the surface of the target product.

[0096] In specific implementation, the fluctuation characteristics of the light intensity around the production line can be determined in the following way: using a light sensor to collect the light intensity around the production line in real time, arranging all the light intensities in chronological order of collection time, taking the resulting sequence as the light intensity sequence, calculating the difference between each adjacent light intensity in the light intensity sequence, and taking all the differences as the fluctuation characteristics of the light intensity around the production line, wherein the fluctuation characteristics represent the degree of fluctuation of the light intensity around the production line; other methods can also be used in other embodiments, which are not limited here.

[0097] In specific implementation, the light compensation coefficient of the product photographed on the production line can be determined based on the fluctuation characteristics in the following way: a linear regression model is established, and the linear regression model is trained by combining a large amount of historical light intensity data. The fluctuation characteristics are input into the trained linear regression model, and the mapping relationship between light intensity and light compensation coefficient is determined through the linear regression model, thereby obtaining the light compensation coefficient of the product photographed on the production line. The light compensation coefficient represents the parameter of the degree of light compensation when photographing the product on the production line. Other methods can also be used to determine the light compensation coefficient in other embodiments, which are not limited here.

[0098] In specific implementation, the morphological compensation of surface defects of the target product is performed using the light compensation coefficient and the positioning loss to obtain the contour morphological gradient of each type of defect on the surface of the target product. This can be achieved in the following way: using the OpenCV library in Python in combination with the light compensation coefficient, the pixel values ​​in the candidate boxes of the defect region in the surface image are scaled to adjust the brightness. An optimization algorithm based on gradient descent is used in combination with the positioning loss to iteratively adjust the contours of each bounding rectangle in the candidate boxes of the defect region after light compensation. The average gradient of the pixel values ​​of each bounding rectangle in the candidate boxes of the defect region in the corresponding region of the surface image is used as the contour morphological gradient of the corresponding defect type, thereby obtaining the contour morphological gradient of each type of defect on the surface of the target product. Other compensation methods can be used in other embodiments, which are not limited here.

[0099] It should be noted that the contour morphology gradient in this application represents the gradient of the pixel value of the contour morphology of the surface defect type of the target product, which can be used to judge the surface defects of the target product.

[0100] In step 104, the impact level of the surface defects of the target product on the target product is determined by the contour morphology gradient of each type of defect and the vibration intensity of the production line, and a graded alarm signal for the surface defects of the target product is output based on the impact level.

[0101] In some embodiments, determining the impact level of surface defects on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line can be achieved through the following steps:

[0102] Select a type of defect as the selected defect, and normalize the contour morphology gradient of the selected defect to obtain the standardized defect contour morphology gradient.

[0103] Monitor the vibration intensity of the production line;

[0104] The vibration intensity and defect contour morphology gradient obtained from the monitoring are fused to obtain the fused feature vector of the selected defect.

[0105] The confidence score of the selected defects on the surface of the target product is determined based on the fused feature vector.

[0106] Continue to determine the confidence score of the remaining defects on the surface of the target product;

[0107] Determine the impact level range of the target product;

[0108] The impact level of the surface defects of the target product is obtained by judging all the confidence scores through the impact level range.

[0109] It should be noted that quantitative analysis of the contour morphology gradient can preliminarily determine the possibility of defect existence. However, the vibration intensity of the production line, as an important environmental factor, will have a multifaceted impact on defect detection, such as causing instability of the detection equipment and changes in the defect morphology, thereby interfering with the accurate judgment of defects. Combining the two can comprehensively consider the characteristics of the product's own defects as well as the interference of the production environment on detection, and comprehensively weigh the characteristics of the defect itself and environmental interference factors, thereby more accurately assessing the possibility of defect existence and giving a confidence score for each type of defect on the surface of the target product.

[0110] In specific implementation, firstly, the Min-Max normalization method is used to normalize the contour gradient of the selected defect, mapping the contour gradient value of the selected defect to the [0,1] interval. The result of the normalization process is used as the standardized defect contour gradient. For example, if the defect contour gradient includes length and width values, this processing can eliminate the influence of dimensions. Then, a vibration sensor is deployed near the camera on the production line to collect the vibration intensity of the production line in real time. In this embodiment, the vibration intensity when capturing surface images is used. Finally, the monitored vibration intensity can be directly connected to the defect contour gradient vector using a serial splicing method. This forms a new vector. For example, if the dimension of the defect contour morphology gradient vector is n and the vibration intensity is a single value, the dimension of the fused feature vector will be n+1. In Python, the concatenate function of the NumPy library can be used to complete the concatenation operation. Finally, a support vector machine (SVM) model is selected, and the model is trained using historical surface defect data to determine the model parameters (such as kernel function, penalty factor, etc.). After training, the fused feature vector is input into the trained SVM model, which outputs the confidence score of the selected defect by calculating the decision function. In other embodiments, other methods can also be used to determine the confidence score, which is not limited here.

[0111] It should be noted that the confidence score in this application represents the degree of credibility of the existence of defects in the target product, and can be used to judge defects on the surface of the target product.

[0112] In practice, firstly, machine learning algorithms (such as decision trees) are used to establish a correspondence between different confidence score intervals and the impact level of surface defects of the target product, based on a large amount of historical defect data. This yields the impact level interval of the target product. For example, the impact level interval of the target product includes dividing confidence scores of 0-0.3 into low impact level, 0.3-0.7 into medium impact level, and 0.7-1 into high impact level. Then, all confidence scores are iterated through, and each confidence score is compared with the impact level interval to determine its impact level.

[0113] It should be noted that the impact level in this application represents the degree of impact of the defect on the target product, and is used to judge the surface defects of the target product.

[0114] In some embodiments, outputting a graded alarm signal for the surface defect of the target product based on the impact level can be achieved by the following steps:

[0115] The impact level will be input into the edge computing box;

[0116] The edge computing box outputs graded alarm signals for surface defects of the target product.

[0117] In practice, the determined impact level is transmitted to the edge computing box. The edge computing box maps different impact levels to corresponding graded alarm signals according to preset alarm rules. For example, a green alarm signal is output for a low impact level, a yellow alarm signal is output for a medium impact level, and a red alarm signal is output for a high impact level. The alarm signal is then sent to relevant personnel so that appropriate measures can be taken in a timely manner. Other methods can also be used to determine the impact level in other embodiments, which are not limited here.

[0118] Furthermore, in another aspect of this application, in some embodiments, this application provides a visual inspection system including a surface defect detection unit, with reference to... Figure 4 The figure is a schematic diagram of the structure of a surface defect detection unit 400 according to some embodiments of this application. The surface defect detection unit 400 includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0119] The acquisition module 401 in this application is mainly used to acquire surface images of target products through cameras deployed at the edge of the production line, perform multi-scale edge detection on the surface images, and generate edge feature maps of different scales.

[0120] Processing module 402, in this application, is used to perform feature fusion on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product and determine the gradient variance of pixels in each edge feature map.

[0121] It should be noted that the processing module 402 in this application is also used to perform localization analysis on the surface defects of the target product based on the candidate boxes of the defect region and all gradient variances, to obtain the localization loss of the surface defects of the target product, and to determine the contour morphology gradient of each type of defect on the surface of the target product based on the localization loss and the fluctuation characteristics of the light intensity around the production line.

[0122] The execution module 403 in this application is mainly used to determine the impact level of the surface defects of the target product on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line, and output the graded alarm signal of the surface defects of the target product based on the impact level.

[0123] In other aspects of this application, an edge computing box is also provided, deployed at the end of a production line. The edge computing box includes the aforementioned vision inspection system, which can analyze product images captured by a camera in real time and detect product surface defects.

[0124] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described visual inspection method.

[0125] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a visual inspection method according to some embodiments of this application. The visual inspection method in the above embodiments can be implemented through... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0126] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0127] The communication bus 502 can be used to transmit information between the aforementioned components.

[0128] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0129] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method used in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0130] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0131] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0132] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0133] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual detection method.

[0134] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0135] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A visual inspection method for defect detection in a visual inspection system, characterized in that, The method includes the following steps: The surface images of the target product are captured by cameras deployed at the edge of the production line, and multi-scale edge detection is performed on the surface images to generate edge feature maps of different scales. Feature fusion is performed on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product, and the gradient variance of pixels in each edge feature map is determined. The surface defects of the target product are located based on the candidate boxes of the defect region and all gradient variances to obtain the location loss of the surface defects of the target product. Based on the location loss and the fluctuation characteristics of the light intensity around the production line, the contour morphology gradient of each type of defect on the surface of the target product is determined. The impact level of the surface defects on the target product is determined by the contour morphology gradient of each type of defect and the vibration intensity of the production line, and a graded alarm signal for the surface defects of the target product is output based on the impact level. Specifically, determining the contour morphology gradient of each type of defect on the target product surface based on the positioning loss and the fluctuation characteristics of the light intensity around the production line includes: Determine the fluctuation characteristics of the light intensity around the production line; The light compensation coefficient of the photographed products on the production line is determined based on the fluctuation characteristics. The surface defects of the target product are morphologically compensated by the light compensation coefficient and the positioning loss, thereby obtaining the contour morphological gradient of each type of defect on the surface of the target product.

2. The method as described in claim 1, characterized in that, Performing multi-scale edge detection on the surface image to generate edge feature maps of different scales specifically includes: The surface image is preprocessed to obtain a preprocessed surface image; The surface image is sampled at multiple scales based on multi-scale space to obtain surface sub-images at different scales; Edge detection is performed on each surface sub-image to obtain edge feature maps at different scales.

3. The method as described in claim 1, characterized in that, Feature fusion is performed on all edge feature maps to obtain candidate bounding boxes for defect regions on the surface of the target product, specifically including: Feature fusion is performed on all edge feature maps to obtain a fused feature map; The fused feature map is then enhanced to obtain an enhanced feature map; Candidate bounding boxes for defect regions on the surface of the target product are extracted from the enhanced feature map.

4. The method as described in claim 1, characterized in that, Determining the gradient variance of pixels in each edge feature map specifically includes: Select an edge feature map as the selected edge feature map, determine the gradient magnitude of each pixel in the selected edge feature map in the horizontal and vertical directions, and then determine the gradient value of each pixel in the selected edge feature map; The gradient variance of pixels in the selected edge feature map is determined by the gradient values ​​of all pixels. Continue to determine the gradient variance of pixels in the remaining edge feature map.

5. The method as described in claim 1, characterized in that, Based on the candidate boxes of the defect regions and all gradient variances, the surface defects of the target product are located using a localization analysis. The localization loss of the surface defects of the target product specifically includes: Obtain the various defect types of the products on the production line; The surface defect deviation of the target product is determined based on the candidate defect region box and all defect types. All gradient variances are fused to obtain the fused gradient variance of the surface image; The location loss of the surface defect of the target product is determined by the surface defect deviation and the fusion gradient variance.

6. The method as described in claim 1, characterized in that, Determining the impact level of surface defects on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line specifically includes: Select a type of defect as the selected defect, and normalize the contour morphology gradient of the selected defect to obtain the standardized defect contour morphology gradient. Monitor the vibration intensity of the production line; The vibration intensity and defect contour morphology gradient obtained from the monitoring are fused to obtain the fused feature vector of the selected defect. The confidence score of the selected defects on the surface of the target product is determined based on the fused feature vector. Continue to determine the confidence score of the remaining defects on the surface of the target product; Determine the impact level range of the target product; The impact level of the surface defects of the target product is obtained by judging all the confidence scores through the impact level range.

7. A visual inspection system, comprising a surface defect detection unit, wherein the visual inspection system performs visual inspection using the method described in any one of claims 1 to 6, characterized in that, The surface defect detection unit includes: The acquisition module is used to acquire surface images of the target product through cameras deployed at the edge of the production line, perform multi-scale edge detection on the surface images, and generate edge feature maps of different scales. The processing module is used to perform feature fusion on all edge feature maps to obtain candidate boxes of defect areas on the surface of the target product and determine the gradient variance of pixels in each edge feature map. The processing module is also used to perform localization analysis on the surface defects of the target product based on the candidate boxes of the defect regions and all gradient variances, to obtain the localization loss of the surface defects of the target product, and to determine the contour morphology gradient of each type of defect on the surface of the target product based on the localization loss and the fluctuation characteristics of the light intensity around the production line. The execution module is used to determine the impact level of the surface defects of the target product on the target product by using the contour morphology gradient of each type of defect and the vibration intensity of the production line, and output a graded alarm signal for the surface defects of the target product based on the impact level.

8. An edge computing box, characterized in that, Includes the visual inspection system as described in claim 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the visual detection method as described in any one of claims 1 to 6.

Images