Appearance defect detection method, device, equipment, storage medium and product

Abstract

This application provides a method, apparatus, device, storage medium, and product for detecting appearance defects, including: receiving image data of a product appearance sent by an image acquisition device, and preprocessing the image data to obtain processed image data; extracting texture features of the product appearance from the processed image data; extracting color features of the product appearance from the processed image data; fusing the texture features and color features to obtain fused features of the product appearance; and inputting the fused features into a pre-trained defect recognition model to output defect detection results of the product appearance; wherein the defect recognition model is a detection Transformer model trained using historical product appearance defect images to improve detection accuracy.

Description

Technical Field

[0001] This application relates to the field of defect identification technology, and in particular to a method, apparatus, equipment, storage medium and product for detecting appearance defects. Background Technology

[0002] In modern manufacturing, the appearance quality of products has a crucial impact on their market competitiveness and customer satisfaction. With the continuous expansion of production scale and the increasing demands of consumers for product quality, efficient and accurate product appearance defect detection has become an indispensable part of the industrial production process.

[0003] In existing technologies, cameras are used to capture images of the product surface, which are then analyzed using preset image processing algorithms to identify potential defects. However, early defect detection methods suffer from low detection accuracy when faced with complex and diverse defect types. Summary of the Invention

[0004] This application provides methods, apparatus, equipment, storage media, and products for detecting appearance defects, in order to improve detection accuracy.

[0005] In a first aspect, embodiments of this application provide a method for detecting appearance defects, applied to a computer device, comprising: receiving image data of a product appearance sent by an image acquisition device, and preprocessing the image data to obtain processed image data; extracting texture features of the product appearance from the processed image data; extracting color features of the product appearance from the processed image data; fusing the texture features and color features to obtain fused features of the product appearance; and inputting the fused features into a pre-trained defect recognition model to output a defect detection result of the product appearance; wherein the defect recognition model is a detection Transformer model trained using historical product appearance defect images.

[0006] In one possible implementation, extracting texture features of the product appearance from the processed image data includes: converting the processed image data into a grayscale image; traversing each pixel of the grayscale image and calculating the grayscale value relationship between each pixel and its neighboring pixels in a preset direction and position; updating the frequency of occurrence of the grayscale value relationship in a predefined grayscale co-occurrence matrix; extracting feature vectors from the grayscale co-occurrence matrix of the grayscale image; and determining the feature vectors as texture features of the product appearance.

[0007] In one possible implementation, extracting color features of the product appearance from the processed image data includes: obtaining the RGB color space of the processed image; calculating the HSV color space of the processed image based on the RGB color space; dividing the HSV color space into a predetermined number of intervals; traversing each pixel of the processed image and determining the interval to which each pixel belongs in the HSV color space; constructing a color histogram based on the interval to which each pixel belongs; calculating the color moments of the processed image based on the color histogram to obtain the statistical features of the color distribution; and clustering the statistical features of each pixel using a preset clustering algorithm to quantify the color features of the product appearance.

[0008] In one possible implementation, preprocessing the image data to obtain processed image data includes: identifying product edges in the image data using a preset algorithm; removing the image background based on the product edges using a preset image segmentation technique to obtain cropped image data; denoising the cropped image data to obtain denoised image data; and enhancing the denoised image data to obtain processed image data.

[0009] In one possible implementation, the process of constructing a defect identification model includes: constructing an initial defect identification model; annotating historical product appearance defect images to obtain annotated historical images; dividing the annotated historical images into a training set and a test set; training the initial defect identification model using the training set; testing the trained defect identification model using the test set to obtain test results; and adjusting the parameters of the trained defect identification model using cross-validation based on the test results to obtain the defect identification model.

[0010] In one possible implementation, after outputting the defect detection results of the product appearance, the method further includes: locating the defective product to obtain the product's location information; controlling a robotic arm to grasp the product based on the location information; classifying the product according to the defect detection results; repairing the product according to a preset repair plan if the product is repairable; and scrapping the product if it is irreparable.

[0011] Secondly, embodiments of this application provide an appearance defect detection device, applied to computer equipment, comprising:

[0012] The image preprocessing module is used to receive image data of the product appearance sent by the image acquisition device, and to preprocess the image data to obtain processed image data.

[0013] The texture feature extraction module is used to extract the texture features of the product appearance from the processed image data.

[0014] The color feature extraction module is used to extract the color features of the product appearance from the processed image data.

[0015] The feature fusion module is used to fuse texture features and color features to obtain the fused features of the product appearance.

[0016] The defect recognition module is used to input fused features into a pre-trained defect recognition model to output defect detection results for the product appearance; the defect recognition model is a detection Transformer model trained using historical product appearance defect images.

[0017] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0019] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0020] The appearance defect detection method, apparatus, device, storage medium, and product provided in this application embodiment can capture subtle changes in the surface texture of a product based on the gray-level co-occurrence matrix. Through multi-step analysis in the HSV color space, the color distribution and characteristics can be grasped. At the same time, the features after fusing texture features and color features cover richer product appearance information. The detection Transformer model trained based on historical product appearance defect images is used as the defect recognition model, which can handle complex and minute defects, resulting in higher detection accuracy. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 A schematic diagram of a scenario for the appearance defect detection method provided in the embodiments of this application;

[0023] Figure 2 A schematic flowchart illustrating the appearance defect detection method provided in this application embodiment;

[0024] Figure 3This is a schematic diagram of the appearance defect detection device provided in the embodiments of this application;

[0025] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0026] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] First, let me explain the terms used in this application:

[0029] GLCM (Gray-Level Co-occurrence Matrix) is a tool for analyzing image texture features. It describes image texture information by counting the frequency of simultaneous occurrences of pixels with gray value i and gray value j at a given direction and distance. Essentially, GLCM is a two-dimensional matrix where the row and column indices correspond to different gray levels. Each element (i, j) in the matrix represents the number of times pixels with gray values ​​i and j appear at a specific relative position in the image.

[0030] The HSV color space is a color model based on human visual perception.

[0031] To clearly understand the technical solution of this application, the existing technology solutions will first be described in detail. In modern manufacturing, the appearance of a product has a crucial impact on its market competitiveness and customer satisfaction. With the continuous expansion of production scale and the increasing demands of consumers for product quality, efficient and accurate product appearance defect detection has become an indispensable part of the industrial production process. With the rapid development of computer technology, image processing technology, and machine vision technology, product appearance defect detection methods based on visual analysis have gradually become a research and application hotspot. Image acquisition devices are used to acquire images of the product's appearance, and then image processing algorithms are used to analyze and process the images to automatically identify various defects on the product surface. However, for some subtle, hidden, or textured defects, the sensitivity and accuracy of detection remain relatively low.

[0032] To address the aforementioned technical problems, the inventors devised a method that integrates deep learning models and computer vision technology. This involves extracting texture features to accurately capture subtle changes in the surface texture of products, and extracting color features to effectively detect defects such as color deviations, blemishes, and uneven color distribution. The fusion of texture and color features allows for multi-dimensional evaluation of product appearance, overcoming the limitations of using texture or color features alone. Furthermore, the introduction of the DETR (Detection Transformer) model enables more precise handling of complex and minute defects.

[0033] Based on the above-mentioned inventive discovery, the inventor has proposed the technical solution of this application.

[0034] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0035] Figure 1 This is a schematic diagram of a scenario for the appearance defect detection method provided in the embodiments of this application, such as... Figure 1 As shown, the specific application scenarios of this application embodiment include: image acquisition device 101 and computer device 102.

[0036] The image acquisition device 101 typically employs equipment with high-resolution imaging capabilities, such as industrial cameras or high-definition cameras. The image acquisition device 101 is installed on the product production line, and its position and shooting angle are precisely calibrated to acquire product appearance images that meet the requirements for subsequent analysis.

[0037] Specifically, the image acquisition device 101 acquires image data of the product appearance on the product line in real time and sends the image data of the product appearance to the computer device 102. The computer device 102 preprocesses the image data to obtain processed image data, extracts the texture features and color features of the product appearance from the processed image data, and fuses them to obtain fused features. The fused features are then input into a pre-trained defect recognition model to output the defect detection results of the product appearance.

[0038] Figure 2 This is a schematic flowchart of the appearance defect detection method provided in the embodiments of this application, as shown below. Figure 2 As shown, the method includes:

[0039] S201: Receive image data of the product appearance sent by the image acquisition device, and preprocess the image data to obtain processed image data.

[0040] Specifically, high-precision cameras or sensors are used to capture real-time images of the product appearance on the production line. The product edges in the image data are identified, and the background is removed from the image data based on these edges. The cropped image data is then denoised and enhanced to obtain the processed image data. The specific process includes:

[0041] Sa1: Uses a preset algorithm to identify product edges in image data.

[0042] Specifically, the Canny algorithm is used to identify product edges in image data. The Canny algorithm is capable of accurately locating the edge information of objects in an image.

[0043] Sa2: Based on the product edge, use preset image segmentation technology to remove the image background and obtain cropped image data.

[0044] Specifically, the Canny algorithm is used to obtain the product edge thickness, the gradient magnitude information of edge pixels is analyzed, the distribution of gradient magnitudes of edge pixels is statistically analyzed, and statistical measures such as the mean and standard deviation of gradient magnitudes are calculated. Based on these statistical measures, an appropriate threshold range is set to distinguish pixels in edge regions from background regions. Pixels in the image with gradient magnitudes greater than the high threshold are initially identified as product edge pixels or pixels within the product region, while pixels with gradient magnitudes less than the low threshold are identified as background pixels. After threshold segmentation, edge thinning algorithms, such as morphological thinning operations, are used to refine the segmented edge regions.

[0045] Sa3: Denoise the cropped image data to obtain the denoised image data.

[0046] Specifically, Gaussian filtering or termination filtering is used to remove noise from the cropped image data to obtain denoised image data.

[0047] Sa4: Enhances the denoised image data to obtain the processed image data.

[0048] Specifically, by adjusting the image contrast and brightness, the visual effect of the image is enhanced, resulting in processed image data.

[0049] S202: Extract texture features of the product appearance from the processed image data.

[0050] Specifically, the texture features of the product appearance are extracted using a gray-level co-occurrence matrix. The specific steps include:

[0051] Sb1: Converts the processed image data into a grayscale image.

[0052] Specifically, a weighted average method is used, assigning different weight coefficients based on the differences in sensitivity to red, green, and blue colors. The weight coefficients are set as follows: red channel weight 0.299, green channel weight 0.587, and blue channel weight 0.114. The corresponding grayscale value is calculated for each pixel in the processed image, thereby converting the entire color processed image into a grayscale image.

[0053] Sb2: Traverse each pixel of the grayscale image and calculate the relationship between the grayscale value of each pixel and its neighboring pixels in a preset direction and position.

[0054] Specifically, after converting to a grayscale image, in order to construct the grayscale co-occurrence matrix, it is necessary to traverse every pixel in the image to obtain the grayscale value relationship between each pixel and its specific neighboring pixels. Here, the preset direction and position are key parameters that are determined in advance. Usually, multiple directions such as 0°, 45°, 90° and 135° are selected, as well as a certain distance, such as 1 pixel unit.

[0055] For example, the 0° direction, i.e., the horizontal rightward direction, is set to a distance of 1 pixel unit. For any pixel (x, y) in a grayscale image, its neighboring pixel is the pixel with coordinates (x+1, y) that is 1 pixel away from it in that direction. The grayscale values ​​of these two pixels are compared. If the grayscale value of the current pixel is i and the grayscale value of the neighboring pixel is j, the grayscale value relationship of this pair of pixels is determined to be (i, j). The same method is used to calculate the grayscale value relationship between each pixel and its corresponding neighboring pixels in other preset directions.

[0056] Sb3: Updates the frequency of occurrence of gray-level value relationships in the predefined gray-level co-occurrence matrix.

[0057] Specifically, before traversing the pixels, a gray-level co-occurrence matrix (GLCM) needs to be predefined. The row and column indices correspond to different gray levels, and the size of the matrix is ​​determined by the set number of gray levels. For example, if the number of gray levels is set to N, then the GLCM is an N×N square matrix. After traversing each pixel of the grayscale image and calculating its gray-level value relationship with neighboring pixels, the frequency of this gray-level value relationship in the GLCM is updated. Whenever a specific gray-level value relationship (i,j) occurs, the element value at the corresponding (i,j) position in the GLCM matrix is ​​incremented by 1, indicating that the gray-level value combination has occurred again.

[0058] Sb4: Extracts feature vectors from the gray-level co-occurrence matrix of a grayscale image.

[0059] Among them, multiple statistical measures that can characterize the texture features of an image can be extracted from the gray-level co-occurrence matrix. Commonly used eigenvalues ​​include energy, entropy, contrast, correlation, and uniformity. These statistical measures are combined to form the feature vector.

[0060] Energy is calculated by summing the squares of the elements in the gray-level co-occurrence matrix. Entropy measures the load on image texture; high entropy indicates complex texture with frequent combinations of different gray values, while low entropy indicates simple texture. Contrast reflects the clarity of image texture; images with high contrast have significant gray value variations, which in GLCM manifests as larger values ​​on the diagonal. Correlation reflects the similarity of image texture; its calculation is relatively complex, usually obtained by calculating the linear relationship between pixel pairs, with a value ranging from -1 to 1. Values ​​close to 1 indicate similar gray value trends for pixel pairs, while values ​​close to -1 indicate opposite trends. Uniformity reflects the uniformity of texture and is related to energy; high uniformity means the image texture is more evenly distributed in gray levels.

[0061] Specifically, the formula for calculating energy is:

[0062]

[0063] The formula for calculating entropy is:

[0064]

[0065] The formula for calculating contrast is:

[0066]

[0067] The formula for calculating uniformity is:

[0068]

[0069] In the formula, P(i,j) represents the element value at position (i,j) in GLCM, Energy represents energy, Entropy represents entropy, Contrast represents contrast, and Homogeneity represents uniformity.

[0070] Sb5: Define the feature vector as the texture features of the product appearance.

[0071] S203: Extract color features of the product appearance from the processed image data.

[0072] Specifically, the image is converted from the RGB color space to the HSV color space, a color histogram is calculated and analyzed to obtain an overview of the color distribution, color moments are calculated to obtain the statistical characteristics of the color distribution, and the main colors of the image are identified through clustering algorithms, thereby quantifying color features. The specific steps include:

[0073] Sc1: Obtain the RGB color space of the processed image.

[0074] Sc2: Calculates the hue-saturation-brightness values ​​of the processed image based on the RGB color space in the HSV color space.

[0075] Specifically, the formula involved in converting from RGB to HSV is:

[0076]

[0077]

[0078]

[0079] In the formula, H represents the hue of the HSV color space, S represents the saturation of the HSV color space, and V represents the lightness of the HSV color space. R represents the red channel, G represents the green channel, and B represents the blue channel.

[0080] Sc3: Within the HSV color space, divide the HSV color space into a predetermined number of intervals.

[0081] Specifically, the entire HSV color space is divided into several intervals. For the hue (H) dimension, since its value range is 0°-360°, it can typically be divided into 180 or more intervals (e.g., one interval every 2°) to more accurately capture the distribution differences of different hues; the saturation (S) and lightness (V) dimensions, whose value range is generally 0-1, can each be divided into about 25 intervals (e.g., divided at equal intervals, with each interval having a width of 0.04). This division of the HSV color space forms a multi-dimensional interval grid.

[0082] Sc4: Traverses each pixel of the processed image and determines the range of each pixel in the HSV color space.

[0083] Specifically, after dividing the HSV color space into intervals, it is necessary to traverse and access each pixel in the processed image. For each pixel's corresponding HSV value, determine which interval its hue (H), saturation (S), and lightness (V) fall into.

[0084] For example, if the hue value of a pixel is 120°, it can be determined that it belongs to the 60th interval of the hue dimension (assuming each interval is 2°) according to the interval division of the hue dimension; if its saturation value is 0.6, it can be determined that it belongs to the 15th interval of the saturation dimension according to the interval division of the saturation dimension (assuming each interval width is 0.04); similarly, the interval to which it belongs in the lightness dimension can be determined.

[0085] Sc5: Construct a color histogram based on the range to which each pixel belongs.

[0086] Specifically, after traversing all pixels in the image and determining their respective intervals, for each pixel's specific interval combination (such as hue interval m, saturation interval n, and brightness interval k), add 1 to the corresponding histogram array element position to indicate that the color represented by that interval combination appeared once in the image.

[0087] Sc6: Calculate the color moments of the processed image based on the color histogram to obtain the statistical characteristics of the color distribution.

[0088] Color moments are a way to quantitatively describe the color features of an image from a statistical perspective. They are achieved by calculating the first, second, and third moments to reflect the statistical characteristics of color, such as mean, variance, and skewness.

[0089] Sc7: A preset clustering algorithm is used to cluster the statistical features of each pixel to quantify the color features of the product's appearance.

[0090] For example, using the K-Means clustering algorithm, the number of clusters is first determined. For product images with a single color, 3-5 clusters can be used; for images with rich colors, K can be set to 8-10. Then, K cluster centers are randomly initialized. The statistical feature vector corresponding to each pixel is traversed, and the distance between the pixel and each cluster center is calculated. The pixel is then assigned to the cluster represented by the nearest cluster center. This process of cluster assignment and cluster center update is repeated until the cluster centers no longer change significantly. Color richness is measured by calculating the proportion of the dominant color in the image, color contrast is judged by analyzing the differences in color moment features between the dominant colors of different categories, and color uniformity is evaluated by observing the uniformity of the distribution of each category in the image.

[0091] S204: Merge texture features and color features to obtain the fused features of the product appearance.

[0092] Specifically, the texture feature vector and color feature vector are first normalized. The normalized texture feature vector and color feature vector are then concatenated together in sequence to form a new, longer feature vector, which serves as the fusion feature of the product appearance.

[0093] S205: Input the fused features into the pre-trained defect recognition model to output the defect detection results of the product appearance; wherein the defect recognition model is a detection Transformer model trained using historical product appearance defect images.

[0094] Specifically, a DETR model is constructed; historical product appearance defect images are collected, and the DETR model is trained and optimized using these images to obtain a defect recognition model; based on the defect recognition model, the fused features are identified to obtain the appearance defects.

[0095] DETR (Detection Transformer) is an end-to-end object detection framework based on the Transformer architecture, comprising a backbone feature extraction network, an encoder, a decoder, and a prediction head. The backbone feature extraction network typically uses a CNN as the backbone to extract image features, such as ResNet-50 or ResNet-101, and performs dimensionality reduction through 1x1 convolutions. The encoder uses a multi-head self-attention mechanism and a feedforward network (FFN) to encode features into a sequential form, and introduces positional encoding to preserve spatial information. The decoder also uses multi-head self-attention and encoder-decoder attention mechanisms, interacting with the encoder's output and generating prediction results through object queries. The prediction head includes a classifier and a bounding box regressor, used to ultimately determine the category and location of the predicted bounding box.

[0096] Specifically, in this embodiment, the process of constructing the defect identification model includes:

[0097] Sd1: Construct the initial defect identification model.

[0098] Specifically, by setting key parameters in each component, such as the number of heads in the multi-head self-attention mechanism, the number of neurons in each layer, and the feature dimension, an initial Transformer model framework suitable for product appearance defect recognition tasks is built.

[0099] Sd2: Annotate historical product appearance defect images to obtain annotated historical images.

[0100] Specifically, images of historical product appearance defects are classified according to pre-established defect classification standards.

[0101] Sd3: Divide the labeled historical images into training and testing sets.

[0102] For example, if there are 1,000 labeled historical images, and they are randomly divided in an 8:2 ratio, then 800 images are selected by random sampling to form the training set, and the remaining 200 images form the test set.

[0103] Sd4: Train the initial defect identification model using the training set.

[0104] Specifically, the image data in the training set is preprocessed, including normalizing the image size to a fixed size suitable for the model, and using data augmentation techniques such as rotation, flipping, and adding appropriate noise to expand data diversity and improve the model's generalization ability. Then, the processed images are grouped and loaded according to a predetermined batch size. Next, the fused features corresponding to each batch of images are input into the model. According to the model architecture, the multi-head self-attention mechanism in the encoder first mines feature associations and extracts deep semantic information, then further processes them through a feedforward neural network, interacting and collaborating with the decoder to strengthen defect-related features. Finally, the prediction head outputs the predicted defect category probability distribution and location coordinate information.

[0105] Sd5: Test the trained defect recognition model using the test set to obtain the test results.

[0106] Specifically, after preprocessing, the fusion features corresponding to the test images are sequentially input into the trained defect recognition model to obtain the detection results.

[0107] Sd6: Based on the test results, the parameters of the trained defect recognition model are adjusted using cross-validation to obtain the defect recognition model.

[0108] Specifically, the K-fold cross-validation method is used to adjust the parameters of the trained defect recognition model based on the test results to obtain the final model. First, the entire set of labeled historical images is divided into K subsets. Then, K rounds of training and validation are performed. In each round, K-1 subsets are selected as the new training set, and the remaining subset is used as the validation set. The model is retrained using the new training set and validated on the validation set to obtain validation metrics such as accuracy. Based on these results, the model performance is analyzed, and key parameters such as the number of multi-head self-attention heads, the number of neurons in each layer, and the learning rate are adjusted. Through multiple rounds of iteration and parameter adjustment, the optimal parameter configuration for model performance is determined, thus obtaining the defect recognition model.

[0109] In summary, the gray-level co-occurrence matrix can capture subtle changes in the surface texture of a product. Through multi-step analysis in the HSV color space, color distribution and characteristics can be grasped. At the same time, the features after fusing texture and color features cover richer product appearance information. Using a detection Transformer model trained on historical product appearance defect images as a defect recognition model can handle complex and minute defects, resulting in higher detection accuracy.

[0110] In another embodiment of this application, after outputting the defect detection results of the product appearance, a process for recycling, repairing, or scrapping the product based on the detection results is also included, specifically including:

[0111] S301: Locate the defective product and obtain its location information.

[0112] Specifically, the exact location of defective products is determined by using a positioning system or specialized position detection device in conjunction with the image acquisition equipment. When acquiring product images, the image acquisition equipment simultaneously records the product's relative coordinate information within the image. By using a pre-established and calibrated conversion model between image coordinates and actual physical coordinates, the two-dimensional coordinate information in the image is converted into three-dimensional coordinates (X, Y, Z coordinates) in actual physical space, thereby accurately determining the product's position on the production line or within the inspection area.

[0113] S302: Based on location information, control the robotic arm to grasp products.

[0114] Specifically, based on the acquired location information of the defective product, the computer equipment issues a gripping command to the control system of the robotic arm. The end effector of the robotic arm dynamically adjusts the opening range, gripping force, and gripping angle of the grippers according to the contour shape of the product, thereby transporting the defective product from its original position to the subsequent sorting and processing area.

[0115] S303: Classify products based on the results of defect detection.

[0116] Specifically, after defective products are transferred to the sorting and processing area by the robotic arm, they are classified and judged according to the severity and type of defects based on the detailed defect detection results. Products with defects that are merely slight deviations in color from the standard color, or minor scratches or abrasions on the surface that do not affect the product's structural strength or normal function, are classified as repairable. Products with more serious cracks, damage, large-area material deformation, or other defects that seriously affect the product's quality and functional integrity, such as severe wear on critical parts or breakage of structural components, are judged as irreparable.

[0117] S304: If the product is repairable, repair it according to the preset repair plan.

[0118] Specifically, for products deemed repairable, repair work is carried out according to pre-established repair plans for different types of defects. For example, when the product's appearance color differs from the standard, the product surface is first cleaned and pre-treated to remove impurities and oil. Then, depending on the specific color difference, paint or dye matching the product's standard color is mixed, and professional spraying equipment or dyeing processes are used to recolor the product. During the process, the uniformity, thickness, and accuracy of the color application are strictly controlled to ensure that the repaired product's color is highly consistent with the standard color and has a natural and aesthetically pleasing appearance. For products with scratches, depending on the depth and width of the scratches, sandpaper or polishing tools of appropriate grit are first used to finely polish the scratched areas, gradually removing the scratches and restoring the surface to a smooth finish, followed by polishing. After repair, a second inspection is conducted, and only products that pass the inspection are allowed to re-enter the production process.

[0119] S305: If the product is irreparable, it shall be scrapped.

[0120] Specifically, products that are determined to be irreparable will be scrapped.

[0121] In summary, by locating and classifying defective products, products with appearance defects can be quickly and accurately identified on the production line, and then diverted to the appropriate processing path based on the characteristics of the defects. For repairable products, timely repair allows them to re-enter the production process or finished product sequence after meeting quality standards, effectively reducing the number of products mistakenly discarded due to appearance flaws and improving the overall product pass rate and output quality. The robotic arm, based on precise position information, automates the grasping of defective products, greatly shortening the transfer time between the defect detection stage and subsequent processing stages, avoiding production delays caused by the inefficiency and instability of manual operation.

[0122] Figure 3 This is a schematic diagram of the appearance defect detection device provided in an embodiment of this application. Figure 3 As shown, the device includes: an image preprocessing module 301, a texture feature extraction module 302, a color feature extraction module 303, a feature fusion module 304, and a defect recognition module 305.

[0123] The image preprocessing module 301 is used to receive image data of the product appearance sent by the image acquisition device, and to preprocess the image data to obtain processed image data.

[0124] The texture feature extraction module 302 is used to extract the texture features of the product appearance from the processed image data.

[0125] The color feature extraction module 303 is used to extract the color features of the product appearance from the processed image data.

[0126] The feature fusion module 304 is used to fuse texture features and color features to obtain fused features of the product appearance.

[0127] The defect recognition module 305 is used to input the fused features into a pre-trained defect recognition model to output the defect detection results of the product appearance; wherein the defect recognition model is a detection Transformer model trained using historical product appearance defect images.

[0128] In one possible implementation, the texture feature extraction module 302 is specifically used to convert the processed image data into a grayscale image; traverse each pixel of the grayscale image and calculate the grayscale value relationship between each pixel and its neighboring pixels in a preset direction and position; update the occurrence frequency of the grayscale value relationship in a predefined grayscale co-occurrence matrix; extract feature vectors from the grayscale co-occurrence matrix of the grayscale image; and determine the feature vectors as the texture features of the product appearance.

[0129] In one possible implementation, the color feature extraction module 303 is specifically used to obtain the RGB color space of the processed image; calculate the HSV color space of the processed image based on the RGB color space; divide the HSV color space into a predetermined number of intervals; traverse each pixel of the processed image and determine the interval to which each pixel belongs in the HSV color space; construct a color histogram based on the interval to which each pixel belongs; calculate the color moments of the processed image based on the color histogram to obtain the statistical characteristics of the color distribution; and cluster the statistical characteristics of each pixel using a preset clustering algorithm to quantify the color characteristics of the product appearance.

[0130] In one possible implementation, the image preprocessing module 301 is specifically used to identify product edges in image data using a preset algorithm; based on the product edges, remove the image background using a preset image segmentation technique to obtain cropped image data; perform denoising processing on the cropped image data to obtain denoised image data; and perform enhancement processing on the denoised image data to obtain processed image data.

[0131] In one possible implementation, the device further includes a module building module for constructing an initial defect recognition model; annotating historical product appearance defect images to obtain annotated historical images; dividing the annotated historical images into a training set and a test set; training the initial defect recognition model using the training set; testing the trained defect recognition model using the test set to obtain test results; and adjusting the parameters of the trained defect recognition model using cross-validation based on the test results to obtain the defect recognition model.

[0132] In one possible implementation, the device further includes a processing module for locating the defective product and obtaining its location information; controlling a robotic arm to grasp the product based on the location information; classifying the product according to the defect detection results; repairing the product according to a preset repair plan if the product is repairable; and scrapping the product if it is irreparable.

[0133] The appearance defect detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0134] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application. For example... Figure 4 As shown, the computer device provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the computer device further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0135] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0136] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0137] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0138] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0139] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0140] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0141] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0142] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0143] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0144] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0147] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0149] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An appearance defect detection method characterized by, Applied to computer equipment, including: Receive image data of the product appearance sent by the image acquisition device, and preprocess the image data to obtain processed image data; Extract the texture features of the product's appearance from the processed image data; Extract the color features of the product's appearance from the processed image data; The texture features and the color features are fused together to obtain the fused features of the product appearance; The fused features are input into a pre-trained defect recognition model to output the defect detection results of the product appearance; wherein the defect recognition model is a detection Transformer model trained using historical product appearance defect images.

2. The method of claim 1, wherein, Extracting the texture features of the product appearance from the processed image data includes: The processed image data is converted into a grayscale image; Traverse each pixel of the grayscale image and calculate the relationship between the grayscale value of each pixel and its neighboring pixels at a preset direction and position; The frequency of occurrence of the gray-level value relationships is updated in the predefined gray-level co-occurrence matrix; For the gray-level co-occurrence matrix of the gray-level image, extract feature vectors from the gray-level co-occurrence matrix; The feature vector is determined as the texture feature of the product appearance.

3. The method of claim 1, wherein, Extracting the color features of the product appearance from the processed image data includes: Obtain the RGB color space of the processed image; The hue-saturation-brightness values ​​of the processed image are calculated based on the RGB color space in the HSV color space. Within the HSV color space, the HSV color space is divided into a predetermined number of intervals; Traverse each pixel of the processed image and determine the range of each pixel in the HSV color space; Construct a color histogram based on the range to which each pixel belongs; Based on the color histogram, the color moments of the processed image are calculated to obtain the statistical characteristics of the color distribution; A preset clustering algorithm is used to cluster the statistical features of each pixel to quantify the color features of the product's appearance.

4. The method of claim 1, wherein, The preprocessing of the image data to obtain processed image data includes: The product edges in the image data are identified using a preset algorithm; Based on the product edge, a preset image segmentation technique is used to remove the image background to obtain cropped image data; The cropped image data is then denoised to obtain denoised image data. The denoised image data is enhanced to obtain the processed image data.

5. The method of claim 1, wherein, The process of constructing the defect identification model includes: Construct an initial defect identification model; The historical product appearance defect images are annotated to obtain an annotated historical image; The labeled historical images are divided into a training set and a test set; The initial defect identification model is trained using the training set. The trained defect identification model was tested using the test set, and the test results were obtained. Based on the test results, the parameters of the trained defect identification model are adjusted using cross-validation to obtain the defect identification model.

6. The method of claim 1, wherein, After outputting the defect detection results of the product appearance, the method further includes: The defective product is located to obtain its location information; Based on the location information, the robotic arm is controlled to grasp the product; The products are classified according to the defect detection results. If the product is repairable, it will be repaired according to the preset repair plan; If the product is irreparable, it shall be scrapped.

7. An appearance defect detection apparatus characterized by comprising: Applied to computer equipment, including: The image preprocessing module is used to receive image data of the product appearance sent by the image acquisition device, and to preprocess the image data to obtain processed image data. A texture feature extraction module is used to extract the texture features of the product appearance from the processed image data; A color feature extraction module is used to extract the color features of the product appearance from the processed image data; The feature fusion module is used to fuse the texture features and the color features to obtain the fused features of the product appearance; The defect identification module is used to input the fused features into a pre-trained defect identification model to output the defect detection results of the product appearance; wherein the defect identification model is a detection Transformer model trained using historical product appearance defect images.

8. A computer device, characterized in that, Including memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images