Image quality detection method, surface defect detection method, device, and storage medium

Abstract

The application belongs to the technical field of visual detection, and discloses an image quality detection method, a surface defect detection method, equipment and a storage medium. The method comprises the following steps: acquiring image data; extracting image features from the image data; calculating distance information according to the image features and pre-trained distribution features to form a distance matrix; generating a target image containing score information according to the distance matrix; and positioning an abnormal position in the target image according to the score information in the target image to obtain an abnormality detection result. In the foregoing manner, a defective product is detected according to image data of the product, low efficiency caused by manual detection is avoided, the accuracy and detection efficiency of defective product detection are improved, and the abnormal position in the image is positioned, thereby facilitating data support for standardizing production operations.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to an image quality inspection method, a surface defect inspection method, an apparatus, and a storage medium. Background Technology

[0002] If defective products are not detected in time during the production of various products, it will affect the subsequent production process, resulting in a waste of human and material resources. If they enter the market, it will affect the sales and circulation of products, and even more so, it will affect the company's social image and reputation.

[0003] Currently, the relevant defect detection methods generally rely on manual inspection. However, since the frequency of product defects and abnormalities on the current production line is very low, and repeated and long-term visual inspection can easily cause fatigue, relying on manual inspection is not ideal and has low inspection efficiency.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art.

[0005] Application content

[0006] The main objective of this application is to provide an image quality inspection method, a surface defect inspection method, an apparatus, and a storage medium, aiming to solve the technical problem of how to efficiently and accurately detect defective products.

[0007] To achieve the above objectives, this application provides an image quality detection method, the method comprising the following steps:

[0008] Acquire image data;

[0009] Extract image features from the image data;

[0010] Distance information is calculated based on the image features and pre-trained distribution features, and a distance matrix is ​​constructed based on the distance information.

[0011] Generate a target image containing score information based on the distance matrix;

[0012] The anomaly location is located in the target image based on the score information in the target image, and the anomaly detection result is obtained.

[0013] Optionally, the distribution characteristics include the sample mean and sample covariance of a multivariate Gaussian distribution;

[0014] The method further includes:

[0015] Obtain positive sample data;

[0016] Based on the positive sample data, fit a multivariate Gaussian distribution model to determine the sample mean and sample covariance of the multivariate Gaussian distribution.

[0017] Optionally, the image data includes image data of the product to be inspected, and the acquisition of image data includes:

[0018] Acquire an initial image of the product to be inspected;

[0019] The initial image is registered with the product template image to obtain the first image;

[0020] The first image is cropped according to a preset product area to obtain the second image;

[0021] The second image is placed on a preset background image to generate image data.

[0022] Optionally, after registering the initial image with the product template image to obtain the first image, the method further includes:

[0023] Store the mapping matrix between the initial image and the first image;

[0024] After locating the anomaly in the target image based on the score information in the target image to obtain the anomaly detection result, the method further includes:

[0025] Determine the coordinates of the abnormal location;

[0026] Based on the preset product area and the mapping matrix, the coordinates of the abnormal location are converted into the coordinates of the defect target corresponding to the initial image;

[0027] A defect target image is generated based on the defect target coordinates and the initial image.

[0028] Optionally, the step of locating the anomaly in the target image based on the score information in the target image to obtain the anomaly detection result includes:

[0029] Obtain multiple target thresholds;

[0030] The target image is binarized according to each target threshold and the score information in the target image to obtain multiple images to be detected.

[0031] Spot detection is performed on the multiple images to be detected to determine the coordinates of abnormal locations where spots exist, and anomaly detection results are obtained.

[0032] Optionally, the step of performing blob detection on the plurality of images to be detected, determining the coordinates of abnormal locations where blobs exist, and obtaining anomaly detection results includes:

[0033] Edge detection is performed on each of the multiple images to be detected to obtain the contour information of each image to be detected;

[0034] The contour center position of each of the images to be detected is determined based on the contour information;

[0035] Clustering is performed on the contour center positions of the multiple images to be detected to obtain a set of spots;

[0036] Determine the radius information of the spot set;

[0037] The spot set is filtered based on the radius information to determine the coordinates of abnormal locations that meet the preset requirements, thereby obtaining the anomaly detection result.

[0038] Optionally, extracting image features from the image data includes:

[0039] The image data is normalized to obtain a standard image;

[0040] Feature extraction is performed on the standard image to generate an embedding vector;

[0041] The embedding vector is then subjected to dimensionality reduction processing to obtain image features.

[0042] Furthermore, to achieve the above objectives, this application also proposes a method for detecting surface defects in industrial products, the method comprising:

[0043] Determine the product identification of the industrial product to be tested;

[0044] Obtain the product information corresponding to the product identifier based on the product identifier;

[0045] Acquire an initial image of the product to be inspected, and preprocess the image according to the product information to obtain image data;

[0046] Extract image features from the image data;

[0047] Distance information is calculated based on the image features and pre-trained distribution features, and a distance matrix is ​​constructed based on the distance information.

[0048] Generate a target image containing score information based on the distance matrix;

[0049] Based on the score information in the target image, anomaly location is determined in the target image to obtain anomaly detection results;

[0050] The anomaly detection results are sent to the manufacturing execution system.

[0051] Furthermore, to achieve the above objectives, this application also proposes an image quality inspection device, which includes: a memory, a processor, and an image quality inspection program stored in the memory and executable on the processor. The image quality inspection program is configured to implement the image quality inspection method described above, or the image quality inspection program is configured to implement the surface defect detection method for industrial products described above.

[0052] In addition, to achieve the above objectives, this application also proposes a storage medium storing an image quality detection program, which, when executed by a processor, implements the image quality detection method as described above, or the image quality detection program, when executed by a processor, implements the surface defect detection method for industrial products as described above.

[0053] In this application, a target image containing score information is constructed based on the image features extracted from the product's image data and the pre-trained distribution features. Anomalies are located based on the score information in the target image, thereby enabling the detection of defective products. This avoids the inefficiency caused by manual inspection, improves the accuracy and efficiency of defective product detection, and locates anomalies in the image, which facilitates data support for standardized production operations. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the structure of the image quality detection device in the hardware operating environment involved in the embodiments of this application;

[0055] Figure 2 This is a flowchart illustrating the first embodiment of the image quality detection method of this application;

[0056] Figure 3 This is an example diagram of the automatic detection system for the image quality detection method of this application;

[0057] Figure 4 This is a flowchart illustrating the second embodiment of the image quality detection method of this application;

[0058] Figure 5(a) is a schematic diagram of the template image in the door sheet metal surface inspection scenario of the image quality inspection method of this application;

[0059] Figure 5(b) is a schematic diagram of the production line camera acquiring images in the door sheet metal surface inspection scenario of the image quality inspection method of this application;

[0060] Figure 5(c) is a schematic diagram of the preprocessed image in the door sheet metal surface inspection scenario of the image quality inspection method of this application;

[0061] Figure 5(d) is a schematic diagram of the black and white image after binarization processing in the door sheet metal surface inspection scenario of the image quality detection method of this application;

[0062] Figure 5(e) is a schematic diagram of the abnormal positioning image in the door sheet metal surface detection scenario of the image quality detection method of this application;

[0063] Figure 5(f) is a schematic diagram of the abnormal positioning image mapped back to the original image in the door sheet metal surface inspection scenario of the image quality detection method of this application;

[0064] Figure 6(a) is a schematic diagram of the template image in the injection molding structural component inspection scenario of the image quality inspection method of this application;

[0065] Figure 6(b) is a schematic diagram of the production line camera acquiring images in the injection molding structural component inspection scenario of the image quality inspection method of this application;

[0066] Figure 6(c) is a schematic diagram of the preprocessed image in the injection molding structural component inspection scenario of the image quality detection method of this application;

[0067] Figure 6(d) is a schematic diagram of the black and white image after binarization in the injection molding structural component inspection scenario of the image quality detection method of this application;

[0068] Figure 6(e) is a schematic diagram of the abnormal location image in the injection molded structural component detection scenario of the image quality detection method of this application;

[0069] Figure 6(f) is a schematic diagram of the abnormal localization image mapped back to the original image in the injection molding structural component detection scenario of the image quality detection method of this application;

[0070] Figure 7(a) is a schematic diagram of the template image in the screen printing detection scenario of the image quality detection method of this application;

[0071] Figure 7(b) is a schematic diagram of the image acquired by the camera on the production line in the screen printing inspection scenario of the image quality detection method of this application;

[0072] Figure 7(c) is a schematic diagram of the preprocessed image in the screen printing detection scenario of the image quality detection method of this application;

[0073] Figure 7(d) is a schematic diagram of the black and white image after binarization in the screen printing detection scenario of the image quality detection method of this application;

[0074] Figure 7(e) is a schematic diagram of anomaly localization image in the screen printing detection scenario of the image quality detection method of this application;

[0075] Figure 7(f) is a schematic diagram of the abnormal localization image mapped back to the original image in the screen printing detection scenario of the image quality detection method of this application;

[0076] Figure 8 This is a flowchart illustrating the third embodiment of the image quality detection method of this application;

[0077] Figure 9 This is a flowchart illustrating the first embodiment of the surface defect detection method for industrial products according to this application.

[0078] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0079] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0080] Reference Figure 1 , Figure 1 This is a schematic diagram of the image quality detection device structure in the hardware operating environment involved in the embodiments of this application.

[0081] like Figure 1 As shown, the image quality inspection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0082] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the image quality inspection device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an image quality detection program.

[0084] exist Figure 1 In the image quality inspection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the image quality inspection device of this application can be set in the image quality inspection device. The image quality inspection device calls the image quality inspection program stored in the memory 1005 through the processor 1001 and executes the image quality inspection method or the surface defect inspection method of industrial products provided in the embodiments of this application.

[0085] In some embodiments, the image quality detection method of this application can be applied to product defect quality detection scenarios in industrial settings. Visual inspection of product defects is of great significance in industrial settings. In the production scenarios of household appliances, such as sheet metal stamping of appliance doors, injection molding of structural parts, and screen printing of control panels, defective products exist. Failure to detect defective products in a timely manner will affect subsequent production processes, resulting in a waste of human and material resources. If these defective products enter the market, it will affect product sales and circulation, and even damage the company's social image and reputation. Currently, relevant defect detection methods generally rely on manual inspection. Since the frequency of product defects and anomalies on industrial production lines is very low, and repetitive, long-term visual inspection is prone to fatigue, manual inspection is inefficient and ineffective. The image quality detection method provided in this application can automatically locate abnormal positions in images through image recognition and processing, realizing the detection of defective products. This avoids the inefficiency caused by manual inspection, improves the accuracy and efficiency of defect detection, and provides data support for standardizing production operations by locating abnormal positions in the image.

[0086] It is understood that, in other embodiments, the image quality detection method provided in this application can also be applied to other application scenarios rather than just industrial scenarios, and no limitation is made here.

[0087] This application provides an image quality detection method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image quality detection method of this application.

[0088] In this embodiment, the image quality detection method includes the following steps:

[0089] Step S10: Acquire image data.

[0090] It is understood that the execution subject of this embodiment is an image quality detection device, which can be a computer, server, or other device with artificial intelligence reasoning capabilities. This embodiment does not limit this.

[0091] It should be noted that the image data can be image data corresponding to the product to be inspected. In some embodiments, the image data can be an image captured by a camera set on the production line when the product to be inspected is within the camera's shooting range. In some embodiments, it may also include: preprocessing the image captured by the camera to remove irrelevant information from the image and enhance the detectability of relevant information.

[0092] Optionally, the image data preprocessing process is as follows: identify the reference region in the original image captured by the camera, rotate the original image so that the reference region in the rotated image reaches the preset region position, and obtain the preprocessed image data.

[0093] Optionally, the image data preprocessing process involves cropping the key regions from the original image captured by the camera and placing them on a substrate image to obtain preprocessed image data. This can address, in some aspects, the problem that scattered or elongated areas of interest with defects are not conducive to algorithm processing.

[0094] Step S20: Extract image features from the image data.

[0095] Specifically, step S20 includes: normalizing the image data to obtain a standard image; extracting features from the standard image to generate an embedding vector; and reducing the dimensionality of the embedding vector to obtain image features.

[0096] In some embodiments, the image data can be normalized using the mean and standard deviation. The mean may include the mean of at least one of the three channels R, G, and B of the image, and the standard deviation may include the standard deviation of at least one of the three channels R, G, and B. The mean and standard deviation are calculated in advance based on a large batch of images.

[0097] It should be noted that, optionally, a convolutional neural network (CNN) model can be used to extract features from the standard image, and the vectors from different layers in the activation graph can be connected to generate an embedding vector carrying information. The CNN can optionally adopt the ResNet18 architecture and use its weights pre-trained on the ImageNet database. Alternatively, a VGG network model can also be used for feature extraction.

[0098] In practical implementation, the embedding vectors obtained from the network model may carry redundant information. The embedding vectors are reduced in dimensionality by randomly reducing the vector dimension or by PCA dimensionality reduction, thereby reducing model complexity, reducing model training and testing time, and maintaining the model's excellent performance.

[0099] Step S30: Calculate distance information based on the image features and the pre-trained distribution features, and construct a distance matrix based on the distance information.

[0100] It should be understood that pre-trained distribution features refer to the statistical characteristics of the distribution of image pixel values ​​learned in advance from known images. Pre-trained distribution features include the sample mean and sample covariance of a multivariate Gaussian distribution. In some embodiments, distance information can be calculated using Mahalanobis distance and a Mahalanobis matrix can be constructed. The position of image feature (i,j) and the learned multivariate Gaussian distribution N(u) can then be calculated according to the following formula. ij ,c ij Mahalanobis distance between )

[0101]

[0102] Where, x ij For the (i,j) position in the image feature, u ij To train the learned multivariate Gaussian distribution sample mean, c ij The sample covariance of the multivariate Gaussian distribution learned during training.

[0103] In some embodiments, the distribution features include the sample mean and sample covariance of a multivariate Gaussian distribution;

[0104] The method further includes: acquiring positive sample data; fitting a multivariate Gaussian distribution model based on the positive sample data, and determining the sample mean and sample covariance of the multivariate Gaussian distribution.

[0105] It should be noted that while there are currently methods for product defect detection based on binary classification, these methods suffer from low detection efficiency due to the long-tailed distribution of defective samples on the production line (few defective samples but abundant positive samples) and the difficulty in collecting negative samples. This makes fully supervised training of the model impossible. In this embodiment, positive sample data is used for training, reducing the difficulty of sample collection. Positive sample data refers to product images without defects. A multivariate Gaussian distribution can capture information from different semantic levels during positive sample training, including compatibility between levels, which is beneficial for improving subsequent anomaly localization performance. Specifically, cameras installed on the production line are used to collect positive sample images of various products in batches. Embedded vector sets at positions (i,j) are extracted from multiple positive sample images using the aforementioned feature extraction method. M represents the number of positive sample images. The sample mean and sample covariance are calculated as follows:

[0106]

[0107]

[0108] Where, suppose X ij The multivariate Gaussian distribution model N(u ij ,cij ) generates, u ij It is X ij The sample mean, c ij It is X ij The sample covariance, where α is a regularization term to ensure c ij It is full-rank and invertible, and can be solved by Mahalanobis distance. It fits a multivariate Gaussian distribution model based on the embedding vector set of multiple positive sample images. Finally, each position is associated with the multivariate Gaussian distribution through the Gaussian distribution matrix.

[0109] Step S40: Generate a target image containing score information based on the distance matrix.

[0110] It should be understood that the distance information calculated using Mahalanobis distance forms a Mahalanobis matrix. According to step S30, the Mahalanobis matrix M = (M(x...) is used to calculate the anomaly map. ij )) 1＜i＜W,1＜j＜H , where W and H are the resolutions of the maximum activation map used to generate the embedding vectors. In the specific implementation, bilinear upsampling and multidimensional Gaussian filtering are used to process the Mahalanobis matrix, and the values ​​are normalized to control the values ​​between (0,1) to obtain the target image containing score information. Each point in the target image carries the corresponding score information.

[0111] Step S50: Locate the abnormal position of the target image based on the score information in the target image to obtain the surface defect detection result.

[0112] It should be noted that, optionally, the target image is binarized according to a preset detection threshold, wherein points with scores greater than the preset detection threshold are assigned a value of 1, and points with scores less than the preset detection threshold are assigned a value of 0, resulting in a black and white image. This black and white image is then subjected to image morphological processing, specifically an opening operation (erosion followed by dilation to remove noise and preserve the original image), to determine the mapping relationship between the images before and after the opening operation (w). red →w spl and h red →h spl The black and white image after opening operation is mapped to the original image size according to the mapping relationship. A spot detection algorithm is then used to detect spots on the mapped black and white image to determine the location coordinates of the areas where spots exist, and the surface defect detection results are obtained.

[0113] In some embodiments, in order to improve the accuracy of abnormal location positioning, this embodiment sets multiple preset detection thresholds, performs multiple binarization processing on the target image based on the multiple preset detection thresholds to obtain multiple black and white images, uses a spot detection algorithm to detect spots on the multiple black and white images respectively, performs clustering to obtain a spot set, determines the radius of the spot set, performs filtering, determines the location coordinates of the region that meets the preset requirements, and obtains the surface defect detection result.

[0114] Currently, there are also methods for product defect detection using template matching to calculate pixel differences. This method primarily involves matching a correct template image with product images captured from the production line to calculate pixel differences. However, this method has high requirements for the physical environment of the production line, such as the lighting conditions on the product surface and product placement. Furthermore, its processing speed is slow, making it unsuitable for high-efficiency production lines. In specific implementations, refer to... Figure 3 , Figure 3 Here is an example diagram of the automatic detection system for the image quality detection method of this application:

[0115] 1. When the camera on the production line scans the barcode, the system enters the image acquisition mode. If the scan is successful, the camera acquires the image and simultaneously calls the MES (Manufacturing Execution System) to determine the product type on the production line, which is convenient for subsequent product type differentiation. If the scan fails, the client displays a prompt message and a red light illuminates.

[0116] 2. Image preprocessing module: Crops the image, image width w ori Image height h ori The cropped image regions are stitched together according to certain rules, with an image width of w. spl Image height h spl The stitched image is scaled according to the algorithm's image size requirements, with an image width of w. red Image height h red ;

[0117] 3. Anomaly Detection and Location Module: The image captured by the camera is input into the image quality detection method of this embodiment for detection. It will return a list of anomaly scores and a list of corresponding anomaly location coordinates. If any value in the anomaly score list is greater than the set threshold, the client displays NG and lights up red. If all values ​​in the anomaly score list are less than the set threshold, the client displays OK and lights up green.

[0118] This system, in conjunction with a production line client, performs photo inspection, anomaly location, and record display for each product. When a defective or abnormal product is detected, the client outputs an "NG" warning with a red light and saves the image of the abnormal product separately. Simultaneously, the production line rejects the defective product. The system's inspection speed and accuracy meet the requirements of industrial scenarios, ensuring smooth production and high-quality finished products.

[0119] In this embodiment, a target image containing score information is constructed based on the image features extracted from the product's image data and the pre-trained distribution features. Anomalies are located based on the score information in the target image, thus enabling the detection of defective products. This avoids the inefficiency caused by manual inspection, improves the accuracy and efficiency of defective product detection, and locates anomalies in the image, facilitating data support for standardized production operations.

[0120] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the image quality detection method of this application.

[0121] Based on the first embodiment described above, the image data in the image quality detection method of this embodiment includes image data of the product to be detected, and step S10 includes:

[0122] Step S101: Obtain the initial image of the product to be inspected.

[0123] Step S102: Register the initial image with the product template image to obtain the first image.

[0124] It should be understood that this embodiment provides an image preprocessing method to address the problem that scattered or elongated areas of interest are unfavorable for algorithm processing during product defect detection. The initial images captured by the production line camera often show randomly placed products or product areas occupying a small portion of the image, making cropping and stitching difficult. In this embodiment, the initial image is registered based on a product template image, the camera image is corrected, and the mapping matrix H between the registered and aligned images is saved. inv Specifically, the registration process aligns the initial image of the product to be detected with the product template image. In the actual implementation, the product category corresponding to the initial image is determined, and the corresponding product template image is determined based on the product category.

[0125] Step S103: Crop the first image according to the preset product area to obtain the second image.

[0126] It should be noted that the preset product area is an image area that corresponds to the product template image in advance. Optionally, the preset product area is an image area that is set in advance based on the locations where product defects frequently occur. The first image is cropped according to the preset product area to obtain a second image that includes one or more image areas.

[0127] Step S104: Place the second image on a preset background image to generate image data.

[0128] It should be understood that the second image is placed on a preset background image and overlaid to generate image data. If the second image comprises multiple image regions, the cropped image regions can optionally be stitched together and / or scaled, and the processed image is placed on the preset background image and overlaid to generate image data. Optionally, the preset background image is a black background image. In specific implementations, multiple background images of different sizes are preset, and a preset background image of a suitable size is selected from the multiple background images according to the size of the second image (or the stitched and / or scaled image).

[0129] It should be noted that the explanation uses a product template image corresponding to a preset product area as an example. Let's assume the coordinates of the preset product area are (x...). t_min ,y t_min ,x t_max ,y t_max The coordinates of the splicing region are (x... r_min ,y r_min ,x r_max ,y r_max Set a black background image of appropriate size. The dimensions of the cropped second image are:

[0130] w = x t_max -x t_min ;

[0131] h = y t_max -y t_min ;

[0132] The cropped second image is placed on the black background image at the following coordinates:

[0133] x start =x r_min ;

[0134] x end =x r_min +w;

[0135] y start =y r_min ;

[0136] y end =y r_min +h.

[0137] In some embodiments, current product defect detection methods cannot automatically locate defective areas of the product to form electronic image archives, which is not conducive to standardizing production line personnel operations and maximizing data value. To make the detection results more intuitive, facilitate the saving and subsequent viewing of the detection result images, and provide data support for standardizing production operations, after step S102, the method further includes: storing a mapping matrix between the initial image and the first image;

[0138] After step S50, the method further includes: determining the coordinates of the abnormal location; converting the abnormal location coordinates into the defect target coordinates corresponding to the initial image according to the preset product area and the mapping matrix; and generating a defect target image according to the defect target coordinates and the initial image.

[0139] It should be understood that this embodiment provides an image post-processing method to map the coordinates of the detected abnormal location back to the original image. Optionally, a circle is drawn on the image with the mapped defect target coordinates as the center and a preset radius is selected to generate a defect target image.

[0140] It should be noted that, taking the detection of an abnormal region as an example, let's assume the coordinates of the detected abnormal location are (x... center ,y center The calculation process for mapping the coordinates of abnormal locations back to the original image is as follows:

[0141] 1. Eliminate the impact of positional changes caused by displacement during splicing:

[0142] x r_center =x center -x r_min ;

[0143] y r_center =y center -y r_min ;

[0144] 2. Eliminate the impact of positional changes caused by displacement during image cutting:

[0145] x t_center =x r_center +x t_min ;

[0146] y t_center =y r_center +y t_min ;

[0147] 3. Processing using the inverse mapping matrix during registration and alignment (x) t_center ,y t_center To restore it to its original position coordinates in the original image:

[0148] (x ori_center ,y ori_center )=(x t_center ,y t_center )×H inv ;

[0149] 4. Select a suitable radius length r, so that (x ori_center ,y ori_center Mark the location of abnormal areas on the original image captured by the camera with the center of the circle, display and save it on the client.

[0150] Reference Figures 5(a)-5(f) , Figures 5(a)-5(f) The images are template images, images captured by cameras on the production line, pre-processed images, binarized black and white images, anomaly location images, and anomaly location images mapped back to the original image, respectively, for the detection and processing of defects on the surface of electrical appliance door sheet metal. The process is as follows: Select a template image, register and align the images captured by the production line. Since the door sheet metal is large, it is not conducive to the algorithm to process it all at once. Therefore, it is divided into two parts for detection. After cutting, the image to be detected is formed. The image to be detected is input into the image quality detection method of this embodiment to obtain the center coordinates of the abnormal area. The obtained center coordinates of the abnormal area are mapped back to the image captured by the production line for marking the abnormal area and displaying it on the client.

[0151] Reference Figures 6(a)-6(f) , Figures 6(a)-6(f) The images are template images, production line camera images, pre-processed images, binarized black and white images, anomaly location images, and anomaly location images mapped back to the original image, respectively, in the defect detection process of electrical duct plate injection molded structural parts. The template image is selected, and the images acquired from the production line are registered and aligned. Since the injection molded structural parts are large and the areas of interest are scattered, the target area is cut and spliced ​​to form the image to be detected. The image to be detected is input into the image quality detection method of this embodiment to obtain the center coordinates of the abnormal area. The obtained center coordinates of the abnormal area are mapped back to the image acquired from the production line, and the abnormal area is marked and displayed on the client.

[0152] Reference Figures 7(a)-7(f) , Figures 7(a)-7(f)The images are template images, production line camera images, pre-processed images, binarized black and white images, anomaly location images, and anomaly location images mapped back to the original image, respectively, in the screen printing inspection scenario. The process of screen printing inspection of electrical control panels is as follows: Select a template image, register and align the production line image. Since the target appears as a long strip shape in the image, it is cut and spliced ​​to form the image to be inspected. Input the image to be inspected into the image quality detection method of this embodiment to obtain the center coordinates of the abnormal area. Map the obtained center coordinates of the abnormal area back to the production line image, mark the abnormal area and display it on the client.

[0153] This embodiment improves the accuracy and efficiency of defect detection by registering, cutting, and stitching product images together to concentrate the product area. This solves the problem that the long strip-shaped or scattered distribution of defective areas of concern is not conducive to subsequent algorithm processing and discrimination.

[0154] refer to Figure 8 , Figure 8 This is a flowchart illustrating the third embodiment of the image quality detection method of this application.

[0155] Based on the first embodiment described above, step S50 of the image quality detection method in this embodiment includes:

[0156] Step S501: Obtain multiple target thresholds.

[0157] It should be understood that multiple target thresholds can be multiple different preset parameter values. Optionally, the controller stores a preset minimum threshold, maximum threshold, and threshold step size, and obtains a series of thresholds between the minimum and maximum thresholds at intervals of the threshold step size to obtain multiple target thresholds.

[0158] Step S502: Binarize the target image according to each target threshold and the score information in the target image to obtain multiple images to be detected.

[0159] It should be noted that the target image is binarized using multiple target thresholds to obtain multiple images to be detected. Points in the target image with scores greater than the target threshold are assigned a value of 1, and points with scores less than the target threshold are assigned a value of 0.

[0160] Step S503: Perform blob detection on the multiple images to be detected, determine the coordinates of abnormal locations where blobs exist, and obtain the abnormal detection results.

[0161] It should be understood that, optionally, spots in multiple images to be detected can be identified, and the center position of each spot can be determined as the coordinates of the abnormal position to obtain the surface defect detection result.

[0162] Optionally, step S503 includes: performing edge detection on the plurality of images to be detected to obtain contour information of each image to be detected; determining the contour center position of each image to be detected based on the contour information; clustering the contour center positions of the plurality of images to be detected to obtain a set of spots; determining the radius information of the set of spots; filtering the set of spots based on the radius information to determine the coordinates of abnormal positions that meet preset requirements, and obtaining the surface defect detection result.

[0163] It should be noted that the controller stores pre-set minimum spot spacing, minimum area, and maximum area values. Edge detection is performed on multiple binarized images, and the center position of each contour is calculated. The detected contour center positions from multiple images are extracted, and those with center positions less than the minimum spot spacing are grouped into a cluster, corresponding to a spot set. This results in one or more spot sets. The radius of each spot set is estimated, and its area is calculated based on the radius. The spot sets are then filtered according to the minimum and maximum area values ​​to determine the coordinates of abnormal locations that meet preset requirements, thus obtaining the surface defect detection result. For example, if the area S of spot set A is calculated based on its radius, and S is greater than the maximum area value or less than the minimum area value, then spot set A is filtered out.

[0164] In this embodiment, multiple target thresholds are used to binarize the target image containing score information to obtain multiple images to be detected. Spot detection is performed on the multiple images to be detected to determine the coordinates of abnormal positions where spots exist, and the surface defect detection results are obtained. This improves the accuracy of abnormal position location and further improves the accuracy and efficiency of defective product detection.

[0165] This application provides a method for detecting surface defects in industrial products, referring to... Figure 9 , Figure 9 This is a flowchart illustrating the first embodiment of the surface defect detection method for industrial products according to this application.

[0166] like Figure 9 As shown in the embodiments of this application, the surface defect detection method for industrial products includes:

[0167] Step S01: Determine the product identifier of the industrial product to be tested.

[0168] Understandably, industrial products to be inspected on the production line are affixed with identification codes to identify and differentiate each product. These codes can be barcodes (one-dimensional codes) or two-dimensional codes; this implementation does not impose any restrictions on this. The identification code of the industrial product to be inspected is determined by scanning the identification code of the industrial product within the inspection range using a camera on the production line.

[0169] Step S02: Obtain the product information corresponding to the product identifier based on the product identifier.

[0170] It should be noted that, optionally, the Manufacturing Execution System (MES) can be invoked based on the product identifier to obtain the corresponding product information, which shall at least include the product category of the industrial product to be tested.

[0171] Step S03: Acquire an initial image of the product to be inspected, and preprocess the image according to the product information to obtain image data.

[0172] In a specific implementation, the corresponding product template image and preset product area are determined according to the product category included in the product information. The preprocessing process can be as follows: register the initial image according to the product template image to obtain a first image; crop the first image according to the preset product area to obtain a second image; place the second image on a preset background image to generate image data.

[0173] Step S04: Extract image features from the image data.

[0174] Specifically, step S04 includes: normalizing the image data to obtain a standard image; extracting features from the standard image to generate an embedding vector; and reducing the dimensionality of the embedding vector to obtain image features.

[0175] In some embodiments, the image data can be normalized using the mean and standard deviation. The mean may include the mean of at least one of the three channels R, G, and B of the image, and the standard deviation may include the standard deviation of at least one of the three channels R, G, and B. The mean and standard deviation are calculated in advance based on a large batch of images.

[0176] It should be noted that, optionally, a convolutional neural network (CNN) model can be used to extract features from the standard image, and the vectors from different layers in the activation graph can be connected to generate an embedding vector carrying information. The CNN can optionally adopt the ResNet18 architecture and use its weights pre-trained on the ImageNet database. Alternatively, a VGG network model can also be used for feature extraction.

[0177] In practical implementation, the embedding vectors obtained from the network model may carry redundant information. The embedding vectors are reduced in dimensionality by randomly reducing the vector dimension or by PCA dimensionality reduction, thereby reducing model complexity, reducing model training and testing time, and maintaining the model's excellent performance.

[0178] Step S05: Calculate distance information based on the image features and the pre-trained distribution features, and construct a distance matrix based on the distance information.

[0179] It should be understood that pre-trained distribution features refer to the statistical characteristics of the distribution of image pixel values ​​learned in advance from known images. Pre-trained distribution features include the sample mean and sample covariance of a multivariate Gaussian distribution. In some embodiments, distance information can be calculated using Mahalanobis distance and a Mahalanobis matrix can be constructed. The position of image feature (i,j) and the learned multivariate Gaussian distribution N(u) can then be calculated according to the following formula. ij ,c ij Mahalanobis distance between )

[0180]

[0181] Where, x ij For the (i,j) position in the image feature, u ij To train the learned multivariate Gaussian distribution sample mean, c ij The sample covariance of the multivariate Gaussian distribution learned during training.

[0182] In some embodiments, the preset distribution features include the sample mean and sample covariance of a multivariate Gaussian distribution; the method further includes: acquiring positive sample data; fitting a multivariate Gaussian distribution model based on the positive sample data to determine the sample mean and sample covariance of the multivariate Gaussian distribution.

[0183] It should be noted that while there are currently methods for product defect detection based on binary classification, these methods suffer from low detection efficiency due to the long-tailed distribution of defective samples on the production line (few defective samples but abundant positive samples) and the difficulty in collecting negative samples. This makes fully supervised training of the model impossible. In this embodiment, positive sample data is used for training, reducing the difficulty of sample collection. Positive sample data refers to product images without defects. A multivariate Gaussian distribution can capture information from different semantic levels during positive sample training, including compatibility between levels, which is beneficial for improving subsequent anomaly localization performance. Specifically, cameras installed on the production line are used to collect positive sample images of various products in batches. Embedded vector sets at positions (i,j) are extracted from multiple positive sample images using the aforementioned feature extraction method. M represents the number of positive sample images. The sample mean and sample covariance are calculated as follows:

[0184]

[0185]

[0186] Where, suppose X ijThe multivariate Gaussian distribution model N(u ij ,c ij ) generates, u ij It is X ij The sample mean, c ij It is X ij The sample covariance, where α is a regularization term to ensure c ij It is full-rank and invertible, and can be solved by Mahalanobis distance. It fits a multivariate Gaussian distribution model based on the embedding vector set of multiple positive sample images. Finally, each position is associated with the multivariate Gaussian distribution through the Gaussian distribution matrix.

[0187] Step S06: Generate a target image containing score information based on the distance matrix.

[0188] It should be understood that the distance information calculated using Mahalanobis distance forms a Mahalanobis matrix. Based on the steps described above, the Mahalanobis matrix M = (M(x...) is used to calculate the anomaly map. ij )) 1＜i＜W,1＜j＜H , where W and H are the resolutions of the maximum activation map used to generate the embedding vectors. In the specific implementation, bilinear upsampling and multidimensional Gaussian filtering are used to process the Mahalanobis matrix, and the values ​​are normalized to control the values ​​between (0,1) to obtain the target image containing score information. Each point in the target image carries the corresponding score information.

[0189] Step S07: Based on the score information in the target image, locate the abnormal position in the target image to obtain the abnormal detection result.

[0190] It should be noted that, optionally, the target image is binarized according to a preset detection threshold, wherein points with scores greater than the preset detection threshold are assigned a value of 1, and points with scores less than the preset detection threshold are assigned a value of 0, resulting in a black and white image. This black and white image is then subjected to image morphological processing, specifically an opening operation (erosion followed by dilation to remove noise and preserve the original image), to determine the mapping relationship between the images before and after the opening operation (w). red →w spl and h red →h spl The black and white image after opening operation is mapped to the original image size according to the mapping relationship. A spot detection algorithm is then used to detect spots on the mapped black and white image to determine the location coordinates of the areas where spots exist, and the surface defect detection results are obtained.

[0191] In some embodiments, in order to improve the accuracy of abnormal location positioning, this embodiment sets multiple preset detection thresholds, performs multiple binarization processing on the target image based on the multiple preset detection thresholds to obtain multiple black and white images, uses a spot detection algorithm to detect spots on the multiple black and white images respectively, performs clustering to obtain a spot set, determines the radius of the spot set, performs filtering, determines the location coordinates of the region that meets the preset requirements, and obtains the surface defect detection result.

[0192] Step S08: Send the anomaly detection result to the manufacturing execution system.

[0193] Understandably, a manufacturing execution system (MES) should at least have production process control and product quality management functions. Transmitting anomaly detection results into the MES facilitates the MES's control over the subsequent production process of abnormal products, preventing defective products from entering the market. On the other hand, it also allows the MES to store information on normal and abnormal products for later retrieval by management personnel.

[0194] Currently, there is also a method for product defect detection using template matching to calculate the difference. This method mainly involves matching a correct template image with a product image collected from the production line to calculate the pixel difference. However, this method has high requirements for the physical environment of the production line, such as the lighting conditions on the product surface and the product placement. Furthermore, this method has a slow processing speed, making it unsuitable for high-efficiency production lines. (Refer to...) Figure 3 :

[0195] 1. When the camera on the production line scans the barcode, the system enters image acquisition mode. If the scan is successful, the camera acquires the image and simultaneously calls the MES (Manufacturing Execution System) to determine the product type on the production line, facilitating subsequent product type differentiation. If the scan fails, the client displays a prompt message and a red light illuminates.

[0196] 2. Image preprocessing module: Crops the image, image width w ori Image height h ori The cropped image regions are stitched together according to certain rules, with an image width of w. spl Image height h spl The stitched image is scaled according to the algorithm's image size requirements, with an image width of w. red Image height h red ;

[0197] 3. Anomaly Detection and Location Module: The image captured by the camera is input into the surface defect detection method of industrial products in this embodiment for detection. It will return a list of anomaly scores and a list of corresponding anomaly location coordinates. If any value in the anomaly score list is greater than the set threshold, the client will display NG and light up red. If all values ​​in the anomaly score list are less than the set threshold, the client will display OK and light up green.

[0198] This system, in conjunction with a production line client, performs photo inspection, anomaly location, and record display for each product. When a defective or abnormal product is detected, the client outputs an "NG" warning with a red light and saves the image of the abnormal product separately. Simultaneously, the production line rejects the defective product. The system's inspection speed and accuracy meet the requirements of industrial scenarios, ensuring smooth production and high-quality finished products.

[0199] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0200] In this embodiment, the product identification code is scanned to obtain product information. Based on the product information, the product image data is preprocessed to obtain image data. Based on the image features extracted from the image data and the pre-trained distribution features, a target image containing score information is constructed. Based on the score information in the target image, the abnormal location is located, and the abnormal detection result is sent to the manufacturing execution system. This is suitable for industrial product production scenarios, realizes the detection of defective products, avoids the inefficiency caused by manual inspection, improves the accuracy and efficiency of defective product detection, and locates the abnormal location in the image, which facilitates the provision of data support for standardized production operations.

[0201] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0202] In addition, for technical details not described in detail in this embodiment, please refer to the image quality detection method provided in any embodiment of this application, which will not be repeated here.

[0203] Furthermore, this application also proposes a storage medium storing an image quality detection program, which, when executed by a processor, implements the image quality detection method or the surface defect detection method for industrial products as described above.

[0204] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0205] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0206] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0207] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0208] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image quality detection method, characterized by, The method includes: Acquire image data; Extract image features from the image data; Distance information is calculated based on the image features and pre-trained distribution features, and a distance matrix is ​​constructed based on the distance information. Generate a target image containing score information based on the distance matrix; Based on the score information in the target image, anomaly location is determined in the target image to obtain anomaly detection results; The step of locating anomalies in the target image based on the score information in the target image to obtain anomaly detection results includes: Obtain multiple target thresholds; The target image is binarized according to each target threshold and the score information in the target image to obtain multiple images to be detected. Edge detection is performed on each of the multiple images to be detected to obtain the contour information of each image; the contour center position of each image to be detected is determined based on the contour information; the contour center positions of the multiple images to be detected are clustered to obtain a set of spots; the radius information of the set of spots is determined; the set of spots is filtered based on the radius information to determine the coordinates of abnormal positions that meet preset requirements, and an anomaly detection result is obtained.

2. The image quality detection method of claim 1, wherein, The distribution characteristics include the sample mean and sample covariance of a multivariate Gaussian distribution; The method further includes: Obtain positive sample data; Based on the positive sample data, fit a multivariate Gaussian distribution model to determine the sample mean and sample covariance of the multivariate Gaussian distribution.

3. The image quality detection method as described in claim 1, characterized in that, The image data includes image data of the product to be inspected, and the acquisition of image data includes: Acquire an initial image of the product to be inspected; The initial image is registered with the product template image to obtain the first image; The first image is cropped according to a preset product area to obtain the second image; The second image is placed on a preset background image to generate image data.

4. The image quality detection method as described in claim 3, characterized in that, After registering the initial image with the product template image to obtain the first image, the method further includes: Store the mapping matrix between the initial image and the first image; After locating the anomaly in the target image based on the score information in the target image to obtain the anomaly detection result, the method further includes: Determine the coordinates of the abnormal location; Based on the preset product area and the mapping matrix, the coordinates of the abnormal location are converted into the coordinates of the defect target corresponding to the initial image; A defect target image is generated based on the defect target coordinates and the initial image.

5. The image quality detection method according to any one of claims 1-4, characterized in that, Extracting image features from the image data includes: The image data is normalized to obtain a standard image; Feature extraction is performed on the standard image to generate an embedding vector; The embedding vector is then subjected to dimensionality reduction processing to obtain image features.

6. A method for detecting surface defects in industrial products, characterized in that, The image quality detection method according to any one of claims 1 to 5, and the surface defect detection method for the industrial product, include: Determine the product identification of the industrial product to be tested; Obtain the product information corresponding to the product identifier based on the product identifier; Acquire an initial image of the product to be inspected, and preprocess the image according to the product information to obtain image data; Extract image features from the image data; Distance information is calculated based on the image features and pre-trained distribution features, and a distance matrix is ​​constructed based on the distance information. Generate a target image containing score information based on the distance matrix; Based on the score information in the target image, anomaly location is determined in the target image to obtain anomaly detection results; The anomaly detection results are sent to the manufacturing execution system.

7. An image quality inspection device, characterized in that, The device includes: a memory, a processor, and an image quality detection program stored in the memory and executable on the processor, the image quality detection program being configured to implement the image quality detection method as described in any one of claims 1 to 5, or the image quality detection program being configured to implement a surface defect detection method for an industrial product as described in claim 6.

8. A storage medium, characterized in that, The storage medium stores an image quality detection program, which, when executed by a processor, implements the image quality detection method as described in any one of claims 1 to 5, or, when executed by a processor, implements a surface defect detection method for an industrial product as described in claim 6.

Images