A Weak Feature Surface Defect Detection Method, Terminal and Storage Medium Based on Gray-Level Co-Occurrence Matrix

Abstract

The present invention discloses a method, a terminal and a storage medium for detecting weak feature surface defects based on a gray-level co-occurrence matrix, belonging to the technical field of image processing. The steps include S1. Generating a gray-level co-occurrence matrix based on non-uniform threshold segmentation of an image and extracting texture features, and S2. Constructing a YOLOv3 deep learning detection model that fuses texture features. The present invention combines the gray-level co-occurrence matrix with convolutional neural network technology for the problem of detecting weak feature image surface defects. An adaptive image grayscale algorithm is used to process the source image, and then a non-uniform threshold segmentation algorithm is used to complete the multi-threshold segmentation of the image. On this basis, an image gray-level co-occurrence matrix is generated and corresponding feature quantities are calculated. Then, the feature quantities of the gray-level co-occurrence matrix are combined with the YOLOv3 neural network algorithm, and the loss function of the YOLOv3 algorithm is improved, thereby effectively improving the accuracy and reliability of detecting weak feature image surface defects, and having a broad application market space and economic value.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and specifically relates to a method, a terminal and a storage medium for detecting weak feature surface defects based on a gray-level co-occurrence matrix, aiming at the problem of detecting surface defects of weak feature images. Background Art

[0002] Defect detection, as an important means to improve the quality of industrial products, has received more and more attention. Among them, the detection of low-recognizability surface defects is an important research direction in industrial vision detection, mainly aiming at defects with low contrast, complex texture, small size or irregular shape. Surface defects (such as cracks and scratches) directly affect the product performance. Early detection can reduce the scrap rate. An excellent detection method is of great significance in improving product quality, reducing production costs, increasing production efficiency, ensuring safety, etc. Currently, the commonly used defect detection methods mainly rely on technologies such as image enhancement, feature extraction, and machine learning. Although certain results have been achieved, the effects are limited when dealing with low-recognizability and complex texture defects. Aiming at the problem of detecting low-recognizability surface defects, in order to overcome the problems of poor detection accuracy and low detection efficiency existing in the current detection methods. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention proposes a method for detecting weak feature surface defects based on a gray-level co-occurrence matrix, including the following steps:

[0004] Generating a gray-level co-occurrence matrix based on image non-uniform threshold segmentation and extracting texture features, the steps of which include:

[0005] S11. Gray-scale processing the source image through an adaptive image gray-scale algorithm, and dynamically calculating the weight coefficient based on the histogram of each color channel;

[0006] S12. Using a clustering algorithm to perform non-uniform segmentation on the pixel threshold;

[0007] S13. Generating multi-direction gray-level co-occurrence matrices on the segmented image, and extracting four texture feature vectors of correlation, contrast, homogeneity, and energy;

[0008] Constructing a YOLOv3 deep learning detection model integrating texture features, the steps of which include:

[0009] S21. Generating the initial anchor box size, extracting the multi-scale feature maps of the target image, and constructing multiple detection layers corresponding to different scale feature maps respectively;

[0010] S22. Calculating the gray-level co-occurrence matrix feature vector for each candidate region, and generating an annotation vector integrating the gray-level co-occurrence matrix;

[0011] S23. Constructing a loss function based on the texture feature vector;

[0012] S24. Screen the final detection results through the non-maximum suppression algorithm and merge the overlapping detection frames according to the intersection over union threshold.

[0013] Furthermore, the specific steps of the adaptive image grayscale algorithm described in step S11 include:

[0014] Obtain the histogram of each channel of the image , , ;

[0015] Perform continuous sorting on the channel histograms to obtain a new histogram , , ;

[0016] Take the pixel values that account for a significant proportion among them to calculate the weights of each channel. The formula is:

[0017]

[0018] Among them, , , are the weights of the R, G, and B channels respectively, is the pixel proportion value of the pixels that account for a significant proportion in each channel, represents taking the pixel values with a proportion of in the channel, represents calculating the sum of these pixel values.

[0019] Furthermore, the clustering algorithm described in step S12 adopts the K-Means clustering algorithm, and the objective function is:

[0020]

[0021] Among them, is the objective function, is the index of the cluster for histogram clustering, is the index of the histogram elements in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, is the value of each histogram element, is the average value of the histogram elements in each cluster.

[0022] Furthermore, the method for generating the multi-direction gray-level co-occurrence matrix described in step S13 is:

[0023] Any element in the gray-level image with a gray level of M can be expressed as , where is the distance of the pixels in the set original image, is the set direction, taking , and respectively represent the row and column indices of the gray-level co-occurrence matrix, and also represent the gray levels of the image.

[0024] Further, the formula for extracting the texture feature vector is:

[0025] Correlation:

[0026] ;

[0027] Contrast:

[0028] ;

[0029] Homogeneity:

[0030] ;

[0031] Energy:

[0032] ;

[0033] where , are respectively the pixel levels in the gray-level co-occurrence matrix, corresponding to the pixel values in the image, is the value in the normalized gray-level co-occurrence matrix, , are respectively the average value and standard deviation of the pixel values.

[0034] Further, the labeled vector of the fused gray-level co-occurrence matrix in step S22 is:

[0035] ;

[0036] where, is the labeled vector of the candidate region in the YOLOv3 algorithm, the label indicates whether the target is included in the candidate region, represents the position of the candidate region relative to the picture, represents the class label of the object in the candidate region;

[0037] , , , , are respectively the 4 feature quantities of correlation, contrast, homogeneity and energy, , , , For , different combinations

[0038] Furthermore, the loss function based on the texture feature vector described in step S23 is as follows:

[0039] ;

[0040] where is the classification loss, is the confidence loss, is the bounding box loss, is the gray level co-occurrence matrix loss, and

[0041] ;

[0042] wherein, is the index of the candidate region, is the number of candidate regions, is the index of the gray level co-occurrence matrix category, with a total of 4 types, is the weight corresponding to the element in the gray level co-occurrence matrix, and different values can be set according to different situations and experimental results. In the present invention, equal-value weights are adopted, .

[0043] The present invention also provides a terminal, including a memory, a processor, and a computer program stored on the memory. When the processor executes the program, the steps in the above method are implemented.

[0044] The present invention also provides a storage medium storing computer program instructions, which implement the steps in the above method when the instructions are executed by a processor.

[0045] Advantages of the present invention:

[0046] Aiming at the problem of surface defect detection of weak feature images, the present invention combines the gray level co-occurrence matrix with convolutional neural network technology, uses an adaptive image grayscale algorithm to process the source image, then uses a non-uniform threshold segmentation algorithm to complete the multi-threshold segmentation of the image. On this basis, an image gray level co-occurrence matrix is generated and the corresponding feature quantities are calculated. Then, the feature quantities of the gray level co-occurrence matrix are combined with the YOLOv3 neural network algorithm, and the loss function of the YOLOv3 algorithm is improved, thereby effectively improving the accuracy and reliability of surface defect detection of weak feature images, and having broad application market space and economic value. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is a comparison chart of image grayscale conversion between the weighted average method and the adaptive grayscale method;

[0049] Figure 2 It is a comparison chart of multi-threshold segmentation of images between uniform threshold segmentation and non-uniform threshold segmentation;

[0050] Figure 3 It is a schematic diagram of the direction of the gray-level co-occurrence matrix;

[0051] Figure 4 It is the recognition effect of the surface defects of the weak feature image by the method of the present invention Figure 1 ;

[0052] Figure 5 It is the recognition effect of the surface defects of the weak feature image by the method of the present invention Figure 2 。 Detailed implementation manners

[0053] The following will illustrate the present application in combination with specific embodiments:

[0054] Embodiment 1:

[0055] This embodiment provides a weak feature surface defect detection method based on a gray-level co-occurrence matrix, including:

[0056] Step S1. Generate a gray-level co-occurrence matrix based on non-uniform threshold segmentation of the image and extract texture features;

[0057] Step S2. Construct a YOLOv3 deep learning detection model that fuses texture features.

[0058] The gray-level co-occurrence matrix (Grey Level Co-occurrence Matrix, GLCM) is a statistical method for describing the texture features of an image. By analyzing the spatial relationship between pixels in the image and the statistical distribution of gray levels, it captures the texture information of the image, describes the spatial relationship and texture features between pixels, and reflects the joint probability distribution of different gray value combinations that appear simultaneously at a given spatial distance and direction.

[0059] Although the gray-level co-occurrence matrix can reflect the texture features of an image, its computational complexity is relatively high. For example, for an 8-bit gray-scale image, the pixel gray-level has 256 levels, and the corresponding gray-level co-occurrence matrix size is 256×256 at this time, resulting in a large amount of computation and reducing the operating efficiency of the machine vision system.

[0060] To overcome the above defects, the gray-level of the original gray-scale image is usually compressed, such as compressing the pixel gray-level threshold segmentation of 256 levels to 32 levels or 8 levels. For this reason, the present invention further performs threshold segmentation on the gray-scale image by means of adaptive non-uniform threshold segmentation to compress the gray-level.

[0061] Therefore, for step S1, specifically:

[0062] S11. Perform gray-scale processing on the source image through an adaptive image gray-scale algorithm, and dynamically calculate the weight coefficients based on the histograms of each color channel;

[0063] Generally, the image formed by the camera is an RGB color image, and it often needs to be converted into a gray-scale image when further processed. The most commonly used image gray-scale method is the weighted average method. This method starts from the sensitivity of the human eye to different colors, and its calculation method is:

[0064]

[0065] Among them, R, G, and B are the pixel values of the red, green, and blue channels of the image respectively. The above method has the characteristic of simple calculation. However, the imaging of the surface of weak feature objects usually has less color composition, mostly showing single or similar colors, and each color component tends to be in a relatively concentrated area in a certain channel or color space. For this reason, the present invention adaptively determines the weights of image gray-scale based on each color channel of the image.

[0066] First, obtain the histogram of each channel of the image, and set them as 、 、 respectively. Then sort them in ascending order, and set the histograms of each channel after sorting as 、 、 respectively. Then, take the pixel values that account for a significant proportion among them to calculate the weights of each channel, as shown in the following formula:

[0067]

[0068] Among them, 、 、 are the weights of the R, G, and B channels respectively, is the pixel proportion value that accounts for a significant proportion of each channel. In the present invention , Indicates taking the pixel values that account for in the channel, and indicates calculating the sum of these pixel values. The effect of image grayscale conversion is as Figure 1 shown, from which it can be seen that the grayscale conversion method used in this embodiment can more prominently show the defects in the feature image.

[0069] S12. Use the K-Means clustering algorithm to perform non-uniform segmentation on the pixel threshold;

[0070] The currently used pixel threshold segmentation method is usually uniform threshold segmentation, that is, the pixel grayscale range is evenly divided into parts, and the pixel value range covered by each part is , where is the maximum pixel value in the original image, is a positive integer between 0 and , and each pixel value falling within the corresponding range is set to . Uniform threshold segmentation has the characteristics of simple implementation, easy to understand and implement. However, for the case of non-uniform signal distribution, uniform threshold segmentation often cannot provide sufficient accuracy, especially when facing small signal parts such as images of material surfaces with low recognition, it is easy to cause the loss of important information. The present invention uses an improved K-Means clustering algorithm to complete the non-uniform threshold segmentation of the image.

[0071] The K-Means algorithm is a commonly used clustering algorithm, belonging to the unsupervised learning method, and is used to divide data into K clusters (Clusters), so that the data points within the cluster are highly similar, while the similarity between clusters is low.

[0072] First, obtain the histogram of the image after grayscale conversion, denoted as , at this time contains 256 elements, and each element represents the number of a certain grayscale pixel in the image. For images with weak features, the regions with gentle changes in the image often represent the main colors of the object, and the corresponding number of such colors is also more, while the defect regions will have sudden changes in grayscale values, but the number of such pixels is usually small. Based on this, the objective function of the K-Means clustering algorithm used in the present invention is as follows:

[0073]

[0074] Among them, is the objective function, is the index of the cluster for histogram clustering, is the index of the histogram element in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, which is taken as in the present invention, so as to compress the image gray value into 32 levels. is the value of each histogram element. is the average value of histogram elements in each cluster. Equation indicates that the variance of histogram elements in the cluster is taken as the target. The smaller the variance is, the smaller the difference between the corresponding pixel gray values in the same cluster is. The pixels with small differences are divided into one cluster, so as to realize the non-uniform threshold segmentation of the image. The effect is as Figure 2 shown.

[0075] S13. Generate multi-directional gray-level co-occurrence matrices on the segmented image, and extract four texture feature vectors of correlation, contrast, homogeneity and energy.

[0076] Fourteen feature quantities related to the image texture features can be calculated from the gray-level co-occurrence matrix, among which correlation, contrast, homogeneity and energy are four independent feature quantities. The generation method of the gray-level co-occurrence matrix is as follows. Suppose a gray-scale image contains gray levels, then its gray-level co-occurrence matrix is a square matrix with a size of . Each element in the square matrix can be expressed as , where is the set distance of pixels in the original image, is the set direction (bidirectional), usually taking , and respectively represent the row and column indexes of the gray-level co-occurrence matrix, and also represent the gray levels of the image. The elements of the gray-level co-occurrence matrix describe the probability that a pair of pixels with gray levels and separated by and pixels in the

[0077] direction of the gray-scale image, which can specifically reflect the texture features of the image. Let the image gray-level co-occurrence matrix be . The correlation

[0078]

[0079] indicates the similarity degree of elements in the row and column directions, and its calculation method is: The contrast

[0080]

[0081] measures the randomness of the image and is calculated as follows: The homogeneity

[0082]

[0083] Energy Used to characterize the complexity of the texture in the image, and its calculation method is as follows:

[0084]

[0085] Equation - Equation In 、 Are respectively the pixel levels in the gray-level co-occurrence matrix, corresponding to the pixel values in the image, Is the value in the normalized gray-level co-occurrence matrix, 、 Are respectively the average value and the standard deviation of the pixel values.

[0086] The present invention uses the above 4 eigenvalue, combined with the deep learning method, to complete the defect detection of the weak feature image.

[0087] Step S2 constructs a YOLOv3 deep learning detection model integrating texture features.

[0088] The convolutional neural network is a deep learning model specifically used to process data with a grid structure. It has achieved great success in the field of computer vision and is widely used in tasks such as image classification, object detection, and semantic segmentation. Through the local receptive field and weight sharing mechanism, it effectively reduces the number of parameters and can automatically extract the hierarchical features of the input data. Using the convolutional neural network can generate feature maps of different depths of the image, and through further processing of these feature maps, the corresponding functions can be realized. The present invention combines the convolutional neural network YOLOv3 with the gray-level co-occurrence matrix, enabling it to detect the defects of the weak feature surface image. The specific steps are as follows:

[0089] S21. Generate the initial anchor box size, extract the multi-scale feature maps of the target image, and construct multiple detection layers corresponding to different scale feature maps respectively;

[0090] S22. Calculate the gray-level co-occurrence matrix feature vector for each candidate region and generate the annotation vector integrating the gray-level co-occurrence matrix;

[0091] S23. Construct the loss function based on the texture feature vector;

[0092] S24. Screen the final detection results through the non-maximum suppression algorithm and merge the overlapping detection boxes according to the intersection over union threshold.

[0093] In the present invention, in addition to annotating the candidate regions, the gray-level co-occurrence matrix is calculated for each candidate region using Step 1, and four feature quantities, namely correlation, contrast, homogeneity, and energy, are calculated and combined into a vector. , which can be expressed as:

[0094]

[0095] where , , , are the four feature quantities of correlation, contrast, homogeneity, and energy respectively, and is the index of the candidate region. At the same time, each element in the gray-level co-occurrence matrix can be expressed as , and taking different values can form different combinations. In the present invention, according to the characteristics of the weak feature image, , are selected. Therefore, a total of 4 combinations can be formed: , , , . Because the direction corresponding to the gray-level co-occurrence matrix is two-way, the above 4 combinations correspond to eight directions of the image, namely horizontal, vertical, and inclined (left and right), as shown in Figure 3 . Thus, a total of 4 vectors of gray-level co-occurrence matrix feature quantities can be generated, which can be respectively expressed as , , , .

[0096] The annotation vector of the candidate region in the YOLOv3 algorithm is , where the label indicates whether the target is included in the candidate region, represents the position of the candidate region relative to the picture, represents the category label of the object in the candidate region. The gray-level co-occurrence matrix eigenvalue is fused with the annotation vector of YOLOv3 to obtain a new annotation vector :

[0097]

[0098] Step 22: Improve the loss function of the YOLOv3 algorithm.

[0099] The loss function of YOLOv3 is divided into three parts and can be expressed as:

[0100]

[0101] where represents the classification loss, represents the confidence loss, and represents the bounding box loss. In the present invention, the annotation vector incorporates the feature quantities of the gray-level co-occurrence matrix, so the loss function is correspondingly improved as follows:

[0102]

[0103] Among them, is the gray-level co-occurrence matrix loss, and its specific definition is:

[0104]

[0105] Among them, is the index of the candidate region, is the number of candidate regions, is the index of the gray-level co-occurrence matrix category, with a total of 4 types, is the weight corresponding to the element in the gray-level co-occurrence matrix, and different values can be set according to different situations and experimental results. In the present invention, equal-value weights are adopted, .

[0106] In order to test the detection ability of the method of the present invention for surface defects of weak-feature images, its effect was verified through experiments. Taking the cloth surface defect detection scenario as an example, the YOLOv3 algorithm and the algorithm of the present invention were first trained using the same training set to obtain the trained neural network model, and then the traditional defect detection algorithm, the YOLOv3 algorithm, and the algorithm of the present invention were used to detect and compare the test images. The experimental results are shown in Table 1, and the detection effect obtained by the present invention is as Figure 4 , Figure 5 shown. It can be seen from the experimental results that the method of the present invention has the advantages of high accuracy and strong reliability, and can better meet the needs of surface defect detection of weak-feature images.

[0107] Table 1 Comparison of surface defect detection performance of weak-feature images

[0108] Method P(%) R(%) mAP0.5 (%) mAP (%) Traditional detection algorithm 56.8 32.2 42.1 27.6 YOLOv3 algorithm 82.3 44.8 50.9 31.2 Algorithm of the present invention 89.3 49.6 55.6 34.4

[0109] Example 2:

[0110] This example provides a terminal, including a memory, a processor, and a computer program stored on the memory. When the processor executes the program, it implements the steps of the weak-feature surface defect detection method based on the gray-level co-occurrence matrix mentioned in Example 1.

[0111] Example 3:

[0112] This embodiment provides a storage medium storing computer program instructions, which implement the steps of the weak feature surface defect detection method based on the gray-level co-occurrence matrix mentioned in Embodiment 1 when the instructions are executed by a processor.

[0113] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0114] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A weak feature surface defect detection method based on gray level co-occurrence matrix, characterized in that It includes the following steps: Generate a gray-level co-occurrence matrix based on non-uniform threshold segmentation of the image and extract texture features: S11. Gray-scale the source image through an adaptive image gray-scale algorithm and dynamically calculate the weight coefficients based on the histograms of each color channel; S12. Use a clustering algorithm to perform non-uniform segmentation on the pixel thresholds; S13. Generate multi-direction gray-level co-occurrence matrices on the segmented image and extract four texture feature vectors of correlation, contrast, homogeneity, and energy; Construct a YOLOv3 deep learning detection model that fuses texture features: S21. Generate the initial anchor box sizes, extract the multi-scale feature maps of the target image, and construct multiple detection layers corresponding to different scale feature maps; S22. Calculate the gray-level co-occurrence matrix feature vectors for each candidate region and generate the annotation vectors that fuse the gray-level co-occurrence matrices; S23. Construct a loss function based on the texture feature vectors; S24. Screen the final detection results through the non-maximum suppression algorithm and merge the overlapping detection boxes according to the intersection-over-union threshold; The annotation vector that fuses the gray-level co-occurrence matrices described in step S22 is: ； Among them, is the annotation vector of the candidate region in the YOLOv3 algorithm, The label indicates whether the target is included in the candidate region, indicates the position of the candidate region relative to the picture, indicates the class label of the object in the candidate region; They are four characteristic quantities: correlation, contrast, homogeneity, and energy, which is the distance of pixels in the original image , and set the direction with different combinations; The loss function based on the texture feature vectors described in step S23 is: ； where is the classification loss, is the confidence loss, is the bounding box loss, is the gray-level co-occurrence matrix loss, and ； where is the index of the candidate region, S is the number of candidate regions, is the index of the gray-level co-occurrence matrix category, with a total of 4 types, is the weight corresponding to the elements in the gray-level co-occurrence matrix, and equal-value weights are used, .

2. The method for detecting weak feature surface defects based on a gray-level co-occurrence matrix according to claim 1, wherein : The specific steps of the adaptive image gray-scale algorithm described in step S11 include: Obtain the histogram of each channel of the image ; Perform an incremental sort on the histograms of each channel to obtain a new histogram ; Take the pixel values that account for a significant proportion to calculate the weights of each channel, and the formula is: ； Among them, are respectively R, G, B the weights of the channels, is the pixel proportion value of each channel that accounts for a significant proportion, means to take the pixel values in the channel with a proportion of , means to calculate the sum of these pixel values.

3. A weak feature surface defect detection method based on gray level co-occurrence matrix according to claim 1, characterized in that : The clustering algorithm described in step S12 uses the K-Means clustering algorithm, and the objective function is: ； Among them, is the objective function, is the index of the clusters obtained by histogram clustering, is the index of the histogram elements in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, is the value of each histogram element, is the average value of the histogram elements in each cluster.

4. A weak feature surface defect detection method based on a gray-level co-occurrence matrix according to claim 1, characterized in that : The method for generating multi-direction gray-level co-occurrence matrices described in step S13 is: Any element in a grayscale image with a grayscale level of M can be expressed as , where is the set distance of pixels in the original image, is the set direction, taking , and respectively represent the row and column indices of the gray-level co-occurrence matrix.

5. The method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 4, characterized in that : The formula for extracting the texture feature vectors is: Correlation: ； Contrast: ； Homogeneity: ； Energy: ； where and are the row and column indices in the gray-level co-occurrence matrix respectively, is the value in the normalized gray-level co-occurrence matrix, and are the average value and standard deviation of the pixel values respectively.

6. A terminal, characterized in that, It includes a memory, a processor, and a computer program stored on the memory. When the processor executes the program, it implements the steps of the method described in any one of claims 1-5.

7. A storage medium, characterized in that, Stores computer program instructions, and when the instructions are executed by the processor, it implements the steps of the method described in any one of claims 1-5.

Images