Weak feature surface defect detection method based on gray level co-occurrence matrix, terminal and storage medium

Abstract

The invention discloses a weak feature surface defect detection method based on a gray level co-occurrence matrix, a terminal and a storage medium, and belongs to the technical field of image processing, and the method comprises the steps of S1, generating a gray level co-occurrence matrix based on image non-uniform threshold segmentation and extracting texture features, and S2, constructing a YOLOv3 deep learning detection model fusing the texture features. In order to solve the problem of weak feature image surface defect detection, a gray-level co-occurrence matrix and a convolutional neural network technology are combined, a source image is processed by using an adaptive image graying algorithm, and then multi-threshold segmentation of the image is completed by using a non-uniform threshold segmentation algorithm. On this basis, an image gray-level co-occurrence matrix is generated and a corresponding characteristic quantity is calculated, then the characteristic quantity of the gray-level co-occurrence matrix is combined with a YOLOv3 neural network algorithm, and a loss function of the YOLOv3 algorithm is improved, so that the accuracy and the reliability of surface defect detection of the weak-characteristic image are effectively improved, and the detection precision of the weak-characteristic image is improved. Wide application market space and economic value are realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, a terminal and a storage medium for detecting surface defects of weak-feature images based on a gray-level co-occurrence matrix. Background Art

[0002] As an important means to improve the quality of industrial products, defect detection has received more and more attention. Among them, low-recognition surface defect detection is an important research direction in industrial visual inspection, mainly targeting defects with low contrast, complex texture, small size or irregular shape. Surface defects (such as cracks and scratches) directly affect product performance. Early detection can reduce the scrap rate. An excellent detection method is of great significance in improving product quality, reducing production costs, improving production efficiency, and ensuring safety. Currently, commonly used defect detection methods mainly rely on technologies such as image enhancement, feature extraction, and machine learning. Although certain results have been achieved, they are limited in effect when dealing with low-recognition and complex texture defects. In order to overcome the problems of poor detection accuracy and low detection efficiency in current detection methods, this paper aims to solve the problem of low-recognition surface defect detection. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a weak feature surface defect detection method based on gray level co-occurrence matrix, comprising the following steps:

[0004] The steps of generating a gray level co-occurrence matrix and extracting texture features based on image non-uniform threshold segmentation include:

[0005] S11. grayscale the source image using an adaptive image grayscale algorithm, and dynamically calculate the weight coefficient based on the histogram of each color channel;

[0006] S12. Using clustering algorithm to perform non-uniform segmentation on pixel thresholds;

[0007] S13. Generate a multi-directional grayscale co-occurrence matrix on the segmented image and extract four texture feature vectors of correlation, contrast, homogeneity and energy;

[0008] The steps to build a YOLOv3 deep learning detection model that integrates texture features include:

[0009] S21. Generate an initial anchor box size, extract a multi-scale feature map of the target image, and construct multiple detection layers corresponding to feature maps of different scales;

[0010] S22. Calculate the gray level co-occurrence matrix eigenvector for each candidate region and generate a label vector of the fused gray level co-occurrence matrix;

[0011] S23. Constructing a loss function based on texture feature vector;

[0012] S24. The final detection result is filtered through the non-maximum suppression algorithm, and the overlapping detection frames are merged according to the intersection-over-union ratio threshold.

[0013] Furthermore, the specific steps of the adaptive image grayscale algorithm in step S11 include:

[0014] Get the histogram of each channel of the image , , ; Sort the histograms of each channel in ascending order to obtain a new histogram , , ; The pixel values ​​that occupy a significant proportion are taken to calculate the weight of each channel. The formula is:

[0015] in, , , are the weights of the R, G, and B channels respectively, is the pixel ratio value that accounts for a significant proportion of each channel, Indicates that the proportion of the channel is The pixel value of Calculates the sum of these pixel values.

[0016] Furthermore, the clustering algorithm in step S12 adopts the K-Means clustering algorithm, and the objective function is:

[0017]

[0018] in, is the objective function, is the index of the cluster for clustering the histogram, is the index of the histogram element in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, is the value of each histogram element, is the mean of the histogram elements in each cluster.

[0019] Furthermore, the method for generating a multi-directional gray-level co-occurrence matrix in step S13 is:

[0020] Any element in a grayscale image with grayscale level M It can be expressed as ,in is the distance between pixels in the original image. For the set direction, take , and They represent the row and column indices of the gray-level co-occurrence matrix, and also represent the grayscale of the image.

[0021] Furthermore, the texture feature vector extraction formula is:

[0022] Relevance:

[0023] ;

[0024] Contrast ratio:

[0025] ;

[0026] Homogeneity:

[0027] ;

[0028] energy:

[0029] ;

[0030] in , are the pixel levels in the gray-level co-occurrence matrix and the corresponding pixel values ​​in the image, is the value in the normalized gray-level co-occurrence matrix, , are the mean and standard deviation of the pixel values, respectively.

[0031] Furthermore, the label vector of the fused gray-level co-occurrence matrix in step S22 is:

[0032] ;

[0033] in, is the label vector of the candidate region in the YOLOv3 algorithm, The label indicates whether the candidate region contains the target. Indicates the position of the candidate region relative to the image, Represents the category label of the object in the candidate area;

[0034] , , , , They are four characteristic quantities: correlation, contrast, homogeneity and energy. , , , for , Under different combination.

[0035] Furthermore, the loss function based on the texture feature vector described in step S23 is:

[0036] ;

[0037] in is the classification loss, for Confidence loss, is the bounding box loss, is the gray-level co-occurrence matrix loss, and

[0038] ;

[0039] in, is the index of the candidate region, is the number of candidate regions, is the index of the gray-level co-occurrence matrix category, there are 4 types in total. is the weight corresponding to the element in the gray level co-occurrence matrix, which can be set to different values ​​according to different situations and experimental results. In the present invention, equal weight is used. .

[0040] The present invention also provides a terminal, comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the steps in the above method when executing the program.

[0041] The present invention also provides a storage medium storing computer program instructions, which implement the steps in the above method when the instructions are executed by a processor.

[0042] Beneficial effects of the present invention:

[0043] The present invention combines the grayscale co-occurrence matrix with the convolutional neural network technology to solve the problem of surface defect detection in weak feature images, uses an adaptive image grayscale algorithm to process the source image, and then uses a non-uniform threshold segmentation algorithm to complete the multi-threshold segmentation of the image. On this basis, the image grayscale co-occurrence matrix is ​​generated and the corresponding feature quantities are calculated. The feature quantities of the grayscale co-occurrence matrix are then combined with the YOLOv3 neural network algorithm, and the loss function of the YOLOv3 algorithm is improved, thereby effectively improving the accuracy and reliability of surface defect detection in weak feature images, and has a broad application market space and economic value. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 This is a comparison chart of image grayscale using the weighted average method and the adaptive grayscale method;

[0046] Figure 2 This is a comparison chart of image multi-threshold segmentation using uniform threshold segmentation and non-uniform threshold segmentation;

[0047] Figure 3 It is a schematic diagram of the gray-level co-occurrence matrix direction;

[0048] Figure 4 The method of the present invention is effective in identifying surface defects of weak feature images. Figure 1 ;

[0049] Figure 5 The method of the present invention is effective in identifying surface defects of weak feature images. Figure 2 . DETAILED DESCRIPTION

[0050] The present application is described below in conjunction with specific embodiments:

[0051] Embodiment 1:

[0052] This embodiment provides a method for detecting weak-feature surface defects based on a gray-level co-occurrence matrix, including:

[0053] Step S1. Generate gray level co-occurrence matrix based on image non-uniform threshold segmentation and extract texture features;

[0054] Step S2: Build a YOLOv3 deep learning detection model that integrates texture features.

[0055] Grey Level Co-occurrence Matrix (GLCM) is a statistical method used to describe image texture features. It captures the texture information of an image by analyzing the spatial relationship between pixels in the image and the statistical distribution of gray levels. It describes the spatial relationship and texture features between pixels. It reflects the joint probability distribution of the simultaneous occurrence of different gray value combinations at a given spatial distance and direction.

[0056] Although the gray-level co-occurrence matrix can reflect the texture characteristics of the image, its calculation amount is relatively high. For example, for an 8-bit grayscale image, the pixel grayscale level is 256, and the corresponding gray-level co-occurrence matrix size is 256×256, which leads to a large amount of calculation and reduces the operating efficiency of the machine vision system.

[0057] In order to overcome the above defects, the grayscale of the original grayscale image is usually compressed, such as compressing the 256-level pixel grayscale threshold segmentation to 32 levels or 8 levels. To this end, the present invention uses an adaptive non-uniform threshold segmentation method to further threshold segment the grayscale image and compress the grayscale level.

[0058] Therefore, for step S1, specifically:

[0059] S11. grayscale the source image using an adaptive image grayscale algorithm, and dynamically calculate the weight coefficient based on the histogram of each color channel;

[0060] Usually, the image captured by the camera is an RGB color image. When further processing it, it is often necessary to convert it into a grayscale image. The most commonly used image grayscale method is the weighted average method. This method starts from the sensitivity of the human eye to different colors. Its calculation method is:

[0061]

[0062] Among them, R, G, and B are the pixel values ​​of the red, green, and blue channels of the image, respectively. The above method has the characteristic of simple calculation, but the imaging of the surface of a weak feature object usually has fewer color components, most of which present a single or similar color, and each color component tends to be in a relatively concentrated area in a certain channel or color space. For this reason, the present invention adaptively determines the weights of the image grayscale based on each color channel of the image.

[0063] First, get the histogram of each channel of the image and set them as , , , and then sort them in ascending order. The sorted histograms of each channel are set as , , Then, the pixel values ​​that occupy a significant proportion are taken to calculate the weight of each channel, as shown in the following formula:

[0064]

[0065] in, , , are the weights of the R, G, and B channels respectively, The pixel ratio value that occupies a significant proportion for each channel, in the present invention , Indicates that the proportion of the channel is The pixel value of Indicates the calculation of the sum of these pixel values. The effect of image grayscale is as follows Figure 1 As shown, it can be seen that the grayscale method used in this embodiment can better highlight defects in the feature image.

[0066] S12. Use K-Means clustering algorithm to perform non-uniform segmentation on pixel thresholds;

[0067] The pixel threshold segmentation method currently used is usually uniform threshold segmentation, which divides the pixel grayscale range evenly into Each copy covers a pixel value range of ,in is the maximum pixel value in the original image, 0~ A positive integer between , each pixel value that falls within the corresponding range is set to Uniform threshold segmentation is simple to implement, easy to understand and implement, but for the case of non-uniform signal distribution, uniform threshold segmentation often cannot provide sufficient accuracy, especially when facing small signal parts such as material surface images with low recognition, which easily leads to the loss of important information. The present invention uses an improved K-Means clustering algorithm to complete the non-uniform threshold segmentation of the image.

[0068] The K-Means algorithm is a commonly used clustering algorithm and an unsupervised learning method. It is used to divide data into K clusters so that the similarity of data points within a cluster is high, while the similarity between clusters is low.

[0069] First, obtain the histogram of the grayscale image, set as ,at this time contains 256 elements, each of which represents the number of a certain grayscale pixel in the image. For images with weak features, the area with a gentle change in the image often represents the main color of the object, and the number of corresponding colors is larger, while the defective area will have a sudden change in grayscale value, but the number of such pixels is usually small. Based on this, the objective function of the K-Means clustering algorithm used in the present invention is as follows:

[0070]

[0071] in, is the objective function, is the index of the cluster for clustering the histogram, is the index of the histogram element in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, and the present invention takes , thereby compressing the image grayscale value to 32 levels, is the value of each histogram element, is the average value of the histogram elements in each cluster. It means that the variance of the histogram elements in the cluster is taken as the target. The smaller the variance, the smaller the difference between the corresponding pixel grayscale values ​​in the same cluster. Pixels with small differences are divided into one cluster to achieve non-uniform threshold segmentation of the image. The effect is as follows: Figure 2 shown.

[0072] S13. Generate a multi-directional gray-level co-occurrence matrix on the segmented image and extract four texture feature vectors of correlation, contrast, homogeneity and energy.

[0073] The gray-level co-occurrence matrix can be used to calculate 14 feature quantities related to image texture features, of which correlation, contrast, homogeneity and energy are four independent feature quantities. The gray-level co-occurrence matrix is ​​generated as follows: Assume that a gray-level image contains gray levels, then its gray-level co-occurrence matrix is ​​a A square matrix of size, each element in the matrix can be expressed as ,in is the distance between pixels in the original image. is the set direction (bidirectional), usually , and Represent the row and column indices of the gray-level co-occurrence matrix, and also represent the gray level of the image. The elements of the gray-level co-occurrence matrix describe the gray-level image in Direction, distance A pair of pixels have gray levels and The probability of can specifically reflect the texture characteristics of the image.

[0074] Assume the image gray level co-occurrence matrix is , correlation Indicates the similarity of elements in the row and column directions, and is calculated as follows:

[0075]

[0076] Contrast The randomness of the image is measured as follows:

[0077]

[0078] Homogeneity It is used to characterize the regularity of image texture and is calculated as follows:

[0079]

[0080] energy It is used to characterize the complexity of the texture in the image, and its calculation method is:

[0081]

[0082] Mode -Mode middle, , are the pixel levels in the gray-level co-occurrence matrix and the corresponding pixel values ​​in the image, is the value in the normalized gray-level co-occurrence matrix, , are the mean and standard deviation of the pixel values, respectively.

[0083] The present invention uses the above four eigenvalues ​​in combination with a deep learning method to complete defect detection of weak feature images.

[0084] Step S2 builds a YOLOv3 deep learning detection model that integrates texture features.

[0085] Convolutional neural network is a deep learning model specially designed for processing data with grid structure. It has achieved great success in the field of computer vision and is widely used in tasks such as image classification, target detection, semantic segmentation, etc. It effectively reduces the number of parameters through local receptive field and weight sharing mechanism, and can automatically extract hierarchical features of input data. The use of convolutional neural network can generate feature maps of different depths of the image, and the corresponding functions can be realized by further processing these feature maps. The present invention combines the convolutional neural network YOLOv3 with the gray level co-occurrence matrix, so that it can detect defects in weak feature surface images. The specific steps are as follows:

[0086] S21. Generate an initial anchor box size, extract a multi-scale feature map of the target image, and construct multiple detection layers corresponding to feature maps of different scales;

[0087] S22. Calculate the gray level co-occurrence matrix eigenvector for each candidate region and generate a label vector of the fused gray level co-occurrence matrix;

[0088] S23. Constructing a loss function based on texture feature vector;

[0089] S24. The final detection result is filtered through the non-maximum suppression algorithm, and the overlapping detection frames are merged according to the intersection-over-union ratio threshold.

[0090] In the present invention, in addition to marking the candidate regions, the gray level co-occurrence matrix is ​​calculated for each candidate region using step 1, and four feature quantities, namely, correlation, contrast, homogeneity and energy, are calculated and combined into a vector , which can be expressed as:

[0091]

[0092] in, , , , They are four characteristic quantities: correlation, contrast, homogeneity and energy. is the index of the candidate region. At the same time, each element in the gray level co-occurrence matrix can be expressed as , and Different values ​​can form different combinations. In the present invention, according to the characteristics of weak feature images, , , so there are 4 combinations: , , , , because the direction corresponding to the gray-level co-occurrence matrix is ​​bidirectional, the above four combinations correspond to eight directions of the image: horizontal, vertical, and tilted (left and right), such as Figure 3 As shown. Thus, a total of 4 vectors of gray-level co-occurrence matrix feature quantities can be generated, which can be expressed as , , , .

[0093] The label vector of the candidate region in the YOLOv3 algorithm is ,in The label indicates whether the candidate region contains the target. Indicates the position of the candidate region relative to the image, Represents the category label of the object in the candidate area, and merges the eigenvalue of the gray-level co-occurrence matrix with the annotation vector of YOLOv3 to obtain a new annotation vector :

[0094]

[0095] Step 22: Improve the loss function of the YOLOv3 algorithm.

[0096] The loss function of YOLOv3 is divided into 3 parts, which can be expressed as:

[0097]

[0098] in, represents the classification loss, express Confidence loss, In this invention, the label vector is fused with the feature quantity of the gray-level co-occurrence matrix, so the loss function is improved accordingly, as shown below:

[0099]

[0100] in, is the gray-level co-occurrence matrix loss, which is specifically defined as:

[0101]

[0102] in, is the index of the candidate region, is the number of candidate regions, is the index of the gray-level co-occurrence matrix category, there are 4 types in total. is the weight corresponding to the element in the gray level co-occurrence matrix, which can be set to different values ​​according to different situations and experimental results. In the present invention, equal weight is used. .

[0103] In order to test the detection ability of the method of the present invention for surface defects of weak feature images, the effect is verified by experiments. Taking the cloth surface defect detection scenario as an example, the present invention first uses the same training set to train the YOLOv3 algorithm and the algorithm of the present invention to obtain the trained neural network model, and then uses the traditional defect detection algorithm, the YOLOv3 algorithm and the algorithm of the present invention to detect and compare the test image. The experimental results are shown in Table 1. The detection effect obtained by using the present invention is as follows: Figure 4 , Figure 5 It can be seen from the experimental results that the method of the present invention has the advantages of high accuracy and strong reliability, and can better meet the needs of surface defect detection of weak feature images.

[0104] Table 1 Comparison of surface defect detection performance of weak feature images

[0105] method P(%) R(%) mAP0.5(%) mAP(%) Traditional detection algorithms 56.8 32.2 42.1 27.6 YOLOv3 Algorithm 82.3 44.8 50.9 31.2 Algorithm of the present invention 89.3 49.6 55.6 34.4

[0106] Embodiment 2:

[0107] This embodiment provides a terminal, including a memory, a processor, and a computer program stored in the memory. When the processor executes the program, the steps of the weak feature surface defect detection method based on gray level co-occurrence matrix mentioned in Example 1 are implemented.

[0108] Embodiment 3:

[0109] This embodiment provides a storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the gray-level co-occurrence matrix-based weak-feature surface defect detection method mentioned in Embodiment 1.

[0110] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0111] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical scheme of the present invention, and these equivalent transformations are all protected by the present invention.

Claims

1. A method for detecting weak feature surface defects based on gray level co-occurrence matrix, characterized in that: The steps include: Generate gray-level co-occurrence matrix and extract texture features based on image non-uniform threshold segmentation: S11. grayscale the source image using an adaptive image grayscale algorithm, and dynamically calculate the weight coefficient based on the histogram of each color channel; S12. Using clustering algorithm to perform non-uniform segmentation on pixel thresholds; S13. Generate a multi-directional grayscale co-occurrence matrix on the segmented image and extract four texture feature vectors of correlation, contrast, homogeneity and energy; Build a YOLOv3 deep learning detection model that integrates texture features: S21. Generate an initial anchor frame size, extract a multi-scale feature map of the target image, and construct multiple detection layers corresponding to feature maps of different scales; S22. Calculate the gray level co-occurrence matrix eigenvector for each candidate region and generate a label vector of the fused gray level co-occurrence matrix; S23. Constructing a loss function based on texture feature vector; S24. The final detection result is filtered through the non-maximum suppression algorithm, and the overlapping detection frames are merged according to the intersection-over-union ratio threshold.

2. A method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 1, characterized in that : The specific steps of the adaptive image grayscale algorithm in step S11 include: Get the histogram of each channel of the image , , ; Sort the histograms of each channel in ascending order to obtain a new histogram , , ; The pixel values ​​that occupy a significant proportion are taken to calculate the weight of each channel. The formula is: ； in, , , are the weights of the R, G, and B channels respectively, is the pixel ratio value that accounts for a significant proportion of each channel, Indicates that the proportion of the channel is The pixel value of Calculates the sum of these pixel values.

3. The method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 1 is characterized in that : The clustering algorithm in step S12 adopts the K-Means clustering algorithm, and the objective function is: ； in, is the objective function, is the index of the cluster for clustering the histogram, is the index of the histogram element in each cluster, is the number of histogram elements contained in each cluster, is the set number of clusters, is the value of each histogram element, is the mean of the histogram elements in each cluster.

4. The method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 1, characterized in that : The method for generating a multi-directional gray-level co-occurrence matrix in step S13 is: Any element in a grayscale image with grayscale level M It can be expressed as ,in is the distance between pixels in the original image. For the set direction, take , and They represent the row and column indices of the gray-level co-occurrence matrix, and also represent the grayscale of the image.

5. A method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 4, characterized in that : The texture feature vector extraction formula is: Relevance: ； Contrast ratio: ； Homogeneity: ； energy: ； in , are the pixel levels in the gray-level co-occurrence matrix and the corresponding pixel values ​​in the image, is the value in the normalized gray-level co-occurrence matrix, , are the mean and standard deviation of the pixel values, respectively.

6. The method for detecting weak feature surface defects based on gray level co-occurrence matrix according to claim 4 is characterized in that : The label vector of the fused gray-level co-occurrence matrix in step S22 is: ； in, is the label vector of the candidate region in the YOLOv3 algorithm, The label indicates whether the candidate region contains the target. Indicates the position of the candidate region relative to the image, Represents the category label of the object in the candidate area; , , , , They are four characteristic quantities: correlation, contrast, homogeneity and energy. , , , for , Under different combination.

7. A method for detecting weak-feature surface defects based on gray-level co-occurrence matrix according to claim 6, characterized in that : The loss function based on the texture feature vector described in step S23 is: ； in is the classification loss, for Confidence loss, is the bounding box loss, is the gray-level co-occurrence matrix loss, and ； in is the index of the candidate region, is the number of candidate regions, is the index of the gray-level co-occurrence matrix category, there are 4 types in total. is the weight corresponding to the elements in the gray-level co-occurrence matrix, using equal weights. .

8. A terminal, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein when the processor executes the program, the steps of any method of claims 1-7 are implemented.

9. A storage medium, characterized in that: Computer program instructions are stored, and when the instructions are executed by a processor, the steps of any method described in claims 1-7 are implemented.

Images