A method for handling uncertainty in multi-defect classification

Abstract

The present invention provides a method for processing the uncertainty of multi-defect classification, which relates to the field of image processing technology, including obtaining and preprocessing the image to be processed to obtain a target image; extracting the defect area in the target image by statistical means to obtain an initial area result; performing morphological processing on the initial area result to obtain a complete defect area, and then performing main defect area screening and background processing to obtain a first result; performing defect classification on the first result based on a deep learning network to obtain a classification result. After extracting and morphologically processing the defect area in the image to obtain a complete defect area, the main defect area screening and background processing are performed to obtain a first result; performing defect classification on the first result based on a deep learning network to obtain a classification result, the defect location, main defect identification and background processing algorithms are comprehensively applied to improve the classification accuracy and reliability of multiple defects in the same image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for processing uncertainty in multi-defect classification. Background Art

[0002] At present, product appearance defect detection in industrial vision is the most widely used function in the surface inspection industry. However, due to the different types of defects, especially when there are multiple defects in a single image, cross-classification interference often occurs. Traditional methods often find it difficult to solve the classification uncertainty problem of multiple defects in the same image.

[0003] Therefore, the present invention provides a method for processing the uncertainty of multi-defect classification, which effectively improves the classification accuracy of multi-defect images through the combined use of defect location, main defect identification and background processing. Summary of the invention

[0004] The present invention provides a method for processing the uncertainty of multi-defect classification, which is used to obtain a complete defect area by extracting and morphologically processing the defect area in the image, and then screening the main defect area and processing the background to obtain a first result; based on a deep learning network, the first result is defect classified to obtain a classification result, thereby realizing the comprehensive use of defect positioning, main defect identification and background processing algorithms to improve the classification accuracy and reliability of multiple defects appearing in the same image.

[0005] The present invention provides a method for processing uncertainty in multi-defect classification, comprising:

[0006] Step 1: Acquire and preprocess the image to be processed to obtain the target image;

[0007] Step 2: extracting the defect area in the target image by statistical method to obtain the initial area result;

[0008] Step 3: Perform morphological processing on the initial region result to obtain a complete defect region, then perform main defect region screening and background processing to obtain a first result;

[0009] Step 4: Classify the defects of the first result based on the deep learning network to obtain a classification result.

[0010] Preferably, obtaining and preprocessing the image to be processed to obtain the target image includes:

[0011] Obtaining an original grayscale image of the image to be processed, and scanning each pixel of the original grayscale image in sequence to obtain a grayscale histogram of the original grayscale image;

[0012] Determining distribution coefficients of different gray levels in the grayscale histogram, and adjusting the distribution coefficients;

[0013] ;in, represents the adjusted coefficient; represents the distribution coefficient; n1 represents the number of pixels at the corresponding gray level, and n1 is a variable; n is the total number of pixels in the corresponding gray histogram; Represents the brightness deviation of all pixel brightness values ​​at the corresponding grayscale level; Represents the average brightness value of all pixel brightness values ​​at the corresponding grayscale level; Indicates the middle brightness value of the brightness range corresponding to the corresponding gray level; Indicates the limit threshold;

[0014] Based on the adjusted coefficients and combined with the histogram equalization principle, the mapping relationship between the pixel input of the grayscale histogram and the pixel output of the enhanced image is determined to obtain the enhanced image:

[0015] The enhanced image is processed by Gaussian filtering to obtain a target image.

[0016] Preferably, the defect area in the target image is extracted by a statistical method to obtain a preliminary area result, including:

[0017] Obtaining a target mean and a target variance of the target image, and calculating a segmentation threshold T according to the obtained target mean and target variance to obtain a binary image G;

[0018] The calculation formula of the segmentation threshold T is as follows:

[0019] ; Where T represents the segmentation threshold; mean represents the target mean of the target image; M represents the threshold coefficient; dev represents the target variance of the target image;

[0020] The coordinates of the pixels with a grayscale of 255 in the binary image G are gathered together to obtain an initial region result.

[0021] Preferably, the initial region result is subjected to morphological processing to obtain a complete defect region, and then main defect region screening and background processing are performed to obtain a first result, including:

[0022] Performing morphological processing on the initial region result, filtering noise points and extracting connected regions;

[0023] By analyzing the regional characteristics of the connected area, defects adjacent to the connected area are selected for expansion and merging, and then the connected area is obtained again to obtain a merged defect area;

[0024] Performing an intersection operation on the merged defect area and the screened defects to obtain a complete defect area;

[0025] If there is only one complete defective area, the current image is output as the first result;

[0026] If there are two or more complete defect areas, the preset screening rules are used to screen and obtain the main defect area;

[0027] Performing mirror expansion processing on the main defect area to retain the background information adjacent to the main defect, and obtaining an image of the main defect area including the local background;

[0028] Extracting and dividing the non-main defect area in the image into blocks to obtain a first background block;

[0029] The first background block containing the main defect is removed, and the remaining first background blocks are sorted in ascending order according to the variance, and the first background block with the smallest variance is selected as the target background block;

[0030] Calculate the size of the original image, and then scale the image of the target background block to be consistent with the original image to obtain the target background image;

[0031] The main defect area image is fused with the target background image to obtain a fused image, and the fused image is output as a first result.

[0032] Preferably, defect classification is performed on the first result based on a deep learning network to obtain a classification result, including:

[0033] Select appropriate sample data to train the ResNet model to obtain a defect classification model;

[0034] Use the test sample data to input the defect classification model to perform defect classification and obtain the test classification result;

[0035] Compare and analyze the obtained test classification results with the actual classification results of the corresponding test samples to obtain the performance evaluation index of the defect classification model;

[0036] If the performance evaluation index of the defect classification model is not less than a set performance evaluation threshold, inputting the first result into the defect classification model for classification to obtain a classification result;

[0037] If the performance evaluation index of the defect classification model is less than a set performance evaluation threshold, optimizing the defect classification model to obtain an optimized defect classification model;

[0038] The first result is defect-classified using the optimized defect classification model to obtain a classification result.

[0039] Preferably, the obtained test classification results are compared and analyzed with the actual classification results of the corresponding test samples to obtain the performance evaluation index of the defect classification model, including:

[0040] Determine the accuracy, precision and recall of each test based on the comparison between the obtained test classification results and the actual classification results of the corresponding test samples;

[0041] The performance evaluation index of the defect classification model is determined by calculating the comprehensive average of the test accuracy, precision and recall;

[0042] The calculation formula of the performance evaluation index is as follows:

[0043] ; In the formula, It is expressed as the performance evaluation index of the defect classification model; It is expressed as the comprehensive average of the precision rate; It is expressed as the composite average of recall rate; It is expressed as the weight of the comprehensive measurement score of precision and recall on performance evaluation; It is expressed as the influence weight of accuracy on performance evaluation; It is expressed as the number of sample categories with a large proportion; Expressed as the total number of sample categories; It is expressed as a balancing factor for balancing accuracy; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is greater than the preset proportion; It is expressed as the number of sample categories whose proportion is not greater than the preset proportion; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is not greater than the preset proportion.

[0044] Preferably, the defect classification model is optimized to obtain an optimized defect classification model, including:

[0045] Based on the analysis of the test classification results, the parameters in the defect classification model are tuned in combination with the grid search method to obtain the optimized parameters;

[0046] A CA attention mechanism module is added to the defect classification model, and then the traditional convolution in the model is replaced with a depth-separable convolution. The optimization parameters are used for training to obtain an initial optimization model, and the network weights are output;

[0047] After normalizing the network weights, neurons whose weight mean value of each layer in the initial optimization model is less than a set weight threshold and whose difference value is greater than a preset difference value are deleted as junk neurons;

[0048] The neurons whose weight mean of each layer in the initial optimization model is less than the set weight threshold and whose difference is not greater than the preset difference are regarded as pending neurons;

[0049] Inputting a preset test sample into an initial optimization model of initial deletion of neurons, and obtaining a first average classification accuracy and a first performance evaluation index of the initial optimization model of initial deletion of neurons according to the test result;

[0050] If the first performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the initial deleted neurons is used as the final optimized defect classification model;

[0051] Otherwise, the number of predictions y is determined based on the first average classification accuracy of the initial optimization model with the initial neurons deleted;

[0052] After arranging the undetermined neurons obtained in each layer of the initial optimization model in order from large to small according to the weight mean, the remaining undetermined neurons in each layer except the first y undetermined neurons are treated as junk neurons and continue to be deleted to obtain the initial optimization model with neurons deleted;

[0053] Using preset test samples to perform a performance test on the initial optimization model of the deleted neurons, and obtaining a second average classification accuracy and a second performance evaluation index of the initial optimization model of the deleted neurons;

[0054] If the second performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the neurons deleted is used as the final optimized defect classification model;

[0055] Otherwise, the weight mean of the first y undetermined neurons in each layer of the initial optimization model is used to adjust the set weight threshold to obtain a new weight threshold;

[0056] The calculation formula of the new weight threshold is as follows:

[0057] ; In the formula, Represented as the new weight threshold of the x-th layer; Represented as the set weight threshold for the x-th layer; It is expressed as the average of the weight means of the first n undetermined neurons in the x-th layer; It is expressed as the average classification accuracy of the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the performance evaluation index of the original defect classification model; The number of deleted neurons in the xth layer of the initial optimization model represented by the initial deleted neurons; It is represented by the number of deleted neurons in the xth layer of the initial optimization model with neurons deleted; It is represented by the number of neurons in the xth layer of the original defect classification model; It is represented as the average value of the weight means of all neurons in the xth layer of the initial optimization model with the initial neurons deleted; It is represented as the average value of all neuron weight means in the xth layer of the initial optimization model after neurons are deleted; Expressed as the coefficient of influence of model loss on the calculation weight threshold; It is expressed as a balancing factor that considers the model’s fitting ability;

[0058] The new weight threshold is used to screen the neurons in each layer of the initial optimization model, and the neurons whose weight mean is less than the new weight threshold are regarded as junk neurons, and continue to delete them to obtain the final optimized defect classification model.

[0059] Preferably, the optimization parameters include the number of neurons in each layer, the number of iterations, batch, dropout, etc.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] After the complete defect area is obtained by extracting and morphologically processing the defect area in the image, the main defect area is screened and the background is processed to obtain a first result; the first result is defect classified based on a deep learning network to obtain a classification result, which realizes the comprehensive use of defect positioning, main defect identification and background processing algorithms to improve the classification accuracy and reliability of multiple defects in the same image.

[0062] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0063] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 A flowchart of a method for processing uncertainty in multi-defect classification according to an embodiment of the present invention;

[0066] Figure 2 The image is obtained by extracting and dividing the non-main defect area of ​​the image into blocks in the embodiment of the present invention;

[0067] Figure 3 is the target background block image in the embodiment of the present invention;

[0068] Figure 4 is the target background image in the embodiment of the present invention;

[0069] Figure 5 It is a fused image in the embodiment of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, such as Figure 1 As shown, including:

[0072] Step 1: Acquire and preprocess the image to be processed to obtain the target image;

[0073] Step 2: extracting the defect area in the target image by statistical method to obtain the initial area result;

[0074] Step 3: Perform morphological processing on the initial region result to obtain a complete defect region, then perform main defect region screening and background processing to obtain a first result;

[0075] Step 4: Classify the defects of the first result based on the deep learning network to obtain a classification result.

[0076] In this embodiment, the image to be processed refers to an image that has defects and requires defect classification and identification; the target image is an image obtained after contrast enhancement and Gaussian filtering of the image to be processed; the initial region result refers to the initial defect region obtained by region extraction based on the target mean and target variance of the target image.

[0077] In this embodiment, morphological processing refers to expansion and merging processing; the complete defect area is the area obtained by morphological processing of the initial area result; the main defect area is obtained by screening the complete defect area according to preset screening rules; background processing includes main defect morphological processing, non-main defect area background splitting, background area screening, target block scaling and main defect and background image fusion; the first result refers to the only complete defect area or the image after the initial main defect and background image are screened from multiple complete defect areas; the deep learning network is an artificial neural network with multiple hidden layers between the input layer and the output layer, which can be used for image processing; the classification result refers to the recognition result of the main defect category in the image.

[0078] The beneficial effects of the above technical solution are: after obtaining the complete defect area by extracting and morphologically processing the defect area in the image, the main defect area is screened and the background is processed to obtain a first result; the first result is defect-classified based on a deep learning network to obtain a classification result, thereby achieving the comprehensive use of defect positioning, main defect identification and background processing algorithms to improve the classification accuracy and reliability of multiple defects in the same image.

[0079] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which obtains and pre-processes an image to be processed to obtain a target image, including:

[0080] Obtaining an original grayscale image of the image to be processed, and scanning each pixel of the original grayscale image in sequence to obtain a grayscale histogram of the original grayscale image;

[0081] Determining distribution coefficients of different gray levels in the grayscale histogram, and adjusting the distribution coefficients;

[0082] ;in, represents the adjusted coefficient; represents the distribution coefficient; n1 represents the number of pixels at the corresponding gray level, and n1 is a variable; n is the total number of pixels in the corresponding gray histogram; Represents the brightness deviation of all pixel brightness values ​​at the corresponding grayscale level; Represents the average brightness value of all pixel brightness values ​​at the corresponding grayscale level; Indicates the middle brightness value of the brightness range corresponding to the corresponding gray level; Indicates the limit threshold;

[0083] Based on the adjusted coefficients and combined with the histogram equalization principle, the mapping relationship between the pixel input of the grayscale histogram and the pixel output of the enhanced image is determined to obtain the enhanced image:

[0084] The enhanced image is processed by Gaussian filtering to obtain a target image.

[0085] In this embodiment, the image to be processed refers to an image with defects that requires defect classification and identification; the original grayscale image refers to an image in which each pixel in the image to be processed has only one sampled color; and the grayscale histogram is a statistic of the grayscale level distribution in the original grayscale image.

[0086] In this embodiment, the histogram equalization principle refers to the process of converting an image into another image with a balanced histogram through grayscale transformation, that is, having the same number of pixels at each grayscale level; the enhanced image refers to the image obtained after the processed image is processed using histogram equalization; the target image is the image obtained after the processed image is contrast enhanced and Gaussian filtered.

[0087] The beneficial effect of the above technical solution is: by enhancing the image to be processed based on the histogram equalization principle and then using Gaussian filtering to denoise it, the target image is obtained, which is beneficial to subsequent image analysis and ensures the recognition effect.

[0088] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which extracts defect areas in the target image in a statistical manner to obtain preliminary area results, including:

[0089] Obtaining a target mean and a target variance of the target image, and calculating a segmentation threshold T according to the obtained target mean and target variance to obtain a binary image G;

[0090] The calculation formula of the segmentation threshold T is as follows:

[0091] ; Where T represents the segmentation threshold; mean represents the target mean of the target image; M represents the threshold coefficient; dev represents the target variance of the target image;

[0092] The coordinates of the pixels with a grayscale of 255 in the binary image G are gathered together to obtain an initial region result.

[0093] In this embodiment, the target image is an image obtained after contrast enhancement and Gaussian filtering of the processed image; the target mean is data used to reflect the brightness of the target image; and the target variance refers to the grayscale value of each pixel in the target image minus the sum of the squares of the average grayscale value of the target image divided by the total number of pixels.

[0094] In this embodiment, the segmentation threshold T is used to divide the pixels in the target image into several categories; the binary image G refers to a target image in which each pixel has only two possible values ​​or grayscale levels, such as a black and white image; the initial area result refers to the area obtained by collecting the coordinates of the pixel points with a grayscale of 255 in the binary image G.

[0095] The beneficial effects of the above technical solution are: the segmentation threshold is obtained by calculating the mean and variance of the acquired target image, and a binary image is determined; and then the initial area result is determined by analyzing the pixel values ​​of the binary image, thereby ensuring accurate extraction of the defect area.

[0096] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which performs morphological processing on the initial region result to obtain a complete defect region, and then performs main defect region screening and background processing to obtain a first result, including:

[0097] Performing morphological processing on the initial region result, filtering noise points and extracting connected regions;

[0098] By analyzing the regional characteristics of the connected area, defects adjacent to the connected area are selected for expansion and merging, and then the connected area is obtained again to obtain a merged defect area;

[0099] Performing an intersection operation on the merged defect area and the screened defects to obtain a complete defect area;

[0100] If there is only one complete defective area, the current image is output as the first result;

[0101] If there are two or more complete defect areas, the preset screening rules are used to screen and obtain the main defect area;

[0102] Performing mirror expansion processing on the main defect area to retain the background information adjacent to the main defect, and obtaining an image of the main defect area including the local background;

[0103] Extracting and dividing the non-main defect area in the image into blocks to obtain a first background block;

[0104] The first background block containing the main defect is removed, and the remaining first background blocks are sorted in ascending order according to the variance, and the first background block with the smallest variance is selected as the target background block;

[0105] Calculate the size of the original image, and then scale the image of the target background block to be consistent with the original image to obtain the target background image;

[0106] The main defect area image is fused with the target background image to obtain a fused image, and the fused image is output as a first result.

[0107] In this embodiment, the initial area result refers to the area obtained by the set of pixel point coordinates with a pixel grayscale of 255 in the binary image G; the morphological processing refers to the expansion and merging processing; the connected area refers to the area composed of pixels with the same pixel value and adjacent positions in the image; the regional features include perimeter, area, center distance, etc.; the merged defect area refers to the defect obtained by expanding and merging the defects adjacent to the connected area and then obtaining the connected area again; the complete defect area refers to the defect area obtained by performing an intersection operation on the merged defect area and the screened defect; the first result refers to the only complete defect area or the image after the initial main defect and background image are screened from multiple complete defect areas.

[0108] In this embodiment, the preset screening rules are set in advance, for example, the defect closest to the center of the image is the main defect; the main defect area refers to the defect area obtained by screening from multiple complete defect areas using the preset screening rules.

[0109] In this embodiment, the first background block refers to a block obtained by extracting and dividing the non-main defect area in the image, such as Figure 2 As shown; the target background block refers to the first background block that does not contain the main defect and has the smallest variance, such as Figure 3 As shown in , the wireframe block is the target background block; the target background image refers to an image obtained by scaling the target background block to the same size as the original image based on the size of the original image, such as Figure 4 As shown; the fused image is an image obtained by fusing the main defect area image with the target background image, such as Figure 5 shown.

[0110] The beneficial effects of the above technical solution are: by performing morphological processing on the initial area results, a complete defect area is obtained; after the main defect area is determined in combination with preset screening rules, main defect morphological processing, non-main defect area background splitting, background area screening, target block scaling, and main defect and background image fusion are performed to ensure that only the image of the main defect is retained, effectively avoiding interference from other non-main defects.

[0111] An embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which classifies the first result based on a deep learning network to obtain a classification result, including:

[0112] Select appropriate sample data to train the ResNet model to obtain a defect classification model;

[0113] Use the test sample data to input the defect classification model to perform defect classification and obtain the test classification result;

[0114] Compare and analyze the obtained test classification results with the actual classification results of the corresponding test samples to obtain the performance evaluation index of the defect classification model;

[0115] If the performance evaluation index of the defect classification model is not less than a set performance evaluation threshold, inputting the first result into the defect classification model for classification to obtain a classification result;

[0116] If the performance evaluation index of the defect classification model is less than a set performance evaluation threshold, optimizing the defect classification model to obtain an optimized defect classification model;

[0117] The first result is defect-classified using the optimized defect classification model to obtain a classification result.

[0118] In this embodiment, suitable sample data refers to enhanced sample data obtained by extracting a preset amount of first sample data containing all defect categories from a defect sample library and performing data enhancement processing; the ResNet model is a deep residual network structure that can be used for image recognition; the defect classification model is used to perform defect recognition and classification on the input image; and the test classification result is the result obtained by inputting the test sample data into the defect classification model for defect classification.

[0119] In this embodiment, the actual classification result refers to the real defect type of the test sample data; the performance evaluation index is used to evaluate the defect classification effect of the defect classification model; and the performance evaluation threshold is set in advance.

[0120] The beneficial effects of the above technical solution are: by selecting appropriate sample data to train the ResNet model to obtain a defect classification model, and using test sample data to test the model performance of the defect classification model; then by analyzing the calculated performance evaluation index, the final defect classification model is determined to classify the defects of the first result, which can effectively improve the classification accuracy and reliability of multiple defects in the same image.

[0121] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which compares and analyzes the obtained test classification results with the actual classification results of the corresponding test samples to obtain a performance evaluation index of the defect classification model, including:

[0122] Determine the accuracy, precision and recall of each test based on the comparison between the obtained test classification results and the actual classification results of the corresponding test samples;

[0123] The performance evaluation index of the defect classification model is determined by calculating the comprehensive average of the test accuracy, precision and recall;

[0124] The calculation formula of the performance evaluation index is as follows:

[0125] ; In the formula, It is expressed as the performance evaluation index of the defect classification model; It is expressed as the comprehensive average of the precision rate; It is expressed as the composite average of recall rate; It is expressed as the weight of the comprehensive measurement score of precision and recall on performance evaluation; It is expressed as the influence weight of accuracy on performance evaluation; It is expressed as the number of sample categories with a large proportion; Expressed as the total number of sample categories; It is expressed as a balancing factor for balancing accuracy; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is greater than the preset proportion; It is expressed as the number of sample categories whose proportion is not greater than the preset proportion; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is not greater than the preset proportion.

[0126] In this embodiment, the precision rate represents the proportion of true positive examples among all samples output as positive; the recall rate refers to the proportion of the number of samples output as positive examples among the real samples; the performance evaluation index is used to evaluate the defect classification effect of the defect classification model.

[0127] The beneficial effect of the above technical solution is: by determining the accuracy, precision and recall of each test based on the comparison results of the obtained test classification results with the actual classification results of the corresponding test samples, the performance evaluation index of the model is obtained, thereby achieving effective evaluation of the defect classification effect of the defect classification model.

[0128] The embodiment of the present invention provides a method for processing uncertainty in multi-defect classification, which optimizes the defect classification model to obtain an optimized defect classification model, including:

[0129] Based on the analysis of the test classification results, the parameters in the defect classification model are tuned in combination with the grid search method to obtain the optimized parameters;

[0130] A CA attention mechanism module is added to the defect classification model, and then the traditional convolution in the model is replaced with a depth-separable convolution. The optimization parameters are used for training to obtain an initial optimization model, and the network weights are output;

[0131] After normalizing the network weights, neurons whose weight mean value of each layer in the initial optimization model is less than a set weight threshold and whose difference value is greater than a preset difference value are deleted as junk neurons;

[0132] The neurons whose weight mean of each layer in the initial optimization model is less than the set weight threshold and whose difference is not greater than the preset difference are regarded as pending neurons;

[0133] Inputting a preset test sample into an initial optimization model of initial deletion of neurons, and obtaining a first average classification accuracy and a first performance evaluation index of the initial optimization model of initial deletion of neurons according to the test result;

[0134] If the first performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the initial deleted neurons is used as the final optimized defect classification model;

[0135] Otherwise, the number of predictions y is determined based on the first average classification accuracy of the initial optimization model with the initial neurons deleted;

[0136] After arranging the undetermined neurons obtained in each layer of the initial optimization model in order from large to small according to the weight mean, the remaining undetermined neurons in each layer except the first y undetermined neurons are treated as junk neurons and continue to be deleted to obtain the initial optimization model with neurons deleted;

[0137] Using preset test samples to perform a performance test on the initial optimization model of the deleted neurons, and obtaining a second average classification accuracy and a second performance evaluation index of the initial optimization model of the deleted neurons;

[0138] If the second performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the neurons deleted is used as the final optimized defect classification model;

[0139] Otherwise, the weight mean of the first y undetermined neurons in each layer of the initial optimization model is used to adjust the set weight threshold to obtain a new weight threshold;

[0140] The calculation formula of the new weight threshold is as follows:

[0141] ; In the formula, Represented as the new weight threshold of the x-th layer; Represented as the set weight threshold for the x-th layer; It is expressed as the average of the weight means of the first n undetermined neurons in the x-th layer; It is expressed as the average classification accuracy of the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the performance evaluation index of the original defect classification model; The number of deleted neurons in the xth layer of the initial optimization model represented by the initial deleted neurons; It is represented by the number of deleted neurons in the xth layer of the initial optimization model with neurons deleted; It is represented by the number of neurons in the xth layer of the original defect classification model; It is represented as the average value of the weight means of all neurons in the xth layer of the initial optimization model with the initial neurons deleted; It is represented as the average value of all neuron weight means in the xth layer of the initial optimization model after neurons are deleted; Expressed as the coefficient of influence of model loss on the calculation weight threshold; It is expressed as a balancing factor that considers the model’s fitting ability;

[0142] The new weight threshold is used to screen the neurons in each layer of the initial optimization model, and the neurons whose weight mean is less than the new weight threshold are regarded as junk neurons, and continue to delete them to obtain the final optimized defect classification model.

[0143] In this embodiment, the grid search method is a very useful mathematical method in the field of machine learning and pattern recognition, which can be used to determine the optimal parameter combination to improve the model performance; including the number of neurons in each layer, the number of iterations, batch, dropout, etc.

[0144] In this embodiment, the CA attention mechanism module is used to reduce the weight of redundant information in the model and improve the accuracy of model recognition; the deep separable convolution is used to reduce the number of network parameters without affecting the classification accuracy, thereby providing the possibility for the deployment of subsequent models.

[0145] In this embodiment, the initial optimization model refers to adding a CA attention mechanism module to the defect classification model, replacing the traditional convolution in the model with a depth-separable convolution, and then training the model using the optimization parameters to obtain the initial optimization model; the weight threshold is set in advance.

[0146] In this embodiment, the preset difference is set in advance; a junk neuron refers to an input neuron whose average value of the absolute value of the weight of all output neurons in the corresponding layer is less than the set weight threshold and the difference is greater than the preset difference; a pending neuron refers to an input neuron whose average value of the absolute value of the weight of all output neurons in the corresponding layer is less than the set weight threshold and the difference is not greater than the preset difference.

[0147] In this embodiment, the preset test samples are set in advance; the initial optimization model for initial deletion of neurons refers to the model obtained after deleting junk neurons and pending neurons in each layer of the initial optimization model; the first average classification accuracy refers to the average value of the model classification accuracy obtained by testing the initial optimization model for initial deletion of neurons using the preset test samples; the first performance evaluation index is used to evaluate the defect classification effect of the initial optimization model for initial deletion of neurons.

[0148] In this embodiment, the performance evaluation threshold is set in advance; the predicted number y is estimated based on the first average classification accuracy of the initial optimization model of the initial deletion of neurons; the initial optimization model of the re-deleted neurons refers to the model obtained after deleting each layer of junk neurons and the remaining pending neurons except the first y pending neurons from the initial optimization model; the second average classification accuracy refers to the average value of the model classification accuracy obtained by testing the initial optimization model of the re-deleted neurons using preset test samples; the second performance evaluation index is used to evaluate the defect classification effect of the initial optimization model of the re-deleted neurons; the new weight threshold is obtained by adjusting the set weight threshold in combination with the model performance evaluation index, the average classification accuracy and the weight mean data of the neurons.

[0149] The beneficial effects of the above technical solution are: by optimizing the parameters of the defect classification model, adding a CA attention mechanism module, and replacing the traditional convolution in the model with a depth-separable convolution, junk neurons are deleted to achieve streamlined processing of the model, and the final optimized defect classification model is obtained for defect classification and identification, which effectively improves the classification accuracy and reliability of multiple defects in the same image.

[0150] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for handling uncertainty in multi-defect classification, characterized in that: include: Step 1: Acquire and preprocess the image to be processed to obtain the target image; Step 2: extracting the defect area in the target image by statistical method to obtain the initial area result; Step 3: Perform morphological processing on the initial region result to obtain a complete defect region, then perform main defect region screening and background processing to obtain a first result; Step 4: classify the first result based on the deep learning network to obtain a classification result; Wherein, step 3 includes: Performing morphological processing on the initial region result, filtering noise points and extracting connected regions; By analyzing the regional characteristics of the connected area, defects adjacent to the connected area are selected for expansion and merging, and then the connected area is obtained again to obtain a merged defect area; Performing an intersection operation on the merged defect area and the screened defects to obtain a complete defect area; If there is only one complete defective area, the current image is output as the first result; If there are two or more complete defect areas, the preset screening rules are used to screen and obtain the main defect area; Performing mirror expansion processing on the main defect area to retain the background information adjacent to the main defect, and obtaining an image of the main defect area including the local background; Extracting and dividing the non-main defect area in the image into blocks to obtain a first background block; The first background block containing the main defect is removed, and the remaining first background blocks are sorted in ascending order according to the variance, and the first background block with the smallest variance is selected as the target background block; Calculate the size of the original image, and then scale the image of the target background block to be consistent with the original image to obtain the target background image; Fusing the main defect area image with the target background image to obtain a fused image, and outputting the fused image as a first result; Wherein, step 4 includes: Select appropriate sample data to train the ResNet model to obtain a defect classification model; Use the test sample data to input the defect classification model to perform defect classification and obtain the test classification result; Compare and analyze the obtained test classification results with the actual classification results of the corresponding test samples to obtain the performance evaluation index of the defect classification model; If the performance evaluation index of the defect classification model is not less than a set performance evaluation threshold, inputting the first result into the defect classification model for classification to obtain a classification result; If the performance evaluation index of the defect classification model is less than a set performance evaluation threshold, optimizing the defect classification model to obtain an optimized defect classification model; Using the optimized defect classification model to classify the first result, to obtain a classification result; The defect classification model is optimized to obtain an optimized defect classification model, including: Based on the analysis of the test classification results, the parameters in the defect classification model are tuned in combination with the grid search method to obtain the optimized parameters; A CA attention mechanism module is added to the defect classification model, and then the traditional convolution in the model is replaced with a depth-separable convolution. The optimization parameters are used for training to obtain an initial optimization model, and the network weights are output; After normalizing the network weights, neurons whose weight mean value of each layer in the initial optimization model is less than a set weight threshold and whose difference value is greater than a preset difference value are deleted as junk neurons; The neurons whose weight mean of each layer in the initial optimization model is less than the set weight threshold and whose difference is not greater than the preset difference are regarded as pending neurons; Inputting a preset test sample into an initial optimization model of initial deletion of neurons, and obtaining a first average classification accuracy and a first performance evaluation index of the initial optimization model of initial deletion of neurons according to the test result; If the first performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the initial deleted neurons is used as the final optimized defect classification model; Otherwise, the number of predictions y is determined based on the first average classification accuracy of the initial optimization model with the initial neurons deleted; After arranging the undetermined neurons obtained in each layer of the initial optimization model in order from large to small according to the weight mean, the remaining undetermined neurons in each layer except the first y undetermined neurons are treated as junk neurons and continue to be deleted to obtain the initial optimization model with neurons deleted; Using preset test samples to perform a performance test on the initial optimization model of the deleted neurons, and obtaining a second average classification accuracy and a second performance evaluation index of the initial optimization model of the deleted neurons; If the second performance evaluation index is not less than the set performance evaluation threshold, the initial optimization model with the neurons deleted is used as the final optimized defect classification model; Otherwise, the weight mean of the first y undetermined neurons in each layer of the initial optimization model is used to adjust the set weight threshold to obtain a new weight threshold; The calculation formula of the new weight threshold is as follows: ; In the formula, Represented as the new weight threshold of the x-th layer; Represented as the set weight threshold for the x-th layer; It is expressed as the average of the weight means of the first n undetermined neurons in the x-th layer; It is expressed as the average classification accuracy of the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in average classification accuracy between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model with the initial neurons deleted and the original defect classification model; It is expressed as the difference in performance evaluation index between the initial optimization model after deleting neurons and the original defect classification model; It is expressed as the performance evaluation index of the original defect classification model; The number of deleted neurons in the xth layer of the initial optimization model represented by the initial deleted neurons; It is represented by the number of deleted neurons in the xth layer of the initial optimization model with neurons deleted; It is represented by the number of neurons in the xth layer of the original defect classification model; It is represented as the average value of all neuron weight means in the xth layer of the initial optimization model with the initial neurons deleted; It is represented as the average value of all neuron weight means in the xth layer of the initial optimization model after neurons are deleted; Expressed as the coefficient of influence of model loss on the calculation weight threshold; It is expressed as a balancing factor that considers the model’s fitting ability; The new weight threshold is used to screen the neurons in each layer of the initial optimization model, and the neurons whose weight mean is less than the new weight threshold are regarded as junk neurons, and continue to delete them to obtain the final optimized defect classification model.

2. A method for processing multi-defect classification uncertainty according to claim 1, characterized in that: Acquire and preprocess the image to be processed to obtain the target image, including: Obtaining an original grayscale image of the image to be processed, and scanning each pixel of the original grayscale image in sequence to obtain a grayscale histogram of the original grayscale image; Determining distribution coefficients of different gray levels in the grayscale histogram, and adjusting the distribution coefficients; ;in, represents the adjusted coefficient; represents the distribution coefficient; n1 represents the number of pixels at the corresponding gray level, and n1 is a variable; n is the total number of pixels in the corresponding gray histogram; Represents the brightness deviation of all pixel brightness values ​​at the corresponding grayscale level; Represents the average brightness value of all pixel brightness values ​​at the corresponding grayscale level; Indicates the middle brightness value of the brightness range corresponding to the corresponding gray level; Indicates the limit threshold; Based on the adjusted coefficients and combined with the histogram equalization principle, the mapping relationship between the pixel input of the grayscale histogram and the pixel output of the enhanced image is determined to obtain the enhanced image: The enhanced image is processed by Gaussian filtering to obtain a target image.

3. A method for processing multi-defect classification uncertainty according to claim 1, characterized in that: The defect area in the target image is extracted by statistical method to obtain the initial area result, including: Obtaining a target mean and a target variance of the target image, and calculating a segmentation threshold T according to the obtained target mean and target variance to obtain a binary image G; The calculation formula of the segmentation threshold T is as follows: ; Where T represents the segmentation threshold; mean represents the target mean of the target image; M represents the threshold coefficient; dev represents the target variance of the target image; The coordinates of the pixels with a grayscale of 255 in the binary image G are gathered together to obtain an initial region result.

4. A method for processing multi-defect classification uncertainty according to claim 1, characterized in that: The obtained test classification results are compared and analyzed with the actual classification results of the corresponding test samples to obtain the performance evaluation index of the defect classification model, including: Determine the accuracy, precision and recall of each test based on the comparison between the obtained test classification results and the actual classification results of the corresponding test samples; The performance evaluation index of the defect classification model is determined by calculating the comprehensive average of the test accuracy, precision and recall; The calculation formula of the performance evaluation index is as follows: ; In the formula, It is expressed as the performance evaluation index of the defect classification model; It is expressed as the comprehensive average of the precision rate; It is expressed as the composite average of recall rate; It is expressed as the weight of the comprehensive measurement score of precision and recall on performance evaluation; It is expressed as the influence weight of accuracy on performance evaluation; It is expressed as the number of sample categories with a large proportion; Expressed as the total number of sample categories; It is expressed as a balancing factor for balancing accuracy; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is greater than the preset proportion; It is expressed as the number of sample categories whose proportion is not greater than the preset proportion; It is expressed as the comprehensive average of the accuracy of the test of sample categories whose proportion is not greater than the preset proportion.

5. A method for processing multi-defect classification uncertainty according to claim 1, characterized in that: The optimization parameters include the number of neurons in each layer, the number of iterations, batch and dropout.

Images