Industrial defect image detection method based on dynamic regularized entropy differentiable architecture search

Abstract

The present invention provides an industrial defect image detection method based on dynamic regular entropy differentiable architecture search, and relates to the field of image analysis technology. Obtain industrial image data to be detected and classified and construct an industrial image data set, and divide it into a training set and a verification set; construct a differentiable search supernet model for searching the architecture, initialize the network weight ω and the architecture parameter α, and calculate the loss function and the loss model and optimizer for gradient update; perform multiple rounds of training, and use the training set and the verification set in each batch of each round to calculate the loss and update ω and α; after training, discretize α to obtain the neural network architecture of the final search, and retrain the architecture on the entire data set, use it as a classification model for detecting abnormal defects in industrial images, input the actual industrial image to be detected and classified, and obtain the defect detection result. The method of the present invention can search for an architecture with excellent detection performance without performance collapse under long-term search.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to an industrial defect image detection method based on dynamic regular entropy differentiable architecture search. Background Art

[0002] In the process of industrial production, the quality of finished products is easily affected by the deficiencies and limitations of existing technologies, working conditions and other factors. Among them, surface defects are the most intuitive manifestation of product quality being affected. Therefore, in order to ensure the pass rate and reliable quality, product surface defect detection must be carried out. "Defect" can generally be understood as a lack, defect or area compared with a normal sample. Surface defect detection refers to the detection of scratches, defects, foreign body obstruction, color pollution, holes and other defects on the surface of the sample, so as to obtain a series of relevant information such as the category, outline, position, size and other surface defects of the sample being tested. Manual defect detection used to be the mainstream method, but this method is inefficient; the detection results are easily affected by human subjective factors and cannot meet the requirements of real-time detection. In addition, the comprehensive intelligent development of the manufacturing industry has put forward new requirements for the quality inspection of industrial products.

[0003] Nowadays, the rapid development of deep learning has made it more and more widely used in the field of defect detection. In recent years, many researchers have explored the application of various deep learning technologies in industrial defect detection scenarios. The main steps include: data preprocessing, feature extraction, neural network construction, model training and tuning, and image classification, where data preprocessing is used to ensure data quality and consistency. Once the model training is completed, it can be used to classify new images.

[0004] However, current technologies are basically based on artificial neural networks designed manually. This approach is not only time-consuming and laborious, but also requires designers to have rich experience in deep learning model design and professional knowledge of the corresponding industrial image field, making the design of neural network models for specific industrial defect detection in various industrial scenarios extremely difficult, resulting in the difficulty of widespread popularization of deep learning in industrial defect detection. In addition, even excellent neural network designers find it difficult to consider all aspects of the application scenario, design a neural network that is close to the optimal for the corresponding data set and industrial scenario, and maintain a low parameter and low computational load of the model to meet the low latency and low overhead conditions in the application. In recent years, neural network architecture search technology has continued to develop, and the searched network architecture has been able to greatly exceed the performance of the manually designed model, and the model's parameter volume and inference time are much smaller than the manually designed model, enabling it to achieve excellent performance in the application environment of industrial defect detection and better meet the requirements of real-time and low overhead.

[0005] Although neural network architecture search technology has shown broad application prospects, it still has the problem of insufficient generalization in industrial defect detection application scenarios. For example, in the mainstream differentiable neural network architecture search, performance collapse is prone to occur, making the searched architecture prone to skip connections, which greatly reduces the number of key parameterized operations, and ultimately leads to lower industrial image recognition performance. This degeneration phenomenon of the search architecture leads to huge challenges in the application of neural network architecture search in industrial defect detection. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide an industrial defect image detection method based on dynamic regularized entropy differentiable architecture search in response to the above-mentioned deficiencies in the prior art. The method has high search convergence, does not experience performance collapse under long-term search, and can search for an architecture with excellent detection performance.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] An industrial defect image detection method based on dynamic regular entropy differentiable architecture search includes the following steps:

[0009] Step 1: Obtain the industrial image data to be detected and classified and construct an industrial image dataset, and divide the industrial image dataset into a training set and a validation set according to a preset ratio;

[0010] Step 2: Construct the required neural network architecture search space, further obtain the supernet model for differentiable architecture search, and initialize the network weights therein and architecture parameters , as well as the loss model and optimizer for computing the loss function and gradient updates;

[0011] Step 3: In the architecture parameters In the loss calculation of The entropy of the distribution is used as an additional regularization term and the architecture parameters are calculated The zero-overhead index SNIP is normalized inside the search block and used as the dynamic control coefficient of the regularization term;

[0012] Step 4: Perform multiple rounds of training, using the training set and validation set to calculate the loss in each batch of each round and update them separately , ;

[0013] Step 5: After all rounds of training, set the architecture parameters The neural network architecture is discretized and finally searched. The architecture is retrained on the entire data set and used as a classification model for detecting abnormal defects in industrial images. The images to be classified are input into the model to obtain the defect detection results of the industrial images.

[0014] Furthermore, the specific method for constructing the required neural network architecture search space in step 2 is:

[0015] The neural network architecture search space, wherein the optional operations to be searched include depth separation convolution operations, void convolution operations, maximum pooling operations, average pooling operations, and skip connection operations, and the fixed operations include addition operations and connection operations; the optional operations to be searched and the fixed operations are combined into a neural network topology graph convolution block, wherein the operations represented by the topology graph structure and the operation edges are available for search, and a search space based on stacking a fixed number of convolution blocks is constructed;

[0016] The deep separation convolution operation first uses a single-channel high-dimensional filter to filter the image input to the operation, and then uses a multi-channel one-dimensional vector to perform an image transformation operation on the filtered image, replacing each pixel of the image with a weighted sum of its adjacent pixels to extract image features;

[0017] The dilated convolution operation, in the image input to the operation, expands the original convolution kernel of the N-dimensional feature field to sample pixels in the 2N-1-dimensional feature field, and replaces each pixel of the image with the weighted sum of its adjacent sampled pixels, so as to extract image features within a larger scale range while keeping the image size unchanged;

[0018] The maximum pooling operation is used to perform maximum pooling on the image input into the operation;

[0019] The average pooling operation is used to perform average pooling on the image input into the operation;

[0020] The skip connection operation is used to perform a skip connection operation on an image input into the operation;

[0021] The adding operation is used to add every two images with the same or different sizes;

[0022] The connection operation is used to connect every two images with the same or different sizes in the channel dimension;

[0023] The search space based on stacking a fixed number of convolution blocks is to chain the network architecture using the searched neural network topology graph convolution blocks, use the downsampling topology graph convolution blocks to perform downsampling operations on image representation at the three-point position of the network, and globally pool the image features at the end of the network, and then send them to the connection layer and the linear classifier for classification, wherein the architecture topology graph in the normal topology graph convolution block and the downsampling topology graph convolution block is the search space to be searched, each block contains multiple graph nodes, each graph node is connected by operations, and the topology graph structure and edge operation types need to be searched.

[0024] Furthermore, in step 2, the discrete architecture parameters of the neural network architecture search space are relaxed to obtain a supernet model that can perform differential architecture search. The specific method is as follows:

[0025] The discrete architecture parameter refers to the operation type represented by each edge in the topological graph convolution block, which is represented by an integer. The parameter is not differentiable and cannot be continuous. In the relaxation operation, the integer representing the operation type of each edge is expanded to a high-dimensional vector with the dimension of the number of operations. Each element of the vector represents the importance of the operation. When the image representation enters a certain edge, it will be weighted and summed with the normalized relaxed edge operation vector to obtain the output of the certain edge. The image representation of a node is the sum of the outputs of each edge with the node as the end. After the discrete architecture parameter is relaxed, the supernet model can calculate the loss function through the network output, and further update the relaxed architecture parameters in the network by gradient descent.

[0026] The output calculation formula of one side is:

[0027] ;

[0028] in, refers to the output of the edge from node i to node j. is the relaxed and normalized architecture parameter of the corresponding edge, is the input of the edge, so the image representation value of node j is calculated as:

[0029] ;

[0030] That is, the image representation value of node j is the sum of the edge outputs of nodes whose node numbers are less than j.

[0031] Furthermore, the step 3 specifically includes:

[0032] Step 3.1: Take the normalized architecture parameters on each operation edge as a whole distribution, calculate the average value of the distribution entropy of each edge architecture parameter in the normal topology convolution block and the downsampled topology convolution block, and add the average value of this distribution entropy as a regularization term to the architecture parameter In the loss function of

[0033] For a single edge, the architecture parameter distribution entropy is calculated as follows:

[0034] ;

[0035] in, represents the architecture parameter distribution of an edge, Indicates that the distribution expectations;

[0036] The regularization term is:

[0037] ;

[0038] in, is the number of edges in the topological graph convolution block;

[0039] Step 3.2: Use the zero-overhead indicator SNIP to control the regularization coefficients of the normal topology convolution block and the down-sampling topology convolution block;

[0040] The calculation method of the zero overhead index SNIP is as follows:

[0041] ;

[0042] in, are the parameters in the neural network, is the loss function;

[0043] In this step, the architecture parameters need to be calculated The SNIP metric relative to the validation set loss function is used to control the coefficient of the entropy regularization term.

[0044] Furthermore, the step 4 specifically includes:

[0045] Step 4.1: Update the architecture parameters using gradient descent on the validation set based on the industrial image dataset , and its update formula is:

[0046] ;

[0047] in, For architecture parameters The learning rate parameter for gradient updates, Compute the loss function for the hypernet on the validation set versus the architecture parameters The gradient of ; the loss function here is calculated using cross entropy, that is, the cross entropy of the image classification label vector in the validation set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged;

[0048] Step 4.2: Update the network weights using gradient descent on the training set based on the industrial image dataset , and its update formula is:

[0049] ;

[0050] in, is the network weight The learning rate parameter for gradient updates, Calculate the function of the network weights for the supernet on the training set The gradient of ; the calculation of the loss function here also uses cross entropy, that is, the cross entropy of the image classification label vector in the training set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged;

[0051] Step 4.3: Learn the supernet parameters by repeating the gradient updates of steps 4.1 and 4.2 in multiple rounds and batches, so that the architecture parameters and network weights The following two-layer optimization conditions are met:

[0052] ;

[0053] ;

[0054] in, Represents the network weight The loss obtained on the validation set after the update; In the training set, by updating the network weights Minimize the loss of the training set; the final learned architecture parameters It can represent the optimal architecture obtained by the differentiable search method.

[0055] Furthermore, the step 5 specifically includes:

[0056] Step 5.1: Select each edge parameter vector The operation subscript with the largest weight is used as the value of the one-dimensional discrete parameter of the current edge, and each topology graph node only retains the two edges with the largest parameter weight of the filtered operation among the operation edges with a number smaller than that of the node, to construct the discretized architecture topology structure;

[0057] Step 5.2: Use the discretized architecture topology to build the searched neural network architecture, and increase the number of channels and layers of the network architecture to expand the scale of the architecture. Finally, train the architecture on the entire industrial image dataset to obtain the retrained architecture model. Input the image to be detected and classified into the model to obtain the defect detection results of the industrial image.

[0058] The beneficial effect of adopting the above technical solution is that the industrial defect image detection method based on dynamic regularized entropy differentiable architecture search provided by the present invention can obtain a high-performance, high-real-time, low-overhead industrial defect detection neural network architecture for a given industrial defect image data set by constructing a neural network supernet for differentiable search and performing differentiable search. The present invention proposes a new regularization method, namely, an architecture parameter entropy regularization method dynamically controlled by the SNIP index, by regularizing and punishing the entropy of the architecture parameters of the differentiable search supernet, and calculating the SNIP index by the gradient of the architecture parameters relative to the loss function and its own weight, to dynamically control the intensity of the regularization penalty in the differentiable search process; thereby eliminating the problem of performance collapse in the differentiable search method, and accelerating the search speed and convergence performance, so that performance collapse does not occur in long-term search, and improving the performance of the industrial defect detection neural network architecture obtained by the search. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flowchart of an industrial defect image detection method based on dynamic regularized entropy differentiable architecture search provided by an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of an industrial defect image detection method based on a dynamic regularized entropy differentiable architecture search provided by an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a macro-level chain network in a search space provided by an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of a topological graph convolution block search space at a micro level in a search space provided by an embodiment of the present invention;

[0063] Figure 5 A schematic diagram of a relaxed architecture at a micro level in a search space provided by an embodiment of the present invention;

[0064] Figure 6 A schematic diagram of dynamically updating architecture parameters by gradient descent during differentiable search provided by an embodiment of the present invention;

[0065] Figure 7 A schematic diagram of a discretized architecture at a microscopic level in a search space provided by an embodiment of the present invention;

[0066] Figure 8 A schematic diagram of an actual network topology structure of a discretized architecture in a normal topology graph convolution block provided by an embodiment of the present invention;

[0067] Fig. 9 A schematic diagram of the actual network topology structure of the discretization architecture in the downsampling topology graph convolution block provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0069] like Figure 1 As shown, the method of this embodiment is described as follows.

[0070] Step 1: If Figure 2 As shown, industrial image data to be detected and classified is obtained and constructed, and an industrial image dataset is constructed. The industrial image dataset is divided into a training set and a validation set according to a preset ratio for subsequent architecture search.

[0071] In this embodiment, industrial image data is obtained from a public data set and an industrial image data set is constructed, and the constructed industrial image data set is divided into a training set and a validation set according to a 1:1 ratio, which are used to update the network weights respectively. and architecture parameters , and normalize the image data according to the data expectation and variance of the image dataset, and appropriately process the input image through image enhancement techniques such as random image flipping and cropping.

[0072] Step 2: Construct the required neural network architecture search space, further obtain the supernet model for differentiable architecture search, and initialize the network weights therein and architecture parameters , and a loss model and optimizer for calculating the loss function and gradient update. In this embodiment, since the differentiable search requires a predefined search space and the existence of a corresponding supernet model, it is first necessary to define a chain neural network search space based on the topological graph convolution block, and to search the topological structure and required operations within each topological graph convolution block, and then construct a supernet used for differentiable search by relaxing the discretized architecture parameters, and initialize the network weights therein. and architecture parameters , as well as the loss model and optimizer for calculating the loss function and gradient update. The specific method is:

[0073] Step 2.1: Figure 2As shown in Figure 1, before starting the search, we first need to define the desired neural network architecture search space.

[0074] The neural network architecture search space, wherein the optional operations to be searched include depth separation convolution operation, void convolution operation, maximum pooling operation, average pooling operation, and skip connection operation, and the fixed operations include addition operation and connection operation. The optional operations to be searched and the fixed operations are combined into a neural network topology graph convolution block, wherein the operations represented by the topology graph structure and the operation edges are available for search, and a search space based on stacking a fixed number of convolution blocks is constructed.

[0075] The deep separation convolution operation first uses a single-channel high-dimensional filter to filter the image input to the operation, and then uses a multi-channel one-dimensional vector to perform an image transformation operation on the filtered image, replacing each pixel of the image with the weighted sum of its adjacent pixels to extract image features. The deep separation convolution operation of 3×3 convolution window and 5×5 convolution window is used in the search space.

[0076] The dilated convolution operation, in the image input to the operation, expands the original convolution kernel of the N-dimensional feature field to sample the pixels in the 2N-1-dimensional feature field, and replaces each pixel of the image with the weighted sum of its adjacent sampled pixels, so as to extract image features in a larger scale range while keeping the image size unchanged. The dilated convolution operation of 3×3 convolution window and 5×5 convolution window is used in the search space.

[0077] The maximum pooling operation is used to perform maximum pooling on the image input to the operation, using a 3×3 maximum pooling window to loop through the covered image features. In each window coverage, the pixel with the largest value in the covered image representation is selected as the numerical result of the maximum pooling, so that important features in the image can be retained.

[0078] The average pooling operation is used to perform average pooling on the image input to the operation, and uses a 3×3 average pooling window to cyclically traverse the covered image features. In each window coverage, the average value of all pixel representations in the covered image representation is selected as the numerical result of the average pooling, which can alleviate and eliminate abnormal noise in the image features.

[0079] The skip connection operation is used to perform a skip connection operation on the image input to the operation, that is, to perform identity mapping on the input image representation of the operation to the output of the operation.

[0080] The addition operation is used to add every two images with the same or different sizes. For images of different sizes, zero padding or cropping operations need to be performed on the different images first to make the images have the same size before adding them.

[0081] The connection operation is used to connect every two images with the same or different sizes in the channel dimension.

[0082] The specific set of operations available for search in the search space is as follows.

[0083] Table 1 The set of operations available for search in the search space

[0084] operate size Abbreviation describe Depthwise Separable Convolution Two vectors sep_conv_3x3 Use a 3×3 window to filter and transform the input image. Depthwise Separable Convolution Two images sep_conv_5x5 Use a 5×5 window to filter and then transform the input image Dilated Convolution Three images dil_conv_3x3 Use a 5×5 window and sample 9 pixels within the window. Dilated Convolution Four images dil_conv_5x5 Use a 9×9 window and sample 25 pixels within the window. Max Pooling An image max_pool_3x3 Use a 3×3 window to perform maximum pooling on the image Average Pooling An image avg_pool_3x3 Use a 3×3 window to average pool the image Skip Connections An image skip_connect Output of the identity mapping of the input image No Operation An image none Outputs a zero tensor of the same size as the input image representation

[0085] The search space is as follows at the macro level: Figure 3 As shown in the figure, the search space here is chain-connected based on the topological map convolution blocks. The input image is first subjected to preliminary feature processing by the convolution block at the front end of the network, and then passes through the subsequent topological map convolution blocks in sequence. The network topology map in each topological map convolution block is the search space to be searched; the topological map convolution blocks in the search space are divided into normal topological map convolution blocks and down-sampled topological map convolution blocks. The network is mainly composed of normal topological map convolution blocks connected in series, and down-sampled topological map convolution blocks are inserted at the three points connected to the normal topological map convolution blocks to perform down-sampling operations on image features. Here, the normal topological map convolution blocks and the down-sampled topological map convolution blocks respectively share a set of topological map results obtained by the search; finally, the processed image features are processed into one-dimensional vectors through global pooling and sent to the linear classifier for classification.

[0086] The search space is as follows at the micro level: Figure 4 As shown in the figure, the numbers 0, 1, 2, and 3 represent the node numbers in the topology graph. The operation topology graph contained in the topology graph convolution block is the architecture to be searched. Each node represents an intermediate image feature, and the edge represents an operation. The image feature of the source node is processed and passed to the destination node. If there are multiple operation edges pointing to the same node, the image feature of the node is the sum of the outputs of each operation edge. In a topology graph convolution block, the input comes from the first two topology graph convolution blocks, and the output is the result of connecting the image representations of each node in the channel dimension. Before the next topology graph convolution block receives the output of the previous topology graph convolution block, it is necessary to reduce the image dimension by a 1×1 dimensional convolution operation on the connected image features. Two sets of topology graph convolution structures and corresponding edge selected operations need to be searched in this search space, which are applied to normal topology graph convolution blocks and downsampling topology graph convolution blocks respectively. The specific real block convolution network topology structure is shown in the figure. Figure 8 and Fig. 9 As shown, they represent the actual network topology structures in the normal topology graph convolution block and the down-sampled topology graph convolution block respectively.

[0087] Step 2.2: Relax the discretized architecture parameters of the neural network architecture search space, such as Figure 2As shown, a supernet model for differentiable architecture search is constructed to start subsequent searches.

[0088] The discretization architecture is as follows Figure 7 As shown, the relaxed architecture is Figure 5 As shown. In the discretized architecture representation, the operation on each edge is represented by a single integer value representing the index of the operation, which is non-differentiable and cannot be continuous. In the relaxed architecture, each edge is parameterized as a set of operation vectors with the size of the operation set as the dimension size, where the weight of each vector element represents the importance of the corresponding operation. The architecture parameter vector parameters are differentiable and can be dynamically updated by gradient descent during differentiable search, as shown in Figure 6 shown.

[0089] In the relaxed differential search supernet model, the output calculation formula for each edge is:

[0090] ;

[0091] in, refers to the output of the edge from node i to node j. is the architecture parameter of the corresponding edge after relaxation and softmax normalization, is the input from node i to node j, so the image representation value of node j is calculated as:

[0092] ;

[0093] That is, the image representation value of node j is the sum of the operation edge outputs of the nodes whose node numbers are less than j.

[0094] Step 3: In the architecture parameters In the loss calculation of , the entropy of the distribution is used as an additional regularization term and the architecture parameters are calculated The zero-overhead index SNIP is normalized inside the search block and used as the dynamic control coefficient of the regularization term.

[0095] Step 3.1: Take the normalized architecture parameters on each operation edge as a whole distribution, calculate the average value of the distribution entropy of the architecture parameters on each operation edge in the normal topology convolution block and the downsampled topology convolution block, and add the average value of the distribution entropy as a regular term to the architecture parameter in the loss function.

[0096] For a single edge, the architecture parameter distribution entropy is calculated as follows:

[0097] ;

[0098] in, represents the architecture parameter distribution of an edge, Refers to the distribution expectations.

[0099] In the regularization term, it is necessary to calculate the average value of the architecture parameter distribution entropy in the normal topology convolution block and the downsampled topology convolution block respectively, and calculate the weighted sum of the average values ​​of the architecture parameter distribution entropy in the two convolution blocks, that is:

[0100] ;

[0101] ;

[0102] ;

[0103] in, is the number of edges in the topological graph convolution block; and are the average values ​​of the architecture parameter distribution entropy in the normal topology graph convolution block and the downsampled topology graph convolution block, respectively. and The regularization coefficients of the weighted sum of the average values ​​of the architecture parameter distribution entropy in the normal topology convolution block and the downsampled topology convolution block are respectively calculated. This is the final regularization term.

[0104] Step 3.2: Use the zero-overhead indicator SNIP to control the regularization coefficients of the normal topology graph convolution block and the downsampled topology graph convolution block and .

[0105] The calculation method of the zero overhead index SNIP is as follows:

[0106] ;

[0107] in, are the parameters in the neural network, is the loss function; in this step, the architecture parameters need to be calculated The SNIP index relative to the validation set loss function is used to control the coefficient of the entropy regularization term, that is, Replaced by the loss size on the validation set , Replace with schema parameters .

[0108] The calculation methods of using SNIP to control the regularization coefficient in the normal topology convolution block and the down-sampling topology convolution block are as follows:

[0109] ;

[0110] ;

[0111] And the regularization coefficient will be updated every time the loss is calculated and the back propagation is performed, so as to achieve the effect of dynamically controlling the regularization coefficient.

[0112] Step 4: Perform multiple rounds of training, using the training set and validation set to calculate the loss in each batch of each round and update them separately , .

[0113] Step 4.1: Based on industrial image datasets, such as Figure 2 As shown in Figure 2, if the maximum number of search iterations is not reached, the loss and regularization terms are calculated on the validation set to update the architecture parameters using gradient descent. , and its update formula is:

[0114] ;

[0115] in, For architecture parameters The learning rate parameter for gradient update, and the Adam optimizer is used. is the hyperparameter of the regularization term, Compute the loss function for the hypernet on the validation set versus the architecture parameters The gradient of is the gradient of the regularization term to the architecture parameters calculated in step 3. The loss function here is calculated using cross entropy, that is, the cross entropy of the image classification label vector in the validation set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged.

[0116] Step 4.2: Based on industrial image datasets, such as Figure 2 As shown, if the maximum number of search iterations is not reached, then update After that, the network weights need to be updated using gradient descent on the training set , and its update formula is:

[0117] ;

[0118] in, is the network weight The learning rate parameter for gradient update and the SGD optimizer are used. Calculate the function of the network weights for the supernet on the training set The calculation of the loss function here also uses cross entropy, that is, the cross entropy between the image classification label vector in the training set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged.

[0119] Step 4.3: Learn the supernet parameters by repeating the gradient updates of steps 4.1 and 4.2 in multiple rounds and batches, so that the architecture parameters and network weights The following two-layer optimization conditions are met:

[0120] ;

[0121] ;

[0122] in, Represents the network weight The loss obtained on the validation set after the update; In the training set, by updating the network weights Minimize the loss of the training set. The final learned architecture parameters It can represent the optimal architecture obtained by the differentiable search method.

[0123] Step 5: Figure 2 As shown in the figure, if the maximum iteration round is reached after all rounds of training, the architecture parameters need to be discretized to obtain the neural network architecture finally searched, and the architecture is retrained on the entire data set and used as a classification model for detecting abnormal defects in industrial images. The image to be detected and classified is input into the model to obtain the defect detection results of the industrial image.

[0124] Step 5.1: Select each operation edge parameter vector The operation subscript with the largest weight is used as the value of the one-dimensional discrete parameter of the current edge, and each topology graph node only retains the two edges with the largest parameter weight of the filtered operation among the edges with a number smaller than that of the node, to construct the discretized architecture topology structure;

[0125] Step 5.2: Use the discretized architecture topology to build the searched neural network architecture, increase the number of channels and layers of the network architecture to expand the scale of the architecture, and add cosine learning rate scheduling in the architecture optimization learning to control the model weight optimization to the optimal point; finally, train the architecture on the entire industrial image dataset to obtain the retrained architecture model, input the image to be detected and classified into the model, and obtain the defect detection results of the industrial image.

[0126] In order to further verify the effectiveness of the method of this embodiment, three classified image datasets of different sizes and one industrial defect detection classification dataset are used: CIFAR10, CIFAR100, ImageNet16-120, and MVTec AD to perform industrial defect image classification simulation tests. The test results are shown in Table 2. The traditional method used in the test is an artificially designed neural network without using a neural architecture search algorithm. According to the test results, it can be seen that compared with the existing methods, the method of the present invention has stronger performance, the model has smaller storage and reasoning overhead, and only takes 2 to 3 GPU hours to search for the neural network architecture. The searched architecture can more accurately predict the category of industrial defect images.

[0127] Table 2 Industrial defect image classification simulation test results data table

[0128] Model CIFAR10 CIFAR100 ImageNet16-120 MVTec AD Parameter quantity Traditional methods Accuracy = 96.54% Accuracy = 77.01% Accuracy = 75.3% AUROC=98.10% More than 10 million Method of the present invention Accuracy = 97.52% Accuracy = 84.00% Accuracy = 83.4% AUROC = 98.77% 3.5 million

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. An industrial defect image detection method based on dynamic regularized entropy differentiable architecture search, characterized by: The steps include: Step 1: Obtain the industrial image data to be detected and classified and construct an industrial image dataset, and divide the industrial image dataset into a training set and a validation set according to a preset ratio; Step 2: Construct the required neural network architecture search space, further obtain the supernet model for differentiable architecture search, and initialize the network weights therein and architecture parameters , and a loss model and optimizer for calculating the loss function and gradient update; the specific method for constructing the required neural network architecture search space is: The neural network architecture search space, wherein the optional operations to be searched include depth separation convolution operation, void convolution operation, maximum pooling operation, average pooling operation, and jump connection operation, and the fixed operations include addition operation and connection operation; the optional operations to be searched and the fixed operations are composed of a neural network topology graph convolution block, wherein the operations represented by the topology graph structure and the operation edges are available for search, and a search space based on stacking a fixed number of convolution blocks is constructed; the search space based on stacking a fixed number of convolution blocks is to chain the network architecture using the searched neural network topology graph convolution blocks, use the downsampling topology graph convolution blocks at the three-point position of the network to perform downsampling operations for image representation, and globally pool the image features at the end of the network, and then send them to the connection layer and the linear classifier for classification, wherein the architecture topology graphs in the normal topology graph convolution block and the downsampling topology graph convolution block are the search spaces to be searched, each block contains multiple graph nodes, each graph node is connected by an operation, and the operation types of the topology graph structure and the edges need to be searched; The deep separation convolution operation first uses a single-channel high-dimensional filter to filter the image input to the operation, and then uses a multi-channel one-dimensional vector to perform an image transformation operation on the filtered image, replacing each pixel of the image with a weighted sum of its adjacent pixels to extract image features; The dilated convolution operation, in the image input to the operation, expands the original convolution kernel of the N-dimensional feature field to sample pixels in the 2N-1-dimensional feature field, and replaces each pixel of the image with the weighted sum of its adjacent sampled pixels, so as to extract image features within a larger scale range while keeping the image size unchanged; The maximum pooling operation is used to perform maximum pooling on the image input into the operation; The average pooling operation is used to perform average pooling on the image input into the operation; The skip connection operation is used to perform a skip connection operation on an image input into the operation; The adding operation is used to add every two images with the same or different sizes; The connection operation is used to connect every two images with the same or different sizes in the channel dimension; The discretized architecture parameters of the neural network architecture search space are relaxed to obtain a supernet model that can perform differential architecture search. The specific method is as follows: The discretized architecture parameter refers to the type of operation represented by each edge in the topological graph convolution block, which is represented by an integer number. The parameter is not differentiable and cannot be continuous. In the relaxation operation, the integer number representing the operation type of each edge is expanded into a high-dimensional vector with the dimension of the number of operations. Each element of the vector represents the importance of the operation. When the image representation enters a certain edge, it will be weighted and summed with the normalized relaxed edge operation vector to obtain the output of the certain edge; the image representation of a node is the sum of the outputs of each edge with the node as the end; when the discretized architecture parameters are relaxed, the supernet model can calculate the loss function through the network output, and further update the relaxed architecture parameters in the network by gradient descent; The output calculation formula of one side is: ; in, refers to the output of the edge from node i to node j. is the relaxed and normalized architecture parameter of the corresponding edge, is the input of the edge, so the image representation value of node j is calculated as: ; That is, the image representation value of node j is the sum of the edge outputs of nodes whose node numbers are less than j; Step 3: In the architecture parameters In the loss calculation of The entropy of the distribution is used as an additional regularization term and the architecture parameters are calculated The zero-overhead index SNIP is normalized inside the search block as the dynamic control coefficient of the regularization term; specifically, it includes: Step 3.1: Take the normalized architecture parameters on each operation edge as a whole distribution, calculate the average value of the distribution entropy of each edge architecture parameter in the normal topology convolution block and the downsampled topology convolution block, and add the average value of this distribution entropy as a regularization term to the architecture parameter In the loss function of For a single edge, the architecture parameter distribution entropy is calculated as follows: ; in, represents the architecture parameter distribution of an edge, Indicates that the distribution expectations; The regularization term is: ; in, is the number of edges in the topological graph convolution block; Step 3.2: Use the zero-overhead indicator SNIP to control the regularization coefficients of the normal topology convolution block and the down-sampling topology convolution block; The calculation method of the zero overhead index SNIP is as follows: ; in, are the parameters in the neural network, is the loss function; In this step, the architecture parameters need to be calculated The SNIP index relative to the validation set loss function is used to control the coefficient of the entropy regularization term; Step 4: Perform multiple rounds of training, using the training set and validation set to calculate the loss in each batch of each round and update them separately , ; Step 5: After all rounds of training, set the architecture parameters The neural network architecture is discretized and finally searched. The architecture is retrained on the entire data set and used as a classification model for detecting abnormal defects in industrial images. The images to be classified are input into the model to obtain the defect detection results of the industrial images.

2. The industrial defect image detection method based on dynamic regularized entropy differentiable architecture search according to claim 1 is characterized in that: The step 4 specifically includes: Step 4.1: Update the architecture parameters using gradient descent on the validation set based on the industrial image dataset , and its update formula is: ; in, For architecture parameters The learning rate parameter for gradient updates, Compute the loss function for the hypernet on the validation set versus the architecture parameters The gradient of ; the loss function here is calculated using cross entropy, that is, the cross entropy of the image classification label vector in the validation set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged; Step 4.2: Update the network weights using gradient descent on the training set based on the industrial image dataset , and its update formula is: ; in, is the network weight The learning rate parameter for gradient updates, Calculate the function of the network weights for the supernet on the training set The gradient of ; the calculation of the loss function here also uses cross entropy, that is, the cross entropy of the image classification label vector in the training set and the classification output vector of the supernet is calculated, and the calculation results of different samples in the same batch are averaged; Step 4.3: Learn the supernet parameters by repeating the gradient updates of steps 4.1 and 4.2 in multiple rounds and batches, so that the architecture parameters and network weights The following two-layer optimization conditions are met: ; ; in, Represents the network weight The loss obtained on the validation set after the update; In the training set, by updating the network weights Minimize the loss of the training set; the final learned architecture parameters It can represent the optimal architecture obtained by the differentiable search method.

3. The industrial defect image detection method based on dynamic regularized entropy differentiable architecture search according to claim 2 is characterized by: The step 5 specifically includes: Step 5.1: Select each edge architecture parameter The operation subscript with the largest weight is used as the value of the one-dimensional discrete parameter of the current edge, and each topology graph node only retains the two edges with the largest parameter weight of the filtered operation among the operation edges with a number smaller than that of the node, to construct the discretized architecture topology structure; Step 5.2: Use the discretized architecture topology to build the searched neural network architecture, and increase the number of channels and layers of the network architecture to expand the scale of the architecture. Finally, train the architecture on the entire industrial image dataset to obtain the retrained architecture model. Input the image to be detected and classified into the model to obtain the defect detection results of the industrial image.

Images