An industrial visual inspection model optimization method
By using network architecture search and Markov chain Monte Carlo methods to prune industrial vision inspection models, the adaptability and efficiency issues of existing models in industrial vision inspection are solved, achieving efficient and accurate model optimization to meet the diverse needs of industrial vision inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHUHAI BOJAY ELECTRONICS
- Filing Date
- 2022-10-28
- Publication Date
- 2026-07-21
AI Technical Summary
Neural network models that perform well in existing computer vision competitions are difficult to adapt to industrial vision inspection, especially in multi-class object detection. The metrics do not match the needs of industry, the models are too large, and the pruning process is time-consuming, which cannot meet the high accuracy and high efficiency requirements of industrial vision inspection.
By designing small and efficient models through network architecture search, and using the Markov chain Monte Carlo method to partition and prune the target detection network model, combined with threshold optimization of overkill and underkill rates, a suitable network framework is formed, enabling rapid model pruning and optimization.
It achieves efficient and accurate model optimization in industrial visual inspection, meeting the needs of high-speed equipment operation and diverse defect detection, shortening model trimming time, and improving inspection efficiency.
Smart Images
Figure CN115936097B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a model optimization method, and more particularly to an industrial vision inspection model optimization method, belonging to the field of industrial vision inspection. Background Technology
[0002] In large deep learning networks, the activation of most neurons tends to zero. These neurons with zero activation are redundant, and removing them can significantly reduce the model size and computational cost without affecting its performance. Therefore, when pruning a model, it is necessary to select unimportant parameters and remove them without significantly impacting the model's performance.
[0003] The standard model pruning process involves three steps: 1) Training a large model with redundant parameters to obtain the best network performance, using this as a benchmark; 2) Pruning the large network model based on certain criteria; and 3) Fine-tuning the pruned network model on the dataset.
[0004] Neural Architecture Search (NAS) uses algorithms to search for the neural network structure needed to solve a problem. Typically, basic blocks describing the neural network architecture are first defined, and then a configuration string is used to express the architecture.
[0005] Classic neural network architecture search methods include:
[0006] (1) Neural network architecture search based on evolutionary algorithm: Randomly initialize some neural network architectures, and evolve the network structure by survival of the fittest. Child nodes are randomly selected through some predetermined network structure change operations, such as adding or deleting a layer, adjusting the hyperparameters of a layer, adding skip connections, etc., to generate new network structures. The network structure is continuously trained and mutated to achieve better performance.
[0007] (2) Reinforcement Learning (RL) based neural network architecture search allows an agent to discover the optimal configuration of the network architecture. Based on the agent's performance on the test set, a certain network is used as the controller to sample and generate strings describing the network structure. The evaluation accuracy will be used as the reward value for the reinforcement learning neural network architecture search.
[0008] (3) Neural network architecture search based on Bayesian Optimization (BO) algorithm, which is based on Gaussian process and focuses on low-dimensional continuous optimization problems. The posterior probability distribution of some points is calculated by Gaussian process regression (assuming that the hyperparameters follow a joint Gaussian distribution), and the expected mean and variance of each hyperparameter at each value point are obtained. The mean represents the final expected effect at this point. The larger the mean, the larger the final index of the model. The variance represents the uncertainty of the effect at this point.
[0009] (4) The gradient-based method for neural network architecture search treats the network structure as a directed acyclic graph with ordered nodes, where each edge has a learnable real value representing the probability of each block being used.
[0010] This method transforms the search space into a continuous space and the objective function into a differentiable function. The architecture is optimized using gradient descent and based on performance on the test set.
[0011] The models designed using the above methods, which perform exceptionally well in various computer vision competitions for tasks such as object detection, semantic segmentation, instance segmentation, and video tracking, are not well-suited for industrial vision inspection. Firstly, in multi-class object detection, the performance metric is mAP (measurand area under the curve), and each class can be plotted using recall and precision. AP is the area under this curve, and mAP is the average of the AP for each class. While mAP is currently the most commonly used metric for evaluating the performance of detection models, it does not have a completely linear relationship with metrics such as false negatives and false positives that are of concern in industry. Secondly, some excellent models, in pursuit of high mAP, often come with a large number of network parameters. Randomly cropping and testing such large network structures requires a long training time for validation. In industrial vision inspection algorithm design, a product typically requires images from multiple channels to achieve comprehensive defect detection. Different channels require customized defect detection models; for example, detecting multiple categories of defects requires a large model, while detecting a single type of defect only requires a small model. Therefore, applying models that perform well in computer vision competitions to industrial visual inspection requires further improvement based on specific circumstances to meet the demands of high accuracy and high efficiency in the operation of inspection equipment. Summary of the Invention
[0012] This invention provides an industrial vision inspection model optimization method that uses network architecture search to design small and precise models that meet the actual operation and inspection needs of equipment.
[0013] This invention provides a method for optimizing an industrial visual inspection model, comprising:
[0014] S1. Obtain the object detection network model and divide the network structure into n blocks;
[0015] S2. Sample and analyze the n blocks after division until the i-th block meets the set criteria; process and evaluate the i-th block and the remaining blocks separately; if the threshold requirement is not met, repeat S2; if the threshold requirement is met, output the model.
[0016] Further, S1. Obtaining the target detection network model and dividing the network structure into n blocks includes:
[0017] S101. Obtain the target detection network model;
[0018] S102. Based on the network structure of the object detection network model, divide it into n blocks, ensuring that each block has a width scaling factor α. i and depth scaling factor β i .
[0019] Furthermore, S1. Obtaining the target detection network model and dividing the network structure into n blocks also includes:
[0020] S103. Update the number of convolutional kernels w for each block using formulas (1) and (2). i The number h of identical network structure blocks that perform convolution operations consecutively i ;
[0021]
[0022]
[0023] Where w i This refers to the number of convolution kernels in the i-th block. h refers to the number of convolutional kernels in the i-th block after the update. i This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block. This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block after the update.
[0024] Furthermore, in step S2, sampling analysis is performed on the n divided blocks until the i-th block meets the set criteria; this block and the remaining blocks are processed and evaluated separately; if the threshold requirement is not met, step S2 is repeated; if the threshold requirement is met, the model output includes:
[0025] S201. Extract from the n blocks sequentially;
[0026] S202. For the i-th block, α i ,β i Random sampling, the block's The number of convolutional kernels and Initialize the weights of the same convolutional structure, set the model as the new model χ1, and train until convergence;
[0027] S203. Evaluate the convergence result of χ1, and perform Markov chain analysis on the evaluated χ1.
[0028] S204. If the convergence result of χ1 at the (n+1)th step is better than the convergence result at the nth step, then α i ,β i It will be saved into the Markov chain at step n+1;
[0029] S205. Determine whether the number of iterations of the χ0 model is greater than the threshold L. If it is not, return the χ0 model to S201. If the threshold L is reached, stop the loop.
[0030] S206. Take the α of each block. i ,β i The median of this stable probability distribution is used to calculate the value of each block layer. The number of convolutional kernels and The same network structure blocks that perform convolution operations continuously are used to fully initialize the χ0 model and train it until convergence.
[0031] Furthermore, S201. Extracting the n Blocks sequentially specifically involves:
[0032] Extract from model χ0 layer by layer up to the i-th block.
[0033] Furthermore, in S202, α for the i-th Block i ,β i Random sampling, the block's The number of convolutional kernels and Initialize the weights of the same convolutional structure, set the model as the new model χ1, and train until convergence, including:
[0034] S2021. Using the Monte Carlo method, for the i-th block, α i ,β i Random sampling is performed within a certain range. When an α is sampled... i and β iThen, the i-th Block layer is calculated according to formulas (1) and (2). The number of convolutional kernels and The same convolutional structure;
[0035] S2022. For the i-th Block layer, according to... The number of convolutional kernels and Initialize the weights of the same convolutional structure, keep the weights of other block layers unchanged, denoted as the new model χ1, and train until convergence.
[0036] Furthermore, step S203. evaluating the convergence result of χ1 and performing Markov chain analysis on the evaluated χ1 includes:
[0037] S2031. Evaluate the convergence result of χ1 on the validation dataset according to the evaluation criteria;
[0038] S2032. Lock the overkill rate and optimize the overkill rate to be less than or equal to the overkill rate threshold δ;
[0039] S2033. While ensuring that the overkill rate is less than or equal to δ, optimize the false negative rate to the target threshold ε.
[0040] Furthermore, in S204, if the convergence result of the (n+1)th step of χ1 is better than the convergence result of the nth step, then α i ,β i The Markov chain that will be saved to the (n+1)th step includes:
[0041] S2041. Perform Markov chain analysis. If the convergence result of the (n+1)th step is better than the convergence result of the nth step, then set α... i ,β i It will be saved into the Markov chain at step n+1;
[0042] S2042. If the convergence result of the (n+1)th step is worse than that of the nth step, then the acceptance-rejection method is used for sampling; if sampling is accepted, then α is used. i ,β i Update the i-th Blocks by adjusting the i-th Block layer according to... The number of convolutional kernels and Initialize the weights of the same network structure blocks that continuously perform convolution operations, keep the weights of other block layers unchanged, train until convergence, and update the model χ0.
[0043] Furthermore, it also includes S2043. If the sampling result is rejected, the Markov chain reverts to the nth step.
[0044] Furthermore, the acceptance rate of the acceptance-rejection method is 0.25.
[0045] Compared with existing technologies, this invention uses a fast random pruning design based on a small detection network structure, combined with the Markov chain Monte Carlo method (MCMC), to effectively search for suitable network frameworks and optimize industrial vision inspection models. Attached Figure Description
[0046] Figure 1 This is a schematic diagram illustrating the principle of an embodiment of the present invention;
[0047] Figure 2 This is a flowchart illustrating the operation of an embodiment of the present invention. Detailed Implementation
[0048] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0049] Under strict over-detection and under-detection criteria, classifying defects in products with diverse variations in shape, size, texture, and color is a challenging task. A good industrial vision inspection model needs to not only complete the inspection task but also meet the requirements of high-speed equipment operation.
[0050] Based on the above requirements, this invention discloses an industrial vision inspection model optimization method, including:
[0051] S1. Obtain the object detection network model and divide the network structure into n blocks;
[0052] S2. Sample and analyze the n blocks after division until the i-th block meets the set criteria; process and evaluate the i-th block and the remaining blocks separately; if the threshold requirement is not met, repeat S2; if the threshold requirement is met, output the model.
[0053] In this process, the network structure is divided into multiple basic functional blocks. S2 is used to analyze each block one by one, so as to process the unimportant functional blocks in the current model and achieve model pruning.
[0054] This invention's embodiments utilize excellent small-scale detection network structures (such as...) Figure 1(As shown) Begin with a fast random pruning design. The Markov Chain Monte Carlo (MCMC) method is excellent for searching for suitable frameworks for networks. MCMC is a method that approximates the posterior distribution of parameters of interest by randomly sampling in the probability space. Its core idea is to find a Markov chain in a state space such that the stable distribution of that Markov chain is the target distribution. MCMC is a method that approximates the posterior distribution of parameters of interest by randomly sampling in the probability space. Its core idea is to find a Markov chain in a state space such that the stable distribution of that Markov chain is the target distribution.
[0055] Optionally, S1. Obtaining the target detection network model and dividing the network structure into n blocks includes:
[0056] S101. Obtain the target detection network model;
[0057] S102. Based on the network structure of the object detection network model, divide it into n blocks, ensuring that each block has a width scaling factor α. i and depth scaling factor β i .
[0058] Specifically, S1. Obtaining the object detection network model and dividing the network structure into n blocks further includes:
[0059] S103. Update the number of convolutional kernels w for each block using formulas (1) and (2). i The number h of identical network structure blocks that perform convolution operations consecutively i ;
[0060]
[0061]
[0062] Where w i This refers to the number of convolution kernels in the i-th block. h refers to the number of convolutional kernels in the i-th block after the update. i This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block. This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block after the update.
[0063] The model is divided into multiple functional blocks, and these blocks are used to extract features such as α. i β i w i , Specific block information, etc.
[0064] Specifically, S2. Samples and analyzes the n divided blocks up to the i-th block that meets the set criteria; processes and evaluates this block and the remaining blocks separately; if the threshold requirement is not met, S2 is repeated; if the threshold requirement is met, the model output includes:
[0065] S201. Extract from the n blocks sequentially;
[0066] S202. For the i-th block, α i ,β i Random sampling, the block's The number of convolutional kernels and Initialize the weights of the same convolutional structure, set the model as the new model χ1, and train until convergence;
[0067] S203. Evaluate the convergence result of χ1, and perform Markov chain analysis on the evaluated χ1.
[0068] S204. If the convergence result of χ1 at the (n+1)th step is better than the convergence result at the nth step, then α i ,β i It will be saved into the Markov chain at step n+1;
[0069] S205. Determine whether the number of iterations of the χ0 model is greater than the threshold L. If it is not, return the χ0 model to S201. If the threshold L is reached, stop the loop.
[0070] S206. Take the α of each block. i ,β i The median of this stable probability distribution is used to calculate the value of each block layer. The number of convolutional kernels and The same network structure blocks that perform convolution operations continuously are used to fully initialize the χ0 model and train it until convergence.
[0071] After completion, the model χ0 and weights are output to obtain the target model.
[0072] This invention is based on the Markov chain Monte Carlo method, which uses Markov chains to effectively analyze and process blocks, thereby achieving effective model pruning.
[0073] Specifically, S201. Extracting the n blocks sequentially involves:
[0074] Extract from model χ0 layer by layer up to the i-th block.
[0075] Specifically, S202. for the i-th Block α i ,β i Random sampling, the block's The number of convolutional kernels and Initialize the weights of the same convolutional structure, set the model as the new model χ1, and train until convergence, including:
[0076] S2021. Using the Monte Carlo method, for the i-th block, α i ,β i Random sampling is performed within a certain range. When an α is sampled... i and β i Then, the i-th Block layer is calculated according to formulas (1) and (2). The number of convolutional kernels and The same convolutional structure;
[0077] S2022. For the i-th Block layer, according to... The number of convolutional kernels and Initialize the weights of the same convolutional structure, keep the weights of other block layers unchanged, denoted as model χ1, and train until convergence.
[0078] In this process, the weights of other blocks are frozen and remain unchanged. Only the weights of the i-th block layer are initialized. The i-th block layer is merged with other blocks and denoted as model χ1. The model is then trained until convergence.
[0079] Specifically, S203. Evaluating the convergence result of χ1 and performing Markov chain analysis on the evaluated χ1 includes:
[0080] S2031. Evaluate the convergence result of χ1 on the validation dataset according to the evaluation criteria;
[0081] S2032. Lock the overkill rate and optimize the overkill rate to be less than or equal to the overkill rate threshold δ;
[0082] S2033. While ensuring that the overkill rate is less than or equal to δ, optimize the false negative rate to the target threshold ε.
[0083] In this embodiment of the invention, the overkill rate threshold δ is 5%, and the target threshold ε is 0%.
[0084] In this embodiment of the invention, the target thresholds of overkill rate and false negative rate are used to effectively evaluate the new model χ1, and the evaluation results are processed separately in the Markov chain.
[0085] Specifically, in S204, if the convergence result of the (n+1)th step of χ1 is better than the convergence result of the nth step, then αi ,β i The Markov chain that will be saved to the (n+1)th step includes:
[0086] S2041. Perform Markov chain analysis. If the convergence result of the (n+1)th step is better than the convergence result of the nth step, then set α... i ,β i It will be saved into the Markov chain at step n+1, using α i ,β i Initialize the weights of the i-th block of model χ0, keeping the weights of other blocks unchanged, train the i-th block layer until convergence, and update model χ0;
[0087] S2042. If the convergence result of the (n+1)th step is worse than that of the nth step, then the acceptance-rejection method is used for sampling; if sampling is accepted, then α is used. i ,β i Update the i-th Blocks by adjusting the i-th Block layer according to... The number of convolutional kernels and Initialize the weights of the same network structure blocks that continuously perform convolution operations, keep the weights of other block layers unchanged, train until convergence, and update the model χ0.
[0088] Among them, the i-th block layer is arranged according to The number of convolutional kernels and The same network structure blocks that continuously perform convolution operations are initialized with weights, while the weights of other Block layers remain unchanged. The model is trained until convergence, and the trained χ0 model is updated to form a new model χ0.
[0089] Specifically, this also includes S2043. If the sampling result is rejected, the Markov chain reverts to the nth step.
[0090] In this process, we revert to the nth step and repeat S201 for analysis.
[0091] Specifically, the acceptance rate of the acceptance-rejection method is 0.25.
[0092] Although S2041 is effectively analyzed, there is still a certain probability that solutions that do not meet the requirements of S2041 will be accepted by the Markov link. In this embodiment of the invention, the acceptance-rejection method is used for sampling to effectively analyze the blocks whose convergence results in the (n+1)th step are worse than those in the nth step, which can avoid getting trapped in local optima.
[0093] This invention takes capacitor visual detection as an example, requiring the target visual model to achieve an over-detection rate of less than 5% and a false negative rate of 0%. To achieve this goal, the existing initial model χ0 is first evaluated on the automotive capacitor verification dataset, that is, the over-detection rate and false negative rate of the current initial model are evaluated on the dataset.
[0094] The following steps will now be performed:
[0095] Step 1: Based on the industrial vision detection model optimization method of this invention, the initial model χ0 is iteratively optimized to form a more suitable model structure, so that the trained model achieves zero missed detections on the automotive capacitor verification dataset.
[0096] Step 2: Set the false negative rate to 0%, and use the industrial vision detection model optimization method of this embodiment of the invention again to iteratively process the model obtained in Step 1, and select the model with the lowest false negative rate;
[0097] Step 3: Iterate the method of the present invention on the model selected in Step 2 until the overkill rate is below 5% and the false negative rate is 0% in the automotive capacitor verification dataset. Then output the structure and weights of the model as the target model.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading this application specification, they can still modify or make equivalent substitutions to the specific implementation of the present invention, but these modifications or changes do not depart from the protection scope of the pending claims of the present invention.
Claims
1. A method for optimizing an industrial visual inspection model, characterized in that, The method is used to optimize an industrial visual defect detection model, including: S1. Obtain the object detection network model, which is used for industrial vision inspection tasks; input the shape, size, texture, and color of a product image from multiple channels; based on the network structure of the object detection network model, divide the network structure into n blocks, each block having a width scaling factor. and depth scaling factor ; S2. Sample and analyze the n blocks after division until the i-th block meets the set criteria; process the i-th block and the remaining blocks separately, and evaluate and analyze them on the validation dataset; if the evaluation result does not meet the industrial detection threshold requirements, repeat S2; if the evaluation result meets the industrial detection threshold requirements, output the optimized model for industrial visual inspection. S2 includes: S201. Extract from the n blocks sequentially; S202. For the i-th block , Random sampling, the block's The number of convolutional kernels and Initialize the weights of the same convolutional structure and set it as the new model. And train until convergence; S203. Regarding The convergence results are evaluated, and the evaluated results are used to evaluate the convergence results. Perform Markov chain analysis; S203 includes: S2031. Based on the evaluation criteria, on the validation dataset... The convergence results are evaluated; S2032. Lock the overkill rate and optimize it to be less than or equal to the overkill rate threshold. ; S2033. Ensure that the overkill rate is less than or equal to In this case, optimize the false negative rate to the target threshold. .
2. The industrial vision inspection model optimization method according to claim 1, characterized in that, S1 includes: S101. Obtain the target detection network model; S102. Based on the network structure of the object detection network model, divide it into n blocks, ensuring that each block has a width scaling factor. and depth scaling factor .
3. The industrial vision inspection model optimization method according to claim 2, characterized in that, S1 further includes: S103. Update the number of convolutional kernels for each block using the following formula. The number of identical network structure blocks that perform convolution operations consecutively. ; (1)； (2)； in This refers to the number of convolution kernels in the i-th block. This refers to the number of convolutional kernels in the i-th block after the update. This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block. This refers to the number of identical network structure blocks that undergo consecutive convolution operations within the i-th block after the update.
4. The industrial vision inspection model optimization method according to claim 2, characterized in that, S2 includes: S204. If If the convergence result of the (n+1)th step is better than the convergence result of the nth step, then... , It will be saved into the Markov chain at step n+1; S205. Judgment Does the number of iterations of the model exceed the threshold L? If not, then reset the model. Return to S201; if the threshold L is reached, the loop stops. S206. Retrieve the value of each block. , The median of this stable probability distribution is used to calculate the value of each block layer. The number of convolutional kernels and A series of identical network structure blocks undergoing convolution operations are fully initialized. The model is trained until convergence.
5. The industrial vision inspection model optimization method according to claim 4, characterized in that, Specifically, S201 is as follows: From the model Extract layer by layer, up to the i-th block.
6. The industrial vision inspection model optimization method according to claim 4, characterized in that, S202 includes: S2021. Using the Monte Carlo method, for the i-th block... , Random sampling is performed within a certain range; when a sample is found... and Then, the i-th Block layer is calculated according to formulas (1) and (2). The number of convolutional kernels and The same convolutional structure; S2022. For the i-th Block layer, according to... The number of convolutional kernels and Initialize the weights of the same convolutional structure, leaving the weights of other block layers unchanged; this is denoted as the new model. And train it until it converges.
7. The industrial vision inspection model optimization method according to claim 4, characterized in that, S204 includes: S2041. Perform Markov chain analysis. If the convergence result of the (n+1)th step is better than the convergence result of the nth step, then... , It will be saved into the Markov chain at step n+1; S2042. If the convergence result of the (n+1)th step is worse than the convergence result of the nth step, then the acceptance-rejection method is used for sampling; if sampling is accepted, then the method is used... , Update the i-th Blocks by adjusting the i-th Block layer according to... The number of convolutional kernels and Initialize the weights of the same network structure blocks that perform consecutive convolution operations, keeping the weights of other block layers unchanged, and train until convergence, then update the model. .
8. The industrial vision inspection model optimization method according to claim 7, characterized in that, It also includes S2043. If the sampling result is rejected, the Markov chain reverts to the nth step.
9. The industrial vision inspection model optimization method according to claim 8, characterized in that, The acceptance rate of the acceptance-rejection method is 0.25.