A defect prediction method based on model pruning

Abstract

The application discloses a defect prediction method based on model pruning, generates corresponding three-layer heat map scale labels according to a defect scale adaptive method; constructs a plurality of equal-interval dilated convolution kernels; assembles the equal-interval dilated convolution kernels into an equal-interval dilated convolution network in series and parallel relationship; splices the network after a standard ResNet50 network, outputs a prediction target of the highest layer, trains a highest layer heat map label H3; prunes the network composed of the equal-interval dilated convolution, and performs loss gradient descent training on three scales and three-layer heat map labels respectively predicted and output by corresponding feature layers of the ResNet50 network for a specified number of rounds. After corresponding post-processing of results of each layer, the results of each layer are integrated, and the predicted results are restored to an original image. The application is formed by adding in series and splicing in parallel, is fully combined, expands a receptive field, and cross-acquires global and local information.

Description

Technical Field

[0001] This invention relates to the field of industrial product defect prediction, and specifically to a defect prediction method based on model pruning. Background Technology

[0002] Modern manufacturing has gradually entered a stage of automation. Under normal circumstances, factories can produce smoothly without human intervention. However, due to the existence of defective products, manual inspection and regular maintenance are still unavoidable. During production, it is necessary to monitor the product's manufacturing process and control its surface quality. For example, in the field of inner pot inspection for small household appliances such as pressure cookers, air fryers, and rice cookers, users pay more attention to the product's appearance quality than its functional quality. If such products have surface defects such as deformation, dirt, or scratches, their aesthetic appeal and value will be greatly reduced.

[0003] In real-world, complex product manufacturing environments, surface defect detection often faces numerous challenges. These include small differences between defect images and the background, low contrast, large variations in defect scale, and diverse defect types. In such defect detection tasks, handling multi-scale targets is particularly important. Addressing multi-scale issues often involves combining multi-level features to ensure that the features possess joint locational details and high-level semantic information, enabling targeted prediction of targets with different receptive fields at multiple levels. However, this multi-level combination approach often introduces redundant information and creates a grid effect. Multi-feature fusion, using progressive upsampling followed by addition, further increases computational cost and generally cannot meet the real-time requirements of production inspection. Summary of the Invention

[0004] To address the aforementioned problems, this invention aims to provide a defect prediction method based on model pruning.

[0005] To achieve this technical objective, the present invention provides a defect prediction method based on model pruning, the specific steps of which are as follows:

[0006] S1. Obtain the defect image and generate corresponding three-layer heatmap scale labels, H1, H2, and H3, based on the defect scale adaptive method.

[0007] S2. Construct several dilated convolution kernels with equal spacing;

[0008] S3: Construct a dilated convolutional network with equal gaps in a series and parallel relationship using the dilated convolutional kernels proposed in step S2;

[0009] S4: After splicing the equal-gap dilated convolutional network from step S3 onto the standard ResNet50 network, output the predicted target of the highest layer and train the highest layer heatmap label H3 mentioned in step S1.

[0010] S5: After training for a specified number of rounds in step S4, prune the distillation model composed of equal-gap dilated convolutions. Then, train the model for a specified number of rounds using loss gradient descent based on the three scales and three-layer heatmap labels predicted by the corresponding feature layers of the ResNet50 network.

[0011] S6: The distillation model training is complete. Input the image to be tested, and after passing through the ResNet network, output the results of the three feature layers respectively. After performing the corresponding post-processing on the results of each layer, integrate the results of each layer and restore the predicted result to the original image.

[0012] Preferably, the heat map described in step S1 is generated based on the Gaussian kernel distribution, and different Gaussian kernel heat maps are generated by three different target sizes.

[0013] The specific Gaussian kernel heatmap is generated based on the width and height of the defect target and the size of its center point, as shown below:

[0014] And distributed to heatmap labels according to category, (x, y) represents the size information of the target, (p x p y () represents the size information of the center point p of the actual target. It is the adaptive standard deviation of the target.

[0015] Preferably, in step S2, three dilation convolution kernels with prime dilation coefficients and equal spacing are selected, and the dilation coefficient combination is one or more of (3, 5, 7), (3, 7, 11), or (1, 7, 13).

[0016] As a preferred option, in step S2, a convolution kernel with dilation coefficients of (3,5,7) is used, and this block is called EsPBlock-1. The receptive field shows a trend of a large central region and a gradually decreasing area towards the outer regions.

[0017] When using convolution kernels with dilation coefficients of (3, 7, 11) respectively, the receptive field is a small block average on the global scale. This block is called EsPBlock-2. Then, these two receptive fields are concatenated and superimposed into a block. It is found that the receptive field as a whole exhibits a local effect in global information extraction.

[0018] When using convolutional kernels with dilation coefficients of (1, 7, 13) respectively, the receptive field is presented as a whole after local information is extracted from each region, and this block is called EsPBlock-3.

[0019] Finally, EsPBlock-1 and EsPBlock-2 are connected in series, and then this whole system is connected in parallel with EsPBlock-3 to form a multi-receptive field information extraction network.

[0020] Preferably, in step S4, after the input defect image is processed by ResNet50, the result is a heatmap of (num_classes,32,32). This heatmap is then processed by the equal-gap dilated convolutional network in step S3, and the final output is still a heatmap of (num_classes,32,32). This heatmap is then processed by a multi-receptive-field information extraction network.

[0021] The heatmap of (num_classes,32,32) and H3 labels are trained using loss gradient descent.

[0022] Preferably, in step S5, multi-scale learning is performed by combining parts of ResNet50, so that each of the three layers in the middle of ResNet50 is trained for the defect targets of three different scales. Specifically, the P1 feature layer is responsible for the delivery training of H1 label, the P2 feature layer is responsible for the delivery training of H2 label, and the P3 feature layer is responsible for the delivery training of H3 label.

[0023] As a preferred option, in step S6, the output result is restored to the original image size, and then the results are stitched together to display the result in the original image.

[0024] The beneficial effects of this invention are as follows: This application employs equal-interval prime-number dilated convolution groups in cascaded and serial structures to combine receptive fields of different sizes in a serial manner and in a parallel manner to form a comprehensive combination; this expands the receptive field while creating a cross-acquisition of global and local information; through the pruning technique in model distillation, redundant branches are removed, which have already given the model some adaptability, and are discarded to improve the training and prediction speed of the model; and during distillation, the labels are softened, allowing the model to fit training samples that better match the true distribution, while this distilled model is as small as possible, thus improving both the accuracy and speed of the model. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention;

[0026] Figure 2 This is a flowchart illustrating the overall operation of the present invention;

[0027] Figure 3 This is a diagram illustrating the effect of the receptive field created by stacking equally spaced dilated convolution kernels according to the present invention.

[0028] Figure 4 This is a diagram illustrating the process of outputting three-layer feature maps using the ResNet of this invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1-4 As shown, the specific embodiment of the present invention is a defect prediction method based on model pruning.

[0031] Example 1

[0032] S101. Generate corresponding three-layer heatmap scale labels according to the defect scale adaptive method, characterized in that the heatmap is generated according to the Gaussian kernel distribution, and different Gaussian kernel heatmaps are generated for three different size targets.

[0033] The Gaussian kernel function is:

[0034] x and x' are the input data, and σ is the degree of dispersion of the data;

[0035] The specific Gaussian kernel heatmap is generated based on the width and height of the defect target and the size of its center point, as shown below:

[0036]

[0037] And distributed to heatmap labels according to category, (x, y) represents the size information of the target, (p x p y ) represents the size information of the center point p of the actual target. R is the adaptive standard deviation of the target. R is the size ratio of the input map to the heatmap. The dimensions of H1, H2, and H3 are (num_classes, 128, 128), (num_classes, 64, 64), and (num_classes, 32, 32), respectively, where num_classes identifies the class name of the defect target.

[0038] For defects at multiple scales in an image, H1, H2, or H3 heatmap labels are generated based on their respective height and width ranges. In other words, an image will generate three heatmap labels, with each label representing the Gaussian kernel generation result for a target at a different scale.

[0039] For heatmap labels, which of H1, H2, or H3 the target is on determines the heatmap generation level, which is determined by the target's scale. Specifically:

[0040]

[0041] x represents the scale size relative to the original image. For example, if the length and width of a target are 114 and 115 respectively, and the original image sizes are 1200 and 900 respectively, then x is 114*115 / (1200*900) = 0.11 < 0.2. Therefore, a heatmap of the target is generated on H1.

[0042] S102. Construct multiple equally spaced dilated convolution kernels, whose dilation coefficients increase according to a certain pattern. For example, a convolutional group composed of kernels with dilation coefficients of (3,5,7) has a receptive field as shown in Figure 102. Figure 3 As shown, the receptive field is the size of the region on the input image that maps a pixel in the feature map output by each layer of the convolutional neural network back to the input image. Different color depths represent the number of times that pixel is mapped in the image. Then, convolutional groups of (3,7,11) and (1,7,13) are constructed respectively.

[0043] S103. The dilated convolutional kernels proposed in step S102 are assembled into an equal-gap dilated convolutional network with a series and parallel relationship. In this invention, convolutional kernels with dilation coefficients of (3,5,7) and (3,7,11) are serially superimposed to form a small-block average receptive field globally. This block, formed by the serial superposition of these two convolutional kernel groups, is called EsPBlock-1. Then, a convolutional kernel group with coefficients of (1,7,13) is constructed. This receptive field, after extracting local information from each region, is presented as a whole globally; this block is called EsPBlock-2. Finally, EsPBlock-1 and EsPBlock-2 are connected in parallel to form a multi-receptive-field information extraction network, called an equal-gap dilated convolutional network.

[0044] S104. After splicing the network constructed in step S103 onto the standard ResNet50 network, the overall network will output the predicted target of the highest layer, and train the highest layer heat map label H3 mentioned in step S101.

[0045] S105. After training the network in step S104 for a certain number of rounds, prune the network composed of equal-gap dilated convolutions. The three feature layers corresponding to the ResNet50 network predict the output results P1, P2, and P3 at three scales respectively. Then, train the network with the three heatmap labels H1, H2, and H3 using loss gradient descent for a certain number of rounds.

[0046] S106. After inputting the image to be tested, it passes through the convolutional layers of the ResNet network, generating prediction results from three feature layers, such as... Figure 4As shown, the results of each layer are post-processed accordingly, including non-maximum suppression, score filtering, and scale restoration, etc., and the results of each layer are integrated to restore the predicted results to the original image. Specifically, if the output dimension of the highest layer feature map is (num_classes, 32, 32), this result is widened by 4 times in height and width dimensions to become (num_classes, 128, 128), and then multiplied by the height and width of the original image by 128 to obtain the corresponding position of the target in the original image.

[0047] The present invention is described by flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor-generating machine of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus so that the processor of the computer or other computer executes the instructions. The programmable data processing apparatus can generate a device for use that implements the functions specified in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0048] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any minor modifications, equivalent substitutions, and improvements made to the above embodiments based on the technical essence of the present invention should be included within the protection scope of the present invention.

Claims

1. A defect prediction method based on model pruning, characterized in that, The specific steps are as follows: S1. Obtain defect images and generate corresponding three-layer heatmap scale labels, H1, H2, and H3, based on the defect scale adaptive method. The heatmaps are generated according to Gaussian kernel distribution, and different Gaussian kernel heatmaps are generated for three different target sizes. The specific Gaussian kernel heatmap is generated based on the width and height of the defect target and the size of its center point, as shown below: ; And distributed to heatmap labels according to category, (x, y) represents the size information of the target, (p x p y () represents the size information of the center point p of the actual target. It is the adaptive standard deviation of the target; S2. Construct several dilated convolution kernels with equal spacing; select three dilated convolution kernels with prime dilation coefficients and equal spacing, and the combination of dilation coefficients is one or more of (3, 5, 7), (3, 7, 11), or (1, 7, 13); S3: Construct a dilated convolutional network with equal gaps in a series and parallel relationship using the dilated convolutional kernels proposed in step S2; S4: After splicing the equal-gap dilated convolutional network from step S3 onto the standard ResNet50 network, output the predicted target of the highest layer and train the highest layer heatmap label H3 mentioned in step S1. After the input defect image is processed by ResNet50, the result is a heatmap of (num_classes,32,32). This heatmap is then processed by the equal-gap dilated convolutional network in step S3, and the final output is still a heatmap of (num_classes,32,32). This heatmap is then processed by a multi-receptive field information extraction network. The heatmap and H3 labels of (num_classes,32,32) are trained using loss gradient descent. S5: After training for a specified number of rounds in step S4, prune the equal-gap dilated convolutional network composed of equal-gap dilated convolutions. Train the loss gradient descent for a specified number of rounds using the corresponding feature layers of the ResNet50 network to predict the output of three scales and three layers of heatmap labels, and obtain the distillation model. Combine the ResNet50 part to perform multi-scale learning, so that each of the three layers in the middle of ResNet50 is trained for the defect targets of three scales respectively. Among them, the P1 feature layer is responsible for the delivery training of H1 label, the P2 feature layer is responsible for the delivery training of H2 label, and the P3 feature layer is responsible for the delivery training of H3 label. S6: The distillation model training is complete. Input the image to be tested, and after passing through the ResNet network, output the results of the three feature layers respectively. After performing the corresponding post-processing on the results of each layer, integrate the results of each layer and restore the predicted result to the original image.

2. The defect prediction method based on model pruning according to claim 1, characterized in that: In step S2, convolution kernels with dilation coefficients of (3,5,7) are used, and this block is called EsPBlock-1. The receptive field shows a trend of a large central region and a gradually decreasing area towards the outer regions. When using convolution kernels with dilation coefficients of (3, 7, 11) respectively, the receptive field is a small block average on the global scale. This block is called EsPBlock-2. Then, these two receptive fields are concatenated and superimposed into a block. It is found that the receptive field as a whole exhibits a local effect in global information extraction. When using convolutional kernels with dilation coefficients of (1, 7, 13), the receptive field is presented as a whole after local information is extracted in each region. This block is called EsPBlock-3. Finally, EsPBlock-1 and EsPBlock-2 are connected in series, and then this whole system is connected in parallel with EsPBlock-3 to form a multi-receptive field information extraction network.

3. The defect prediction method based on model pruning according to claim 1, characterized in that: In step S6, the output results are restored to the original image size, and then the results are stitched together to display the results in the original image.

Images