Lightweight small target defect detection method for precision electronic product

Abstract

The invention discloses a lightweight small target defect detection method for a precision electronic product, and the method comprises the steps: constructing a defect detection model, and carrying out the feature extraction and feature fusion of small target defects through a multi-order fusion backbone network and a dual-upsampling feature fusion network; taking the trained defect detection model as a teacher model, selecting a lightweight YOLOv8n-p2 model as a student model, selecting a feature layer which needs to realize knowledge migration, enabling the student model to learn features of the feature layer in the teacher model, simulating the teacher model to generate a feature map, and performing knowledge distillation to obtain a lightweight defect detection model; and performing defect detection on the to-be-detected image of the precision electronic product by using the lightweight defect detection model to obtain a defect detection result. According to the method, the hierarchical features of the small target defects are enriched, the detection precision is improved, model lightweight processing is carried out through knowledge distillation, and the small target defect detection effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and more specifically, to a lightweight small target defect detection method for precision electronic products. Background Art

[0002] In the manufacturing process of some precision industrial products, especially in complex manufacturing processes, each process link performs appearance defect detection at the loading stage to ensure product qualification rate and reduce subsequent production waste. For some smaller defects, it is difficult to observe with the naked eye because the target area is too small. The widely used detection methods in the industry are mainly based on automatic optical detection technologies such as traditional edge detection and template matching. However, this method is highly dependent on predefined templates, and it is difficult to effectively identify defect types outside the template or small variation defects; at the same time, external factors such as fluctuations in lighting conditions and uneven image quality may cause the matching degree between the template and the image to be inspected to decrease, thereby affecting the accuracy of the detection results; there are certain shortcomings in detection flexibility, detection accuracy, detection speed, adaptability and real-time performance.

[0003] In view of the above shortcomings, the introduction of deep learning-based target detection into the field of defect detection overcomes the limitations of traditional visual inspection methods, further optimizes detection accuracy and improves detection efficiency. Generally, deep learning detection models use convolutional networks for downsampling to extract target object features, but for small target objects, after convolution, their feature maps are very small, the feature representation ability is insufficient, and the layers are not rich enough, resulting in low detection accuracy. Most models for detecting small targets have too many parameters in terms of calculation and cannot be deployed on mobile or embedded devices, or they run too slowly, affecting detection efficiency, so the models need to be lightweight. Therefore, how to improve the feature representation ability of industrial small target detection models and solve the problem of model lightweighting is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a lightweight small target defect detection method for precision electronic products. By constructing a defect detection model, feature extraction and feature fusion are performed on small target objects respectively, thereby enriching the hierarchical features of small target defects and improving the expression ability of small target object features. The model further improves the detection accuracy in detecting small target defects, and lightweight processing of the model improves the speed of deploying the small target defect detection model on a mobile terminal or embedded device.

[0005] The present invention provides a lightweight small target defect detection method for precision electronic products, comprising the following steps:

[0006] Acquire a small target defect data set of precision electronic products, build a defect detection model and use the small target defect data set for training, the defect detection model includes a multi-order feature extraction network and a double up-sampling feature fusion network;

[0007] The multi-order feature extraction network is used to extract multi-scale features of the small target defect image and enhance the features to generate multiple enhanced feature maps, and the double upsampling feature fusion network is used to perform cross-scale feature fusion on the multiple enhanced feature maps to generate a feature fusion map;

[0008] Perform target detection on each feature fusion graph to obtain defect detection results, verify and test the defect detection results after iterative training, and output the trained defect detection model when the model performance meets the standards;

[0009] The trained defect detection model is used as the teacher model, and the lightweight model YOLOv8n-p2 is selected as the student model. The feature layer of the teacher model is transferred to the student model by using knowledge distillation. After distillation, a lightweight defect detection model is obtained.

[0010] Collect the image data to be inspected of precision electronic products, use a lightweight defect detection model to perform target detection on the image data to be inspected, and output the defect detection results.

[0011] In this solution, in the defect detection model, the multi-stage feature extraction network is composed of several 3×3 convolutional layers and a multi-order gated feature module C2f-MogaBlock, wherein the multi-order gated feature module C2f-MogaBlock is formed by replacing Bottlenet in C2f with a multi-order gated aggregation module MogaBlock;

[0012] The double upsampling feature fusion network is composed of a plurality of double upsampling splicing modules and a multi-order gated feature module C2f-MogaBlock module, wherein the double upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module in parallel.

[0013] In this scheme, the small target defect data set is imported into the defect detection model, and a multi-order feature extraction network is used to obtain the multi-scale features of the small target defect image and perform feature enhancement to generate multiple enhanced feature maps, specifically:

[0014] The multi-order gated aggregation module MogaBlock in the multi-order gated feature module C2f-MogaBlock is composed of a spatial aggregation block and a channel aggregation block, and the spatial aggregation block includes a feature decomposition module and a multi-order gated aggregation module;

[0015] In the feature decomposition module of the spatial aggregation block, the feature map first passes through a normalization layer, and after the normalization process, a 1x1 convolution layer is used to obtain local features, and then a global average pooling layer is used to obtain global features, and the global features are subtracted from the local features and multiplied by a scale factor, and summed with the local features, and finally a GELU activation function is used to obtain a multi-order feature map to complete feature decomposition;

[0016] In the multi-order gated aggregation module of the spatial aggregation block, the multi-order feature map is divided into two branches, one branch passes through a 1x1 convolution layer and then passes through a SiLU activation function to obtain an aggregation branch; the other branch passes through a 5x5 depth convolution and then passes through a split layer to be divided into three channels, one channel is not processed, and the other two channels respectively pass through a 5x5 depth convolution with an expansion rate of 2 and a 7x7 depth convolution with an expansion rate of 3 to obtain three depth convolution feature maps, the three depth convolution feature maps are spliced, and after splicing, they pass through a 1x1 convolution, and then pass through a SiLU activation function to obtain a context branch, the aggregation branch is multiplied by the context branch, and passes through a 1x1 convolution layer, and is added to the original input to obtain a multi-order gated feature map;

[0017] In the channel aggregation block, the multi-order gated feature map is sequentially subjected to a normalization layer, a 1x1 convolution layer, a 3x3 depth convolution layer, and a GELU activation function to obtain an optimized channel aggregation feature map. The optimized channel aggregation feature map is subjected to a 1x1 convolution layer and then calculated by a GELU activation function, and then subtracted from the original input optimized channel aggregation feature map. After the subtraction, the map is multiplied by the scale factor and then added to the original input optimized channel aggregation feature map to obtain a channel aggregation feature map. The channel aggregation feature map is added to the multi-order gated feature map to obtain an enhanced feature map.

[0018] In this solution, the calculation formula of the feature map after the multi-stage gated aggregation module Mogablock is expressed as:

[0019] Y=X+SA(Norm(X))

[0020] Z=Y+CA(Norm(Y))

[0021] Among them, X is the multi-order feature map obtained by feature decomposition, SA is the spatial aggregation block, Norm(·) is the normalization process, Y is the optimized channel aggregation feature map, CA is the channel aggregation block, and Z is the enhanced feature map.

[0022] In this scheme, after using a multi-order feature extraction network to obtain multiple enhanced feature maps, the double upsampling feature fusion network is used to perform cross-scale feature fusion on the multiple enhanced feature maps to generate a feature fusion map, specifically:

[0023] A double upsampling splicing module and a multi-order gated feature module in a double upsampling feature fusion network are used to perform cross-scale feature fusion on multiple enhanced feature maps, wherein the double upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module through a parallel structure;

[0024] The multiple enhanced feature maps are divided into two branches and respectively pass through the neighboring interpolation upsampling module and the feature recombination upsampling module to obtain upsampled feature maps of two interpolation modes. The upsampled feature maps are spliced ​​through a Concat layer to obtain a spliced ​​feature map. The spliced ​​feature map is processed by a multi-order gated feature module to obtain a feature fusion map.

[0025] In this scheme, defect detection is performed on each fusion feature map, and the defect detection results are output. After each round of cyclic training of small target defect image samples, the detection and classification effect of the defect detection model is verified using verification data. If the detection and classification effect meets the preset classification effect standard, the current network parameters are retained, otherwise the network parameters are adjusted;

[0026] The generalization ability of the network is tested using the reserved test data. When the detection and classification performance of the defect detection model meets the preset classification performance standard, the network parameters at this time are used as the structural parameters of the defect detection model, and the trained defect detection model is output.

[0027] In this solution, the trained defect detection model is used as the teacher model, the lightweight model YOLOv8n-p2 is selected as the student model, and the feature layer of the teacher model is transferred to the student model by using knowledge distillation. Specifically:

[0028] The trained defect detection model is used as the teacher model, the lightweight model YOLOv8n-p2 is selected as the student model, and the specified feature layer is selected from the teacher model as the knowledge transfer to the specified layer of the student model;

[0029] Input the small target defect image sample into the student model, calculate the specified layer feature map through forward propagation and save it, and use the 1×1 convolution layer to align the features so that the feature map of the student model and the teacher model have the same number of channels;

[0030] The feature maps of the teacher model and the student model are input into the MSE loss function to calculate the loss, and finally the sum is obtained to obtain the distillation loss. Backpropagation is performed through the distillation loss to calculate the gradient, and the parameters of all layers of the student model are updated according to the calculated gradient until the model converges. After distillation is completed, a lightweight defect detection model is obtained.

[0031] In this scheme, the distillation loss function is expressed as:

[0032]

[0033] Where N, C, H, and W represent batch size, number of channels, feature map height, and feature map width, respectively. Represents the eigenvalue of the student model at the cth channel, height h, width w in the i-th layer. It represents the feature value of the teacher model at the cth channel, height h, width w in the i-th layer, and N represents the total number of feature layers.

[0034] In this solution, visual inspection equipment is used to collect image data to be inspected of precision electronic products, and the image data to be inspected is preprocessed;

[0035] The preprocessed image data to be detected is input into a lightweight defect detection model to obtain multiple enhanced feature maps of the image data to be detected, a feature fusion map is generated by feature fusion, the feature fusion map is detected by a detection head, and the defect detection result of the image to be detected is output.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention uses a multi-order feature extraction network and a double upsampling feature fusion network to perform feature extraction, enhancement and multi-scale splicing and fusion on the image to be detected, and performs target detection on the obtained fused feature map, thereby enriching the hierarchical features of small target defects and improving the expressiveness of small target defect features, so that the model can further improve the detection accuracy in detecting small target defects; through knowledge distillation, the knowledge of the small target defect detection model is transferred to a lightweight model, which improves the speed of deploying the small target defect detection model on a mobile terminal or embedded device and improves the effect of small target defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0039] Figure 1 A flow chart showing a lightweight small target defect detection method for precision electronic products;

[0040] Figure 2 The flowchart of the multi-order feature extraction network and the double up-sampling feature fusion network in the defect detection model is shown;

[0041] Figure 3 A schematic diagram of the process flow of the spatial aggregation block in the multi-stage gated aggregation module is shown;

[0042] Figure 4 A schematic diagram of the process flow of the channel aggregation block in the multi-stage gated aggregation module is shown;

[0043] Figure 5 A schematic diagram of the process of the double upsampling splicing module in the double upsampling feature fusion network is shown. DETAILED DESCRIPTION

[0044] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0046] like Figure 1 As shown, an embodiment of the present invention provides a lightweight small target defect detection method for precision electronic products, comprising:

[0047] S102, obtaining a small target defect data set of precision electronic products, building a defect detection model and using the small target defect data set for training, wherein the defect detection model includes a multi-order feature extraction network and a double up-sampling feature fusion network;

[0048] S104, using the multi-order feature extraction network to perform multi-scale feature extraction and feature enhancement on the small target defect image to generate a plurality of enhanced feature maps, and using the double upsampling feature fusion network to perform cross-scale feature fusion on the plurality of enhanced feature maps to generate a feature fusion map;

[0049] S106, performing target detection on each feature fusion graph to obtain defect detection results, verifying and testing the defect detection results after iterative training, and outputting the trained defect detection model when the model performance meets the standard;

[0050] S108, using the trained defect detection model as the teacher model, selecting the lightweight model YOLOv8n-p2 as the student model, and using knowledge distillation to migrate the feature layer of the teacher model to the student model, and obtaining the lightweight defect detection model after distillation;

[0051] S110, collecting image data to be inspected of precision electronic products, using a lightweight defect detection model to perform target detection on the image data to be inspected, and outputting defect detection results.

[0052] It should be noted that, in the defect detection model, the multi-stage feature extraction network is composed of several 3×3 convolutional layers and a multi-order gated feature module C2f-MogaBlock, wherein the multi-order gated feature module C2f-MogaBlock is formed by replacing Bottlenet in C2f with a multi-order gated aggregation module MogaBlock; the double upsampling feature fusion network is composed of several double upsampling splicing modules and multi-order gated feature modules C2f-MogaBlock modules, wherein the double upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module in parallel, which upsamples and splices the input image respectively, and outputs a spliced ​​feature map; a multi-order feature extraction network is used to extract multi-scale features of the image to be detected and perform feature enhancement to generate multiple enhanced feature maps; a double upsampling feature fusion network is used to perform cross-scale feature fusion on multiple feature maps to generate a feature fusion map; target detection is performed on each fused feature map, and the target detection result is output.

[0053] like Figure 2As shown in the figure, the specific process of the defect detection model is as follows: a 3×3 convolutional layer is used to perform a 3×3 convolutional feature extraction operation on the image to be detected, and a first convolutional feature map is output. A multi-order gated aggregation calculation is performed on the first convolutional feature map through a multi-order gated feature module, and a first enhanced feature map is output. After the first enhanced feature map is subjected to a 3×3 convolutional layer convolution operation, it is output to the multi-order gated feature module for multi-order gated aggregation calculation to generate a second enhanced feature map. After the second enhanced feature map is subjected to a 3×3 convolutional layer convolution operation, it is output to the multi-order gated feature module for multi-order gated aggregation calculation to generate a third enhanced feature map. After the third enhanced feature map is subjected to a 3×3 convolutional layer convolution operation, it is output to the multi-order gated feature module for multi-order gated aggregation calculation to generate a fourth enhanced feature map. The fourth enhanced feature map is subjected to a pooling layer to obtain a multi-scale feature map. After the multi-scale feature map is subjected to a double upsampling splicing module, a first up-sampled feature map is output. The first up-sampled feature map is spliced ​​with the third enhanced feature map to generate a first spliced ​​feature map. The first spliced ​​feature map is output to the multi-order gated feature module to obtain the first enhanced spliced ​​feature map. The first enhanced spliced ​​feature map is output to the second up-sampled feature map after passing through the double up-sampling splicing module. The second up-sampled feature map is spliced ​​with the second enhanced feature map to generate the second spliced ​​feature map. The second spliced ​​feature map is output to the multi-order gated feature module to obtain the second enhanced spliced ​​feature map. The second enhanced spliced ​​feature map is output to the double up-sampling splicing module to output the third up-sampled feature map. The third up-sampled feature map is spliced ​​with the first enhanced feature map to generate the third spliced ​​feature map. The third feature splicing map is output to the multi-order gated feature module to obtain the third enhanced splicing feature map. The third enhanced splicing feature map passes through a 3×3 convolution layer, is spliced ​​with the second enhanced splicing feature map, and is output to the multi-order gated feature module to obtain the first feature fusion map. The first feature fusion map passes through a 3×3 convolution layer, is spliced ​​with the first enhanced splicing feature map, and is output to the multi-order gated feature module to obtain the second feature fusion map. The second feature fusion map is concatenated with the multi-scale feature map through a 3×3 convolution layer and output to the multi-order gated feature module to obtain the third feature fusion map. The third enhanced concatenated feature map, the first feature fusion map, the second feature fusion map, and the third feature fusion map are respectively input to the detection head to obtain the detection result.

[0054] During the model training process, the small target defect data set is imported into the defect detection model, and a multi-order feature extraction network is used to obtain the multi-scale features of the small target defect image and perform feature enhancement to generate multiple enhanced feature maps. The multi-order gated aggregation module MogaBlock in the multi-order gated feature module C2f-MogaBlock is composed of a spatial aggregation block and a channel aggregation block, including a normalization layer, a 1x1 convolution layer, a 3x3 depth convolution layer, a GELU activation function layer, a SiLU activation function layer, and a GAP pooling layer. The spatial aggregation block includes a feature decomposition module and a multi-order gated aggregation module; in the feature decomposition module of the spatial aggregation block, the feature map first passes through the normalization layer, and after the normalization process, a 1x1 convolution layer is used to obtain local features, and then the global features are obtained through the global average pooling layer. The global features and the local features are subtracted and multiplied with the scale factor, and summed with the local features. Finally, the GELU activation function is passed to obtain a multi-order feature map to complete the feature decomposition. The calculation formula is expressed as:

[0055] Y j =Conv 1×1 (Norm(X))

[0056] FD=GELU(Y j +γ s ⊙(Y j -GAP(Y j )))

[0057] Among them, X is the input feature map, Y j represents local features, Norm(·) is normalization, FD represents feature decomposition, γ s is the scaling factor, GELU is the GELU activation function, and GAP is the global average pooling layer.

[0058] like Figure 3 As shown, in the multi-order gated aggregation module of the spatial aggregation block, the multi-order feature map is divided into two branches, one branch passes through the 1x1 convolution layer and then passes through the SiLU activation function to obtain an aggregation branch; the other branch passes through a 5x5 deep convolution and then passes through a split layer to be divided into three channels, one channel is not processed, and the other two channels are respectively subjected to a 5x5 deep convolution with an expansion rate of 2 and a 7x7 deep convolution with an expansion rate of 3 to obtain three deep convolution feature maps, which are respectively represented as The three deep convolution feature maps are spliced, and then passed through 1x1 convolution, and then through the SiLU activation function to obtain the context branch. The aggregation branch is multiplied by the context branch, and then passed through the 1x1 convolution layer and added to the original input to obtain a multi-order gated feature map; the SiLU function is used as a gating signal to effectively fuse the aggregation branch and the context branch to achieve gate control. The multi-order gated aggregation module captures features of different sizes through multi-order deep convolutions and aggregates multi-order features. The calculation formula is expressed as:

[0059]

[0060] Z r =SiLU(Conv 1×1 (X a ))⊙SiLU(Conv 1×1 (Y c ))

[0061] SA=Conv 1×1 (Z r )

[0062] Among them, input X a is the multi-order feature map obtained after feature decomposition, DWConv is the deep convolution, dilation and d are the parameter expansion rates in the deep convolution, Spilt is the split layer, S is the number of split layers, SiLU is the SiLU activation function, and three channels 1, 2, and 3 are split out. Y c is the channel concatenation feature map, SA is the spatial aggregation block, and Z r Fusion feature maps for the aggregation branch and the context branch.

[0063] like Figure 4 As shown, in the channel aggregation block, the multi-order gated feature map is sequentially passed through a normalization layer, a 1x1 convolution layer, a 3x3 depth convolution layer, and a GELU activation function to obtain an optimized channel aggregation feature map. The optimized channel aggregation feature map is passed through a 1x1 convolution layer and then calculated by a GELU activation function, and then subtracted from the original input optimized channel aggregation feature map. After subtraction, the map is multiplied by a scale factor and then added to the original input optimized channel aggregation feature map to obtain a channel aggregation feature map. The channel aggregation feature map is added to the multi-order gated feature map to obtain an enhanced feature map. The calculation is expressed as:

[0064] Y = GELU(DWConv 3×3 (Conv 1×1 (Norm(X a ))))

[0065] Z=Y+γ c ⊙(Y-GELU(Conv 1×1 (Y)))

[0066] CA=Conv 1×1 (Z)

[0067] Where X a is the multi-order feature map obtained by feature decomposition, CA is the channel aggregation block, γ c is the scaling factor, Y is the optimized channel aggregation feature map, CA is the channel aggregation block, and Z is the enhanced feature map;

[0068] The calculation formula of the feature map after the multi-stage gated aggregation module Mogablock is expressed as:

[0069] Y=X+SA(Norm(X))

[0070] Z=Y+CA(Norm(Y))

[0071] Among them, X is the multi-order feature map obtained by feature decomposition, SA is the spatial aggregation block, Norm(·) is the normalization process, Y is the optimized channel aggregation feature map, CA is the channel aggregation block, and Z is the enhanced feature map.

[0072] After using the multi-order feature extraction network to obtain multiple enhanced feature maps, the double upsampling feature fusion network is used to perform cross-scale feature fusion on the multiple enhanced feature maps to generate a feature fusion map. The double upsampling splicing module and the multi-order gated feature module in the double upsampling feature fusion network are used to perform cross-scale feature fusion on multiple enhanced feature maps, such as Figure 5 As shown, the dual upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module through a parallel structure. Multiple enhanced feature maps are divided into two branches and passed through the neighboring interpolation upsampling module and the feature reorganization upsampling module respectively to obtain upsampled feature maps of two interpolation modes. The upsampled feature maps are spliced ​​through the Concat layer to obtain a spliced ​​feature map. The calculation formula is expressed as:

[0073] Y p =Concat(Upsample(Z),CARAFE(Z))

[0074] Where Z is the enhanced feature map, Y p It is a feature concatenation diagram, Upsample represents the neighboring interpolation upsampling module, CARAFE represents the feature recombination upsampling module, and Concat represents the concatenation module.

[0075] The spliced ​​feature map is processed using a multi-order gated feature module to obtain a feature fusion map.

[0076] It should be noted that defect detection is performed on each fused feature map, and the defect detection results are output. After each round of training of small target defect image samples, the verification data is used to verify the detection and classification effect of the defect detection model. If the detection and classification effect meets the preset classification effect standard, the current network parameters are retained, otherwise the network parameters are adjusted; the generalization ability of the network is tested using the reserved test data. When the detection and classification performance of the defect detection model meets the preset classification performance standard, the network parameters at this time are used as the structural parameters of the defect detection model, and the trained defect detection model is output.

[0077] It should be noted that the trained defect detection model is used as the teacher model, the lightweight model YOLOv8n-p2 is selected as the student model, and the specified feature layer is selected from the teacher model as the knowledge transfer to the specified layer of the student model. Preferably, the teacher model is selected to generate the feature layers of the third enhanced splicing feature map, the first feature fusion map, the second feature fusion map and the third feature fusion map, and the layer of the student model that accepts knowledge transfer is selected. The small target defect image sample is input into the student model, and the specified layer feature map is calculated and saved through forward propagation. The 1×1 convolution layer is used for feature alignment so that the feature map of the student model and the teacher model have the same number of channels; the feature maps of the teacher model and the student model are input into the MSE loss function to calculate the loss, and finally the sum is obtained to obtain the distillation loss, and the distillation loss is back-propagated to calculate the gradient, and the parameters of all layers of the student model (including the specified layer and other layers) are updated according to the calculated gradient until the model converges and the distillation is completed to obtain a lightweight defect detection model.

[0078] The distillation loss function is expressed as:

[0079]

[0080] Where N, C, H, and W represent batch size, number of channels, feature map height, and feature map width, respectively. Represents the eigenvalue of the student model at the cth channel, height h, width w in the i-th layer. It represents the feature value of the teacher model at the cth channel, height h, width w in the i-th layer, and N represents the total number of feature layers.

[0081] According to an embodiment of the present invention, visual inspection equipment is used to collect image data to be inspected of precision electronic products, and the image data to be inspected is preprocessed; the preprocessed image data to be inspected is input into a lightweight defect detection model to obtain multiple enhanced feature maps of the image data to be inspected, a feature fusion map is generated by feature fusion, the feature fusion map is inspected by a detection head, and the defect detection result of the image to be inspected is output. Preferably, the defect information of the image to be inspected is supplemented into the small target defect data set of precision electronic products, and the defect detection model is regularly updated. Construct a defect detection database for precision electronic products, store all historical defect detection results in the defect detection database, generate defect features based on defect category information, location information, and size information, attribute defects using production parameters corresponding to the defect information, and match the defect attribution information with the defect features; obtain the defect detection results of the current image to be detected, read the defect features in the defect detection results, use the defect features to perform similarity calculations in the defect detection database, filter defect information whose similarity meets a preset similarity standard, extract defect attribution information of the filtered defect information, generate operation and maintenance information of precision electronic product production line equipment based on the extracted defect attribution information, use the operation and maintenance information to remind operation and maintenance personnel to perform equipment operation and maintenance management in a timely manner, avoid large batches of defects, and further improve the yield rate of precision electronic products.

[0082] The lightweight defect detection model uses a multi-order feature extraction network and a double upsampling feature fusion network to extract features, enhance and multi-scale splicing and fusion on the images to be inspected of precision electronic products, and performs target detection on the obtained fused feature map, enriching the hierarchical features of small target defects and improving the expressiveness of small target defect features, so that the model can further improve the detection accuracy, detection speed and detection effect in the small target defect detection task.

[0083] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiment are executed; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks. Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in various embodiments of the present invention. And the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0084] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A lightweight small target defect detection method for precision electronic products, characterized in that: The following steps are involved: Acquire a small target defect data set of precision electronic products, build a defect detection model and use the small target defect data set for training, the defect detection model includes a multi-order feature extraction network and a double up-sampling feature fusion network; The multi-order feature extraction network is used to extract multi-scale features of the small target defect image and enhance the features to generate multiple enhanced feature maps, and the double upsampling feature fusion network is used to perform cross-scale feature fusion on the multiple enhanced feature maps to generate a feature fusion map; Perform target detection on each feature fusion graph to obtain defect detection results, verify and test the defect detection results after iterative training, and output the trained defect detection model when the model performance meets the standards; The trained defect detection model is used as the teacher model, and the lightweight model YOLOv8n-p2 is selected as the student model. The feature layer of the teacher model is transferred to the student model by using knowledge distillation. After distillation, a lightweight defect detection model is obtained. Collect the image data to be inspected of precision electronic products, use a lightweight defect detection model to perform target detection on the image data to be inspected, and output the defect detection results.

2. A lightweight small target defect detection method for precision electronic products according to claim 1, characterized in that: In the defect detection model, the multi-stage feature extraction network is composed of several 3×3 convolutional layers and a multi-order gated feature module C2f-MogaBlock, wherein the multi-order gated feature module C2f-MogaBlock is formed by replacing Bottlenet in C2f with a multi-order gated aggregation module MogaBlock; The double upsampling feature fusion network is composed of a plurality of double upsampling splicing modules and a multi-order gated feature module C2f-MogaBlock module, wherein the double upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module in parallel.

3. A lightweight small target defect detection method for precision electronic products according to claim 2, characterized in that: The small target defect dataset is imported into the defect detection model, and a multi-order feature extraction network is used to obtain the multi-scale features of the small target defect image and perform feature enhancement to generate multiple enhanced feature maps, specifically: The multi-order gated aggregation module MogaBlock in the multi-order gated feature module C2f-MogaBlock is composed of a spatial aggregation block and a channel aggregation block, and the spatial aggregation block includes a feature decomposition module and a multi-order gated aggregation module; In the feature decomposition module of the spatial aggregation block, the feature map first passes through a normalization layer, and after the normalization process, a 1x1 convolution layer is used to obtain local features, and then a global average pooling layer is used to obtain global features, and the global features are subtracted from the local features and multiplied by a scale factor, and summed with the local features, and finally a GELU activation function is used to obtain a multi-order feature map to complete feature decomposition; In the multi-order gated aggregation module of the spatial aggregation block, the multi-order feature map is divided into two branches, one branch passes through a 1x1 convolution layer and then passes through a SiLU activation function to obtain an aggregation branch; the other branch passes through a 5x5 depth convolution and then passes through a split layer to be divided into three channels, one channel is not processed, and the other two channels respectively pass through a 5x5 depth convolution with an expansion rate of 2 and a 7x7 depth convolution with an expansion rate of 3 to obtain three depth convolution feature maps, the three depth convolution feature maps are spliced, and after splicing, they pass through a 1x1 convolution, and then pass through a SiLU activation function to obtain a context branch, the aggregation branch is multiplied by the context branch, and passes through a 1x1 convolution layer, and is added to the original input to obtain a multi-order gated feature map; In the channel aggregation block, the multi-order gated feature map is sequentially subjected to a normalization layer, a 1x1 convolution layer, a 3x3 depth convolution layer, and a GELU activation function to obtain an optimized channel aggregation feature map. The optimized channel aggregation feature map is subjected to a 1x1 convolution layer and then calculated by a GELU activation function, and then subtracted from the original input optimized channel aggregation feature map. After the subtraction, the map is multiplied by the scale factor and then added to the original input optimized channel aggregation feature map to obtain a channel aggregation feature map. The channel aggregation feature map is added to the multi-order gated feature map to obtain an enhanced feature map.

4. A lightweight small target defect detection method for precision electronic products according to claim 3, characterized in that: The calculation formula of the feature map after the multi-stage gated aggregation module Mogablock is expressed as: Y=X+SA(Norm(X)) Z=Y+CA(Norm(Y)) Among them, X is the multi-order feature map obtained by feature decomposition, SA is the spatial aggregation block, Norm(·) is the normalization process, Y is the optimized channel aggregation feature map, CA is the channel aggregation block, and Z is the enhanced feature map.

5. A lightweight small target defect detection method for precision electronic products according to claim 2, characterized in that: After using the multi-order feature extraction network to obtain multiple enhanced feature maps, the double upsampling feature fusion network is used to perform cross-scale feature fusion on the multiple enhanced feature maps to generate a feature fusion map, specifically: A double upsampling splicing module and a multi-order gated feature module in a double upsampling feature fusion network are used to perform cross-scale feature fusion on multiple enhanced feature maps, wherein the double upsampling splicing module is composed of a neighboring interpolation upsampling module and a feature reorganization upsampling module through a parallel structure; The multiple enhanced feature maps are divided into two branches and respectively pass through the neighboring interpolation upsampling module and the feature recombination upsampling module to obtain upsampled feature maps of two interpolation modes. The upsampled feature maps are spliced ​​through a Concat layer to obtain a spliced ​​feature map. The spliced ​​feature map is processed by a multi-order gated feature module to obtain a feature fusion map.

6. A lightweight small target defect detection method for precision electronic products according to claim 5, characterized in that: Perform defect detection on each fusion feature map and output the defect detection results. After each round of cyclic training of small target defect image samples, use the verification data to verify the detection and classification effect of the defect detection model. If the detection and classification effect meets the preset classification effect standard, the current network parameters are retained, otherwise the network parameters are adjusted; The generalization ability of the network is tested using the reserved test data. When the detection and classification performance of the defect detection model meets the preset classification performance standard, the network parameters at this time are used as the structural parameters of the defect detection model, and the trained defect detection model is output.

7. A lightweight small target defect detection method for precision electronic products according to claim 1, characterized in that: The trained defect detection model is used as the teacher model, the lightweight model YOLOv8n-p2 is selected as the student model, and the feature layer of the teacher model is transferred to the student model by using knowledge distillation. Specifically: The trained defect detection model is used as the teacher model, the lightweight model YOLOv8n-p2 is selected as the student model, and the specified feature layer is selected from the teacher model as the knowledge transfer to the specified layer of the student model; Input the small target defect image sample into the student model, calculate the specified layer feature map through forward propagation and save it, and use the 1×1 convolution layer to align the features so that the feature map of the student model and the teacher model have the same number of channels; The feature maps of the teacher model and the student model are input into the MSE loss function to calculate the loss, and finally the sum is obtained to obtain the distillation loss. Backpropagation is performed through the distillation loss to calculate the gradient, and the parameters of all layers of the student model are updated according to the calculated gradient until the model converges. After distillation is completed, a lightweight defect detection model is obtained.

8. A lightweight small target defect detection method for precision electronic products according to claim 7, characterized in that: The distillation loss function is expressed as: Where N, C, H, and W represent batch size, number of channels, feature map height, and feature map width, respectively. Represents the eigenvalue of the student model at the cth channel, height h, width w in the i-th layer. It represents the feature value of the teacher model at the cth channel, height h, width w in the i-th layer, and N represents the total number of feature layers.

9. A lightweight small target defect detection method for precision electronic products according to claim 1, characterized in that: Using visual inspection equipment to collect image data to be inspected of precision electronic products, and preprocessing the image data to be inspected; The preprocessed image data to be detected is input into a lightweight defect detection model to obtain multiple enhanced feature maps of the image data to be detected, a feature fusion map is generated by feature fusion, the feature fusion map is detected by a detection head, and the defect detection result of the image to be detected is output.

Images