Defect detection method and device based on information feature separation and reconstruction

Abstract

The present invention discloses a defect detection method and device based on information feature separation and reconstruction, the method includes the following process: convolution processing is performed on the defect image to obtain a first feature map; the first feature map is input into a weight separation and reconstruction module, a defect region feature map and a background region feature map are obtained through weight separation, the defect region feature map and the background region feature map are feature reconstructed to obtain a weight separation and reconstruction feature map; the weight separation and reconstruction feature map is input into a feature separation and reconstruction module to obtain a deep feature map and a shallow feature map, and pooling is performed after connection to obtain a feature separation and reconstruction feature map; the feature separation and reconstruction feature map is input into a feature pyramid module to obtain an output feature map; the output feature map is upsampled and feature spliced, and input into a detection head for defect detection. The application of the present invention can improve the richness and integrity of defect feature extraction and improve the accuracy of defect detection.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and specifically, to a defect detection technology based on image processing, and more specifically, to a defect detection method and device based on information feature separation and reconstruction. Background Art

[0002] With the rapid development of industrial manufacturing capabilities, the market's requirements for the quality of industrial products are increasing, so the quality inspection of production products is becoming more and more important. In the industrial production process, due to the influence of existing technology, environmental conditions, production raw materials, human errors and other factors, product defects are inevitable. Defect detection, as a part of quality inspection, is to inspect the surface of various products. Surface defects are also the most intuitive manifestation of product quality. In the era of backward technology, manual defect detection was once the mainstream method, but this requires a lot of manpower and material resources, and the detection results are relatively subjective and slow. It has now been replaced by computer methods.

[0003] The surface defect detection method using computers usually adopts feature-based machine vision algorithms, extracting texture features, color features, shape features, etc. from images, and performing defect detection based on the extracted features. Therefore, the extraction of defect features is crucial to the speed and accuracy of detection. In recent years, in order to improve the speed and effect of detection, defect detection based on deep learning has gradually occupied a dominant position. Because the number of defect samples is usually small, the prior art proposes to implement a network for surface defect detection with few samples by adopting two domain generalization strategies and a noise regularization strategy, so that its network performance is equivalent to that of a model trained with a large amount of data. In order to achieve effective extraction of defect features, a multi-level feature extraction module is also used to capture more defect information, and a focus context module is used to fuse defect information, thereby enhancing the model's ability to capture defect information. However, when it comes to complex and usually insignificant defect detection tasks, the prior art is still not rich and complete enough in terms of defect feature extraction, resulting in unsatisfactory defect detection results. Summary of the invention

[0004] The object of the present invention is to provide a defect detection method and device based on information feature separation and reconstruction, so as to improve the richness and completeness of defect feature extraction and improve the accuracy of defect detection.

[0005] In order to achieve the above-mentioned purpose, the defect detection method provided by the present invention is implemented by adopting the following technical scheme:

[0006] A defect detection method based on information feature separation and reconstruction, the method is based on the YOLO architecture and includes the following processes:

[0007] Performing convolution processing on the defect image to obtain a first feature map;

[0008] Inputting the first feature map into a weight separation and reconstruction module, the weight separation and reconstruction module group normalizes the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, obtains a defect area feature map and a background area feature map, performs feature reconstruction on the defect area feature map and the background area feature map, and obtains a weight separation and reconstruction feature map;

[0009] The weight separation and reconstruction feature map is input into a feature separation and reconstruction module, the feature separation and reconstruction module separates the weight separation and reconstruction feature map into an upper channel part and a lower channel part, extracts features from the upper channel part using structural reparameterization to obtain a deep feature map, extracts features from the lower channel part to obtain a shallow feature map, connects the deep feature map and the shallow feature map and performs pooling to obtain a feature separation and reconstruction feature map;

[0010] Inputting the feature separation and reconstruction feature map into a feature pyramid module to obtain an output feature map;

[0011] The output feature map is upsampled and feature concatenated, and input into a detection head for defect detection.

[0012] In some embodiments of the present application, the weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, and obtains a defect area feature map and a background area feature map, including:

[0013] The first feature map is group normalized using the following formula to obtain trainable parameters :

[0014] ;

[0015] The updated channel weight is obtained using the following formula:

[0016] ; ;

[0017] The With the Multiply, and the product is used as the re-weighted feature map;

[0018] The re-weighted feature map is separated into two parts; the first part of the re-weighted feature map is mapped by a Sigmoid function, and weight separation is performed by a first threshold to obtain a first weight group and the second power group ; The re-weighted feature map of the second part is mapped through the Tanh function, and the weight is separated by the second threshold to obtain the third weight group and the fourth estate reorganization The first weight group is greater than the first threshold, the second weight group is not greater than the first threshold, the third weight group is greater than the second threshold, the fourth weight group is not greater than the second threshold;

[0019] The first weight group and the third power group The sum of the additions is then multiplied by the first feature map to obtain a defect area feature map; the second weight group and the fourth power reorganization Adding, and multiplying the added sum by the first feature map to obtain a background area feature map;

[0020] in, is the first feature map; For Results of group normalization; , are the mean and variance within the group, respectively. is a known constant, , is a trainable parameter; is the set of all channel weights; For the The weight of each channel; , All are channel numbers; is the number of channels of the first feature map.

[0021] In some embodiments of the present application, the defect region feature map and the background region feature map are feature reconstructed to obtain a weighted separation and reconstruction feature map, including:

[0022] Separating the defect area feature map and the background area feature map into two parts respectively;

[0023] Performing a weighted summation of the first part of the defect region feature map and the second part of the background region feature map to obtain a first proportional sum; performing a weighted summation of the second part of the defect region feature map and the first part of the background region feature map to obtain a second proportional sum; the weight of the first part of the defect region feature map is greater than the weight of the second part of the background region feature map, and the weight of the second part of the defect region feature map is greater than the weight of the first part of the background region feature map;

[0024] The first ratio sum and the second ratio sum are connected to obtain a weighted separation and reconstruction feature map.

[0025] In some embodiments of the present application, the number of channels in the upper channel portion is greater than the number of channels in the lower channel portion.

[0026] In some embodiments of the present application, the upper channel portion is subjected to feature extraction by structural reparameterization to obtain a depth feature map, including:

[0027] During detection, the upper channel portion is compressed by a convolution module to obtain a feature map of the upper channel after compression during detection; the upper channel feature map after compression during detection is processed by a first deep convolution module to obtain a detection branch output feature map;

[0028] Processing the detection branch output feature map and the training branch output feature map through a squeezing and excitation module to obtain a depth feature map;

[0029] The training branch output feature map is obtained by the following method:

[0030] During training, the feature channel of the upper channel part is compressed by a convolution module to obtain a compressed upper channel feature map during training; the compressed upper channel feature map during training is separated into two parts, which are processed by a second deep convolution module and a third deep convolution module respectively to obtain a second deep convolution processing feature map and a third deep convolution processing feature map; the second deep convolution processing feature map, the third deep convolution processing feature map and the compressed upper channel feature map during training are connected to obtain a training branch output feature map; the convolution kernel of the second deep convolution module is larger than the convolution kernel of the third deep convolution module, and the convolution kernel of the second deep convolution module is equal to the convolution kernel of the first deep convolution module.

[0031] In some embodiments of the present application, feature extraction is performed on the lower channel portion to obtain a shallow feature map, including:

[0032] The convolution module is used to compress the feature channel of the lower channel part to obtain a compressed lower channel feature map;

[0033] Processing the compressed lower channel feature map through point convolution to obtain a point convolution processed feature map;

[0034] The compressed lower channel feature map is connected to the point convolution processing feature map to obtain a shallow feature map.

[0035] In some embodiments of the present application, the deep feature map and the shallow feature map are connected and pooled to obtain a feature separation and reconstruction feature map, including:

[0036] Connecting the deep feature map and the shallow feature map, separating the connected feature map into two parts, processing them respectively through a global maximum pooling module and a global average pooling module, adding the processed results to obtain global spatial information;

[0037] Performing regression processing on the global spatial information through a regression function to obtain regression information;

[0038] The global spatial information is multiplied by the regression information to obtain a feature separation and reconstruction feature map.

[0039] In some embodiments of the present application, the feature separation and reconstruction feature map is input into a feature pyramid module to obtain an output feature map, including:

[0040] The feature separation and reconstruction feature map is divided into a first feature separation and reconstruction feature map and a second feature separation and reconstruction feature map;

[0041] Separating and reconstructing the feature map of the first feature sequentially through n global maximum pooling modules connected in series to perform global maximum pooling processing; Separating and reconstructing the feature map of the second feature sequentially through n global average pooling modules connected in series to perform global average pooling processing; n is a natural number greater than 2;

[0042] Connecting the first feature separation and reconstruction feature map, the second feature separation and reconstruction feature map, the output feature map of each of the global maximum pooling modules, and the output feature map of each of the global average pooling modules to obtain a fused feature map;

[0043] The fused feature map is subjected to dimension-upgrading processing to obtain an output feature map having the same scale as the feature separation and reconstruction feature map.

[0044] In order to achieve the above-mentioned object of the invention, the defect detection device provided by the present invention is implemented by the following method:

[0045] A defect detection device based on information feature separation and reconstruction, the device is based on the YOLO architecture, and includes:

[0046] A first feature map acquisition module, used for performing convolution processing on the defect image to obtain a first feature map;

[0047] A weight separation and reconstruction feature map acquisition module, used for inputting the first feature map into the weight separation and reconstruction module, the weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, obtains a defect area feature map and a background area feature map, performs feature reconstruction on the defect area feature map and the background area feature map, and obtains a weight separation and reconstruction feature map;

[0048] A feature separation and reconstruction feature map acquisition module is used to input the weight separation and reconstruction feature map into a feature separation and reconstruction module, wherein the feature separation and reconstruction module separates the weight separation and reconstruction feature map into an upper channel part and a lower channel part, extracts features from the upper channel part using structural reparameterization to obtain a deep feature map, extracts features from the lower channel part to obtain a shallow feature map, and connects the deep feature map and the shallow feature map and performs pooling to obtain a feature separation and reconstruction feature map;

[0049] An output feature map acquisition module is used to input the feature separation and reconstruction feature map into a feature pyramid module to obtain an output feature map;

[0050] The defect detection module is used to upsample and concatenate the output feature map and input it into a detection head for defect detection.

[0051] Compared with the prior art, the advantages and positive effects of the present invention are:

[0052] The defect detection method and device provided by the present invention utilize a weight separation and reconstruction module to perform weight separation and reconstruction on a defect image, which can not only extract defect features more comprehensively, but also distinguish feature-rich defect areas from information-poor background areas, thereby facilitating the realization of detection tasks; the feature separation and reconstruction module is then utilized to perform feature separation and reconstruction on the feature map after weight separation and reconstruction, and feature extraction and reconstruction are performed from different angles to obtain richer and more complete defect features; the feature map reconstructed by the feature separation and reconstruction module is input into a feature pyramid module for further defect feature extraction, and defect detection is performed based on the extracted features, thereby obtaining higher defect detection accuracy; therefore, the defect detection method and device provided by the present invention can improve the richness and completeness of defect feature extraction, thereby improving the accuracy of defect detection.

[0053] After reading the specific embodiments of the present invention in conjunction with the accompanying drawings, other features and advantages of the present invention will become more clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 It is a flow chart of an embodiment of a defect detection method based on information feature separation and reconstruction according to the present invention;

[0056] Figure 2 It is a structural schematic diagram of an embodiment of a weight separation and reconstruction module in the defect detection method based on information feature separation and reconstruction of the present invention;

[0057] Figure 3 It is a structural schematic diagram of an embodiment of a feature separation and reconstruction module in a defect detection method based on information feature separation and reconstruction of the present invention;

[0058] Figure 4 It is a structural schematic diagram of an embodiment of a feature pyramid module in a defect detection method based on information feature separation and reconstruction of the present invention;

[0059] Figure 5 A schematic diagram for comparing a captured original defect image with an output image obtained by using the defect detection method based on information feature separation and reconstruction of the present invention;

[0060] Figure 6a A schematic diagram of defect detection results obtained by detecting images in a collected tire defect image data set using the defect detection method based on information feature separation and reconstruction of the present invention;

[0061] Figure 6b A schematic diagram of defect detection results obtained by detecting images in the defect data set GC10-DET using the defect detection method based on information feature separation and reconstruction of the present invention;

[0062] Figure 6c A schematic diagram of defect detection results obtained by detecting images in the defect data set NEU-DET using the defect detection method based on information feature separation and reconstruction of the present invention;

[0063] Figure 6d A schematic diagram of defect detection results obtained by detecting images in a PCB defect data set using the defect detection method based on information feature separation and reconstruction of the present invention;

[0064] Figure 7 It is a structural schematic diagram of an embodiment of a defect detection device based on information feature separation and reconstruction of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0066] It should be noted that the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0067] Figure 1 The flowchart of an embodiment of the defect detection method based on information feature separation and reconstruction of the present invention is shown. In this embodiment, the defect detection method is based on the YOLO architecture, including a feature extraction layer, a detection head and an output layer. The feature extraction network extracts image features from the defect image, and the detection head performs target detection and classification according to the feature map output by the feature extraction layer. Finally, the output layer outputs the detection results including the bounding box position, defect category, etc.

[0068] like Figure 1 As shown, this embodiment adopts the following process to perform defect detection.

[0069] S11: Perform convolution processing on the defect image to obtain a first feature map.

[0070] After the defect image is processed by convolution, the first feature map obtained is relatively shallow feature information.

[0071] S12: Input the first feature map into a weight separation and reconstruction module to obtain a weight separation and reconstruction feature map.

[0072] The weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters; then, updates the channel weights based on the trainable parameters, and re-weights the group normalized features based on the updated channel weights; then, the re-weighted results are separated into two parts, and weight separation is performed on each part to obtain a defect area feature map and a background area feature map; finally, the defect area feature map and the background area feature map are feature reconstructed to obtain a weight separation and reconstructed feature map.

[0073] S13: Input the weight separation and reconstruction feature map into the feature separation and reconstruction module to obtain the feature separation and reconstruction feature map.

[0074] The feature separation and reconstruction module first separates the weight separation and reconstruction feature map into the upper channel part and the lower channel part; then, the upper channel part is subjected to feature extraction by structural reparameterization to obtain a deep feature map, and the lower channel part is subjected to feature extraction to obtain a shallow feature map; finally, the deep feature map and the shallow feature map are connected and pooled to obtain a feature separation and reconstruction feature map.

[0075] S14: Input the feature separated and reconstructed feature map into the feature pyramid module to obtain an output feature map.

[0076] The specific process of using the feature pyramid module to extract features and obtain an output feature map can be implemented using existing technologies or methods of subsequent embodiments, which are not described or limited in detail in this embodiment.

[0077] S15: Upsample and concatenate the output feature map and input it into the detection head for defect detection.

[0078] The specific implementation of this step can adopt the existing technology, and this embodiment will not be described and limited in detail.

[0079] The defect detection method of the above embodiment, when extracting features from defect images, adopts a combination of a weight separation reconstruction module and a feature separation reconstruction module to realize feature extraction of information feature separation and reconstruction, which can not only extract defect features more comprehensively, but also distinguish feature-rich defect areas from information-poor background areas, perform feature extraction and reconstruction from different angles, obtain richer and more complete defect features, improve the generalization ability of the defect detection model, and have a better detection effect on small targets, thereby improving the accuracy of defect detection results.

[0080] Figure 2 A structural schematic diagram of an embodiment of a weight separation and reconstruction module in a defect detection method based on information feature separation and reconstruction of the present invention is shown.

[0081] like Figure 2 As shown, in this embodiment, the weight separation and reconstruction module adopts the following process to obtain the weight separation and reconstruction feature map:

[0082] First, the first feature map is group normalized using the following formula to achieve standardization of the first feature map and obtain trainable parameters : .

[0083] In the formula, is the first feature map, whose batch axis, number of channels, height and width are all known; For Results of group normalization; , are the mean and variance of all channels in the group in the same feature map dimension, and the specific calculation method is implemented using existing technology; is a known constant, and the recommended value or set value can be used; , is a trainable parameter.

[0084] Group normalization GN is used to divide the channels of the feature map into groups and normalize them by calculating the mean and variance within each group. Trainable parameters in the group normalization formula The change of determines the variance of the pixels in the channel space within the group to a certain extent. This means that the greater the change in spatial pixels here, the richer the corresponding spatial information. Therefore, To distinguish the defect area from the background area.

[0085] Therefore, the following formula (i.e. Figure 2 N in (N) obtains the updated channel weights to characterize the importance of different feature maps through the channel weights.

[0086] ; .

[0087] in, is the set of all channel weights; For the The weight of each channel; , All are channel numbers; is the number of channels of the first feature map, which is a known value.

[0088] Then, the group normalization result With the updated channel weight Multiply them together, and the product is used as the reweighted feature map.

[0089] Then, the re-weighted feature map is separated into two parts in the channel dimension. Then, the first part of the re-weighted feature map is mapped to the range (0,1) through the Sigmoid function. And the weight separation is performed through the first threshold T1 to obtain the first weight group and the second power group The re-weighted feature map of the second part is mapped through the Tanh function to the range of (-1, 1), and the weight is separated by the second threshold T2 to obtain the third weight group and the fourth estate reorganization Among them, the first equity group Greater than the first threshold T1, the second weight group Not greater than the first threshold T1; the third weight group Greater than the second threshold T2, the fourth weight group The first threshold value T1 and the second threshold value T2 are not greater than the second threshold value T2. Thus, the weights are separated by using the first threshold value T1 and the second threshold value T2. The values ​​of the first threshold value T1 and the second threshold value T2 can be set as required.

[0090] In some embodiments, considering that the mapping ranges of the Sigmoid function and the Tanh function are different, the first threshold T1 and the second threshold T2 are determined to have different values ​​according to the mapping ranges of the corresponding functions.

[0091] After obtaining the new weights, the weights above the threshold and the weights below the threshold generated by different functions are added together, and the sum is then added to the first feature map Multiply them together to obtain feature maps of different regions. That is, the first weight group and third-party restructuring Add, and then add the sum to the first feature map Multiply them together to obtain the defect area feature map, which is recorded as ; Reorganize the second power and the fourth estate reorganization Add, and then add the sum to the first feature map Multiply them together to obtain the background area feature map, recorded as Thus, the feature map is separated into defect area feature maps And background area feature map Two parts, defect area feature map Contains rich defect information, background area feature map Contains very little defect information.

[0092] In order to avoid missing defect features, the defect area feature map And background area feature map Perform feature reconstruction to obtain the weighted separation and reconstruction feature map.

[0093] Specifically, the defect area feature map It is separated into two parts in the channel dimension, namely and ; The background area feature map Separated into two parts, namely and Then, the first part of the defect area feature map And the background area feature map in the second part Perform weighted summation to obtain the first proportion and ; The second part of the defect area feature map Compared with the background area feature map in the first part Perform weighted summation to obtain the second proportional sum Moreover, when weighted summing, the first part of the defect area feature map The weight is greater than the background area feature map in the second part The weight of the second part is the defect area feature map The weight is greater than the background area feature map in the first part Therefore, using the background area feature map The information is supplemented to avoid using only the defect area feature map However, there is a problem of information omission; and in the weighted summation process, a higher weight is given to the defect area feature map with rich defect features to make full use of the information-rich defect features.

[0094] exist Figure 2 In the embodiment, the first part of the defect area characteristic map The weights of the second part and the defect area feature map The weights are all 0.6. The first part is the background area feature map The weights of the second part and the background area feature map The weights of are all 0.4. In some other embodiments, other weight values ​​may be set according to actual detection requirements, as long as the weight value of the defect region feature map is greater than the weight value of the background region feature map.

[0095] Finally, the first ratio and With the second ratio and Connection (ie Figure 2 C in the figure), obtain the weight separation and reconstruction feature map .

[0096] pass Figure 2 The weighted separation and reconstruction feature map obtained by separating and reconstructing the weights of the embodiment It not only contains comprehensive and rich defect features, but also clearly distinguishes the defect area from the background area, which is helpful for the subsequent defect detection.

[0097] Figure 3 A structural schematic diagram of an embodiment of a feature separation and reconstruction module in a defect detection method based on information feature separation and reconstruction of the present invention is shown.

[0098] like Figure 3 As shown, in this embodiment, the input of the feature separation and reconstruction module is the weight separation and reconstruction feature map output by the weight separation and reconstruction module. The feature separation and reconstruction module uses the following process to obtain the feature separation and reconstruction feature map.

[0099] First, in the channel dimension, the weights are separated and the feature map is reconstructed Separated into the upper channel part a and the lower channel part (1-a), and the number of channels in the upper channel part a is greater than the number of channels in the lower channel part (1-a). Reconstruct the feature map by weight separation It is separated into two parts with different numbers of channels, so as to realize multi-branch complex feature extraction for the defect feature area with most number of channels, and simple feature extraction for the defect feature area with small number of channels, so as to effectively extract the defect features while reducing the complexity of the network and improving the defect prediction speed.

[0100] For the upper channel part a with a large number of channels, structural reparameterization is used for feature extraction to obtain the deep feature map Y1. In addition, different network architectures are used in the training process and the detection process during feature extraction.

[0101] Specifically, during detection, execute Figure 3 The predict branch in : The convolution module Conv is used to compress the feature channel of the upper channel part a to improve the computational efficiency and obtain the compressed upper channel feature map during detection. Then, the compressed upper channel feature map during detection is processed by the first deep convolution module, i.e., the first DWC, to obtain the detection branch output feature map P.

[0102] Then, the output feature map P of the detection branch and the output feature map T of the training branch are processed by the squeezing and excitation module SE, and finally the size of the feature map processed by the squeezing and excitation module SE is adjusted by the convolution module Adjust_Conv to obtain the deep feature map Y1.

[0103] By processing the detection branch output feature map P and the training branch output feature map T through the squeeze and excitation module SE, the nonlinear relationship between each channel can be learned and the dependency relationship between each channel can be better obtained. The specific structure and processing process of the squeeze and excitation module SE can be implemented using existing technologies.

[0104] The training branch output feature map T is obtained using the following method:

[0105] During training, execute Figure 3 The train branch in : The convolution module is used to compress the feature channel of the upper channel part a to improve the computational efficiency and obtain the compressed upper channel feature map during training. Then, the compressed upper channel feature map during training is separated into two parts, which are processed by the second deep convolution module, i.e., the second DWC, and the third deep convolution module, i.e., the third DWC, respectively, to obtain the second deep convolution processing feature map and the third deep convolution processing feature map. Then, the second deep convolution processing feature map, the third deep convolution processing feature map and the compressed upper channel feature map during training are connected to obtain the training branch output feature map T.

[0106] Among them, the convolution kernel of the second deep convolution module is larger than the convolution kernel of the third deep convolution module, and the convolution kernel of the second deep convolution module is equal to the convolution kernel of the first deep convolution module used in detection. The convolution kernels of each deep convolution module can be selected according to the detection accuracy and speed requirements. In some embodiments, the convolution kernels of the first deep convolution module and the second deep convolution module are 3×3, which has a theoretically higher computational density and can produce better results in a shorter time. The convolution kernel of the third deep convolution module is 1×1.

[0107] use Figure 3 The structure of the embodiment adopts different network architectures during training and detection for the upper channel part, which can fully utilize the effectiveness of the multi-branch topology structure in extracting features during the training process, and avoid the disadvantage of the slow speed of the multi-branch topology structure during the detection process, thereby realizing the fusion of feature extraction effectiveness and efficiency.

[0108] For the lower channel part (1-a) with a small number of channels, a relatively simple convolution structure is used for feature extraction to obtain a shallow feature map Y2.

[0109] Specifically, for the lower channel part (1-a), the convolution module Conv is first used to compress the feature channel to improve the calculation efficiency and obtain the compressed lower channel feature map.

[0110] Then, the compressed lower channel feature map is processed by point convolution PWC to obtain a point convolution processed feature map. The specific implementation of processing by point convolution PWC can adopt existing technology.

[0111] Finally, the compressed lower channel feature map is connected with the point convolution processing feature map to obtain the shallow feature map Y2.

[0112] Then, the deep feature map Y1 and the shallow feature map Y2 are connected in the channel dimension to obtain the connected feature map Y3.

[0113] The deep feature map Y1 obtained by the above process pays more attention to high-level semantic information, while the shallow feature map Y2 pays more attention to low-level position and shape information. By connecting the deep feature map Y1 and the shallow feature map Y2 in the channel dimension to obtain the connected feature map Y3, the two feature maps can be fully integrated and complement each other, which is conducive to comprehensive mining of defect information and has good robustness to complex and changeable defect features.

[0114] In this embodiment, the connected feature map Y3 continues to be processed.

[0115] First, the connected feature map Y3 is separated into two parts Y31 and Y32. Y31 is pooled by the global maximum pooling module MaxPool2d, and Y32 is pooled by the global average pooling module AvgPool2d. The results of the two parts after pooling are added together to obtain the global spatial information Y4.

[0116] Then, the global spatial information Y4 is regressed through the regression function Softmax to obtain regression information.

[0117] Then, the global spatial information Y4 is multiplied by the regression information to obtain the feature separation and reconstruction feature map .

[0118] use Figure 3 When performing feature separation and reconstruction in the embodiment, the feature map is separated multiple times, and different operations are performed on the separated features, so as to improve the robustness of the model and obtain the features generated by different methods, which is convenient for better understanding of the features in subsequent processing and obtaining more comprehensive defect information, thereby improving the effectiveness and accuracy of feature extraction and defect detection.

[0119] Figure 4 A structural schematic diagram of an embodiment of a feature pyramid module in a defect detection method based on information feature separation and reconstruction of the present invention is shown.

[0120] like Figure 4 As shown, in this embodiment, the input of the feature pyramid module is the feature separation and reconstruction feature map output by the feature separation and reconstruction module. The feature pyramid module uses a structure that processes the pooling results in parallel and in series. Specifically, the feature is separated and the feature map is reconstructed. First, convolution processing is performed through the convolution layer to increase the receptive field of the model and improve the expressiveness of the features; then, the processed features are divided into two parts: the first feature separation and reconstruction feature map Z1 and the second feature separation and reconstruction feature map Z2.

[0121] The first feature separation and reconstruction feature map Z1 is sequentially processed by three global maximum pooling modules MaxPool2d connected in series for global maximum pooling, and three pooled output feature maps Z11, Z12, and Z13 are obtained. The second feature separation and reconstruction feature map Z2 is sequentially processed by three global average pooling modules AvgPool2d connected in series for global average pooling, and three pooled output feature maps Z21, Z22, and Z23 are obtained.

[0122] The first feature separation and reconstructed feature map Z1, the second feature separation and reconstructed feature map Z2, the output feature maps Z11, Z12, Z13 of each global maximum pooling module, and the output feature maps Z21, Z22, Z23 of each global average pooling module, a total of eight feature maps, are connected in the channel dimension to obtain a fused feature map after eight-dimensional fusion.

[0123] The fused feature map is dimensionally upgraded to obtain an output feature map with the same scale as the feature separation and reconstructed feature map.

[0124] use Figure 4 The feature pyramid module of the embodiment separates the input feature map into two parts and extracts features through different pooling methods, which is conducive to a more comprehensive and detailed understanding of feature information in the entire feature extraction layer. For defect detection, this method increases the accuracy of defect area selection and is less likely to miss small information. After pooling separately, the features of multiple dimensions are fused to achieve feature reconstruction, so that the reconstructed features have stronger semantic information, thereby achieving better defect detection effects.

[0125] In some other embodiments, the number of global maximum pooling modules and the number of global average pooling modules connected in series is not limited to three, and can also be two or more than three. The specific number can be selected according to actual needs.

[0126] Figure 5 A schematic diagram showing a comparison between a captured original defect image and an output image obtained by using the defect detection method based on information feature separation and reconstruction of the present invention.

[0127] Figure 5 In the figure, the defect model reconstructed based on information feature separation is represented as YOLO-RVW. The original defect image collected is used as the input image and input into YOLO-RVW for defect detection to obtain the output image. The defect area is selected in the output image, and the defect type and other information are marked.

[0128] Figure 6a-6d The figure shows a schematic diagram of defect detection results obtained by detecting different data sets using the defect detection method based on information feature separation and reconstruction of the present invention.

[0129] Figure 6a This is a schematic diagram of the results obtained by detecting images in the collected tire defect image dataset. Figure 6b This is a schematic diagram of the results obtained by detecting images in the defect dataset GC10-DET. Figure 6c This is a schematic diagram of the results obtained by detecting images in the defect dataset NEU-DET. Figure 6dThe following is a schematic diagram of the results obtained by detecting images in the PCB defect dataset. In each defect detection result schematic diagram, the area selected by the rectangular box is the defect area of ​​the detection output, and the type and confidence of the defect are marked around the rectangular box.

[0130] The defect detection method YOLO-RVW of the present invention and the existing traditional YOLOv8 are respectively used to perform defect detection on the collected tire defect image data set, and the detection results are compared as shown in Table 1 below. In Table 1, Precision is the precision rate, Recall is the recall rate, mAP50 is the average precision mean when the IoU loss function threshold is 0.5, and mAP50-95 is the average precision mean corresponding to the IoU loss function threshold from 0.5 to 0.95. It can be seen from Table 1 that the defect detection method proposed by the present invention has higher detection accuracy.

[0131] Table 1 Comparison of test results

[0132] method Precision Recall mAP50 mAP50-95 YOLOv8 0.679 0.391 0.530 0.236 YOLO-RVW 0.791 0.440 0.582 0.246

[0133] Figure 7 A structural schematic diagram of an embodiment of a defect detection device based on information feature separation and reconstruction of the present invention is shown.

[0134] like Figure 7 As shown, the defect detection device of this embodiment is based on the YOLO architecture, including structural units, functions of the structural units and relationships between them, as follows:

[0135] The defect detection device includes:

[0136] The first feature map acquisition module 21 is used to perform convolution processing on the defect image to obtain a first feature map.

[0137] The weight separation and reconstruction feature map acquisition module 22 is used to input the first feature map acquired by the first feature map acquisition module 21 into the weight separation and reconstruction module, the weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, obtains a defect area feature map and a background area feature map, reconstructs the defect area feature map and the background area feature map, and obtains a weight separation and reconstruction feature map.

[0138] The feature separation and reconstruction feature map acquisition module 23 is used to input the weight separation and reconstruction feature map output by the weight separation and reconstruction feature map acquisition module 22 into the feature separation and reconstruction module, the feature separation and reconstruction module separates the weight separation and reconstruction feature map into an upper channel part and a lower channel part, performs feature extraction on the upper channel part using structural reparameterization to obtain a deep feature map, performs feature extraction on the lower channel part to obtain a shallow feature map, connects the deep feature map and the shallow feature map and performs pooling to obtain the feature separation and reconstruction feature map.

[0139] The output feature map acquisition module 24 is used to input the feature separation and reconstruction feature map output by the feature separation and reconstruction feature map acquisition module 23 into the feature pyramid module to obtain an output feature map.

[0140] The defect detection module 25 is used to upsample and stitch features of the output feature map obtained by the output feature map acquisition module 24, and input the output feature map into the detection head for defect detection.

[0141] The detection device of the above structure runs the corresponding software program and performs the corresponding functions according to Figure 1 Defect detection method embodiment and other embodiments of the process of defect detection, to achieve Figure 1 The corresponding technical effects of the embodiment and other embodiments.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for a person skilled in the art to modify the technical solutions described in the aforementioned embodiments or to replace some of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions claimed to be protected by the present invention.

Claims

1. A defect detection method based on information feature separation and reconstruction, characterized in that: The method is based on the YOLO architecture and includes the following processes: Performing convolution processing on the defect image to obtain a first feature map; Inputting the first feature map into a weight separation and reconstruction module, the weight separation and reconstruction module group normalizes the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, obtains a defect area feature map and a background area feature map, performs feature reconstruction on the defect area feature map and the background area feature map, and obtains a weight separation and reconstruction feature map; The weight separation and reconstruction feature map is input into a feature separation and reconstruction module, the feature separation and reconstruction module separates the weight separation and reconstruction feature map into an upper channel part and a lower channel part, extracts features from the upper channel part using structural reparameterization to obtain a deep feature map, extracts features from the lower channel part to obtain a shallow feature map, connects the deep feature map and the shallow feature map and performs pooling to obtain a feature separation and reconstruction feature map; Inputting the feature separation and reconstruction feature map into a feature pyramid module to obtain an output feature map; The output feature map is upsampled and feature concatenated, and input into a detection head for defect detection.

2. The defect detection method based on information feature separation and reconstruction according to claim 1 is characterized in that: The weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, and obtains a defect area feature map and a background area feature map, including: The first feature map is group normalized using the following formula to obtain trainable parameters : ； The updated channel weight is obtained using the following formula: ； ； The With the Multiply, and the product is used as the re-weighted feature map; The re-weighted feature map is separated into two parts; the first part of the re-weighted feature map is mapped by a Sigmoid function, and weight separation is performed by a first threshold to obtain a first weight group and the second power group ; The re-weighted feature map of the second part is mapped through the Tanh function, and the weight is separated by the second threshold to obtain the third weight group and the fourth estate reorganization The first weight group is greater than the first threshold, the second weight group is not greater than the first threshold, the third weight group is greater than the second threshold, the fourth weight group is not greater than the second threshold; The first weight group and the third power group The sum of the additions is then multiplied by the first feature map to obtain a defect area feature map; the second weight group and the fourth power reorganization Adding, and multiplying the added sum by the first feature map to obtain a background area feature map; in, is the first feature map; For Results of group normalization; , are the mean and variance within the group, respectively. is a known constant, , is a trainable parameter; is the set of all channel weights; For the The weight of each channel; , All are channel numbers; is the number of channels of the first feature map.

3. The defect detection method based on information feature separation and reconstruction according to claim 2 is characterized in that: Reconstructing the defect region feature map and the background region feature map to obtain a weighted separation and reconstruction feature map, including: Separating the defect area feature map and the background area feature map into two parts respectively; Performing a weighted summation of the first part of the defect region feature map and the second part of the background region feature map to obtain a first proportional sum; performing a weighted summation of the second part of the defect region feature map and the first part of the background region feature map to obtain a second proportional sum; the weight of the first part of the defect region feature map is greater than the weight of the second part of the background region feature map, and the weight of the second part of the defect region feature map is greater than the weight of the first part of the background region feature map; The first ratio sum and the second ratio sum are connected to obtain a weighted separation and reconstruction feature map.

4. The defect detection method based on information feature separation and reconstruction according to claim 1 is characterized in that: The number of channels in the upper channel portion is greater than the number of channels in the lower channel portion.

5. The defect detection method based on information feature separation and reconstruction according to claim 4 is characterized in that: The upper channel portion is subjected to feature extraction by structural reparameterization to obtain a deep feature map, including: During detection, the upper channel portion is compressed by a convolution module to obtain a feature map of the upper channel after compression during detection; the upper channel feature map after compression during detection is processed by a first deep convolution module to obtain a detection branch output feature map; Processing the detection branch output feature map and the training branch output feature map through a squeezing and excitation module to obtain a depth feature map; The training branch output feature map is obtained by the following method: During training, the feature channel of the upper channel part is compressed by a convolution module to obtain a compressed upper channel feature map during training; the compressed upper channel feature map during training is separated into two parts, which are processed by a second deep convolution module and a third deep convolution module respectively to obtain a second deep convolution processing feature map and a third deep convolution processing feature map; the second deep convolution processing feature map, the third deep convolution processing feature map and the compressed upper channel feature map during training are connected to obtain a training branch output feature map; the convolution kernel of the second deep convolution module is larger than the convolution kernel of the third deep convolution module, and the convolution kernel of the second deep convolution module is equal to the convolution kernel of the first deep convolution module.

6. The defect detection method based on information feature separation and reconstruction according to claim 4 is characterized in that: Extracting features from the lower channel portion to obtain a shallow feature map includes: The convolution module is used to compress the feature channel of the lower channel part to obtain a compressed lower channel feature map; Processing the compressed lower channel feature map through point convolution to obtain a point convolution processed feature map; The compressed lower channel feature map is connected to the point convolution processing feature map to obtain a shallow feature map.

7. The defect detection method based on information feature separation and reconstruction according to claim 4 is characterized in that: The deep feature map and the shallow feature map are connected and pooled to obtain the feature separation and reconstruction feature map, including: Connecting the deep feature map and the shallow feature map, separating the connected feature map into two parts, processing them respectively through a global maximum pooling module and a global average pooling module, adding the processed results to obtain global spatial information; Performing regression processing on the global spatial information through a regression function to obtain regression information; The global spatial information is multiplied by the regression information to obtain a feature separation and reconstruction feature map.

8. The defect detection method based on information feature separation and reconstruction according to any one of claims 1 to 7, characterized in that: The feature separation and reconstruction feature map is input into the feature pyramid module to obtain an output feature map, including: The feature separation and reconstruction feature map is divided into a first feature separation and reconstruction feature map and a second feature separation and reconstruction feature map; Separating and reconstructing the feature map of the first feature sequentially through n global maximum pooling modules connected in series to perform global maximum pooling processing; Separating and reconstructing the feature map of the second feature sequentially through n global average pooling modules connected in series to perform global average pooling processing; n is a natural number greater than 2; Connecting the first feature separation and reconstruction feature map, the second feature separation and reconstruction feature map, the output feature map of each of the global maximum pooling modules, and the output feature map of each of the global average pooling modules to obtain a fused feature map; The fused feature map is subjected to dimension-upgrading processing to obtain an output feature map having the same scale as the feature separation and reconstruction feature map.

9. A defect detection device based on information feature separation and reconstruction, characterized in that: The device is based on the YOLO architecture and includes: A first feature map acquisition module, used for performing convolution processing on the defect image to obtain a first feature map; A weight separation and reconstruction feature map acquisition module, used for inputting the first feature map into the weight separation and reconstruction module, the weight separation and reconstruction module performs group normalization on the first feature map to obtain trainable parameters, obtains updated channel weights based on the trainable parameters, re-weights the group-normalized features based on the updated channel weights, separates the re-weighted results into two parts, performs weight separation on each part, obtains a defect area feature map and a background area feature map, performs feature reconstruction on the defect area feature map and the background area feature map, and obtains a weight separation and reconstruction feature map; A feature separation and reconstruction feature map acquisition module is used to input the weight separation and reconstruction feature map into a feature separation and reconstruction module, wherein the feature separation and reconstruction module separates the weight separation and reconstruction feature map into an upper channel part and a lower channel part, extracts features from the upper channel part using structural reparameterization to obtain a deep feature map, extracts features from the lower channel part to obtain a shallow feature map, and connects the deep feature map and the shallow feature map and performs pooling to obtain a feature separation and reconstruction feature map; An output feature map acquisition module is used to input the feature separation and reconstruction feature map into a feature pyramid module to obtain an output feature map; The defect detection module is used to upsample and concatenate the output feature map and input it into a detection head for defect detection.

Images