Multi-scale detection methods, systems, equipment and media for industrial surface defects

Abstract

The present invention is applicable to the field of computer vision and provides a multi-scale detection method for industrial surface defects, including: acquiring a first image, performing semantic extraction processing on the first image to obtain a second image; fusing the shallow features of the first image and the deep features of the second image to obtain a third image; using an adaptive channel attention mechanism to process the second image to obtain channel attention, and combining the spatial details of the third image to obtain a fourth image. The advantages of the present invention are mainly concentrated in the following two aspects: first, learning features from a global perspective to capture sufficient semantic information, simplifying the fusion path from semantic features to shallow features; second, introducing an adaptive attention mechanism to better focus on key feature areas in the image, strengthening the learning of local features, so as to more accurately focus on and identify multi-scale defects.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a multi-scale detection method, system, equipment and medium for industrial surface defects. Background Art

[0002] Industrial surface defect data has the characteristics of complex and variable scales, ranging from tiny scratches with a diameter of several millimeters to long and thin scratches across the entire image. Compared with the defect area, the entire image area is often smaller, with an average of less than 5%. In the field of computer vision, semantic segmentation is widely used. It can accurately predict the label of each pixel in the image and realize visual scene understanding. It has a wide range of applications in autonomous driving, medical imaging, image generation and other fields.

[0003] In an existing technology with publication number CN116452813A, an image processing method, system, device and medium based on spatial and semantic information are proposed. Through mutual learning and optimization of shallow spatial information and deep semantic information, the segmentation accuracy is effectively improved, achieving a balance between image processing speed and accuracy.

[0004] However, when processing industrial surface defect images, the detection network based on semantic segmentation tends to pay too much attention to the background area with a large proportion, and ignore the defect area with a small overall proportion but more critical, which in turn affects the defect detection accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a multi-scale detection method, system, device and medium for industrial surface defects, aiming to solve the problem of low detection accuracy caused by processing industrial surface defect images using semantic segmentation method.

[0006] The embodiment of the present invention is implemented as follows: a multi-scale detection method for industrial surface defects, the multi-scale detection method for industrial surface defects comprising:

[0007] Acquire a first image, and perform semantic extraction processing on the first image to obtain a second image;

[0008] fusing shallow features of the first image and deep features of the second image to obtain a third image;

[0009] The second image is processed using an adaptive channel attention mechanism to obtain channel attention, and combined with the spatial details of the third image to obtain a fourth image.

[0010] Further, fusing the shallow features of the first image and the deep features of the second image to obtain a third image specifically includes the following steps:

[0011] The deep features are reduced in dimension through a 1x1 convolution to obtain a feature vector with the same channel dimension as the shallow features;

[0012] The deep features are mapped using a Sigmoid function to obtain a feature weight distribution at each position;

[0013] The feature weights are mapped to the shallow features, and the semantic information portion corresponding to the shallow features and the deep features is strengthened to obtain the third image.

[0014] Further, the step of processing the second image using the adaptive channel attention mechanism to obtain channel attention specifically includes the following steps:

[0015] Learning attention weights for each channel of the second image;

[0016] Use adaptive global average pooling to obtain compressed features;

[0017] Using a convolutional layer to calculate the weights of each channel of the second image;

[0018] The Hard Sigmoid activation function is used for normalization, and the result is multiplied by the attention weight to obtain the channel attention.

[0019] Further, combining the spatial details of the third image to obtain the fourth image specifically comprises: combining the channel attention with the spatial details of the third image using concatenate to obtain the fourth image.

[0020] Furthermore, after obtaining the fourth image, it is necessary to detect image loss, which specifically includes the following steps:

[0021] Obtaining an output loss generated by outputting the fourth image, and obtaining a semantic loss generated by obtaining the second image and the third image;

[0022] Obtaining edge detail loss of the first image;

[0023] Obtaining a mask perception loss generated by the fourth image;

[0024] The output loss, the semantic loss, the edge detail loss and the mask-aware loss are combined to obtain the image loss.

[0025] Furthermore, the method for obtaining the edge detail loss specifically comprises the following steps:

[0026] Performing a convolution operation on the true label of the first image through a Laplacian operator to extract edge information;

[0027] Weightedly fusing the edge information of different scales into a single edge target map, and calculating the edge detail loss;

[0028] The edge region of the first image is learned, and the Dice loss of the edge region is integrated into the edge detail loss as a regularization term.

[0029] Furthermore, the method for obtaining the mask perception loss specifically comprises the following steps:

[0030] Construct a binary mask map using the true label;

[0031] Separating background features and non-background features of the fourth image according to the binary mask image; the non-background features are the sum of defect categories;

[0032] The background features and the non-background features are stacked in the channel dimension, and the binary classification cross entropy loss is calculated in combination with the binary mask map to obtain the mask-perceived loss.

[0033] Another object of an embodiment of the present invention is to provide an industrial surface defect multi-scale detection system, the industrial surface defect multi-scale detection system comprising:

[0034] A backbone network is used to obtain a first image, perform semantic extraction processing on the first image, and obtain a second image;

[0035] An efficient and lightweight mapping module, used for fusing shallow features of the first image and deep features of the second image to obtain a third image;

[0036] An adaptive attention module, configured to process the second image using an adaptive channel attention mechanism to obtain channel attention, and combine the spatial details of the third image to obtain a fourth image;

[0037] An image loss calculation module is used to combine the output loss generated by outputting the fourth image, the semantic loss generated by obtaining the second image and the third image, the edge detail loss of the first image, and the mask perception loss generated by obtaining the fourth image to obtain the image loss.

[0038] Another object of an embodiment of the present invention is a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the multi-scale detection method for industrial surface defects.

[0039] Another object of an embodiment of the present invention is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the multi-scale detection method for industrial surface defects.

[0040] An embodiment of the present invention provides a multi-scale detection method for industrial surface defects. In the field of defect detection, the advantages of the present invention are mainly concentrated in the following two aspects: first, learning features from a global perspective to capture sufficient semantic information simplifies the fusion path from semantic features to shallow features; second, introducing an adaptive attention mechanism to better focus on key feature areas in the image, strengthen the learning of local features, so as to more accurately focus on and identify multi-scale defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a multi-scale detection method for industrial surface defects provided by an embodiment of the present invention;

[0042] Figure 2 A block diagram of the structure of a multi-scale defect detection network based on an adaptive attention mechanism (Adaptive Attention-based Multi-Scale Detail and Semantic Mutually Optimized Network, A2MS-DSMONet) provided in an embodiment of the present invention;

[0043] Figure 3 A structural block diagram of a multi-scale detection system for industrial surface defects provided by an embodiment of the present invention;

[0044] Figure 4 A flowchart of an efficient lightweight mapping module (ELMM) for processing an image provided by an embodiment of the present invention;

[0045] Figure 5 A flowchart of an adaptive attention module (AAM) for processing an image provided by an embodiment of the present invention;

[0046] Figure 6 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0049] Figure 1 The flowchart of the multi-scale detection method of industrial surface defects shows the process of multi-scale detection of industrial surface defects. In the first embodiment, a multi-scale detection method of industrial surface defects is proposed, and the multi-scale detection method of industrial surface defects includes steps S102 to S106:

[0050] Step S102: acquiring a first image, and performing semantic extraction processing on the first image to obtain a second image.

[0051] In this embodiment, the first image is a shallow image, which has a high resolution and contains more spatial detail information, but also has relatively more noise. The second image is a deep image, which is obtained by processing the shallow image, has a relatively low resolution, contains stronger semantic information, but lacks spatial information. Figure 2 The block diagram shows the structure of the multi-scale defect detection network based on the adaptive attention mechanism. The image will first be processed by the backbone network and then enter the mutual optimization network MOM. Figure 2 As shown, the original image is the image to be processed. The original image undergoes 4 steps, and the number of feature map channels is increased and the resolution of the original image is reduced to obtain images 1 to 4 one by one. The original image and images 1 to 4 constitute the backbone network. The backbone network selects lightweight STDCNet, which has a total of 5 stages, each stage stride is 2, the number of feature map channels increases, and the resolution is reduced to 1 / 32 of the input image. In order to obtain features containing global context information, a DAPPM module is added after the backbone network to further extract context information from the low-resolution feature map to obtain the second image. It should be noted that in this embodiment, image 1 is set to be optimized as the first image and the fourth image is optimized as the second image, but in actual operation, as long as the resolution of the first image is greater than that of the second image, the technical solution of this embodiment can be implemented. Therefore, it is not limited to which part of the image in the backbone network the first image and the second image specifically correspond, as long as the resolution requirement is met.

[0052] Step S104: fusing shallow features of the first image and deep features of the second image to obtain a third image.

[0053] In this embodiment, features are learned from a global perspective to capture sufficient semantic information, simplifying the fusion path from semantic features to shallow features.

[0054] Step S106, using an adaptive channel attention mechanism to process the second image to obtain channel attention, and combining it with the spatial details of the third image to obtain a fourth image.

[0055] In this embodiment, an adaptive attention mechanism is introduced to better focus on the key feature areas in the image and strengthen the learning of local features so as to more accurately focus on and identify multi-scale defects. Compared with the mutual learning optimization of shallow spatial information and deep semantic information adopted in the prior art, this embodiment has significantly improved the detection accuracy and speed.

[0056] Specifically, step S104 includes steps S202 to S206:

[0057] Step S202, reducing the dimension of the deep feature through a 1x1 convolution to obtain a feature vector with the same channel dimension as the shallow feature dimension.

[0058] Step S204, mapping the deep features using the Sigmoid function to obtain feature weight distribution at each position.

[0059] Step S206, mapping the feature weights to the shallow features, strengthening the semantic information portion of the shallow features corresponding to the deep features, and obtaining the third image.

[0060] In this embodiment, if Figure 3 and 4 As shown, Figure 3 The structural block diagram of the industrial surface defect multi-scale detection system is shown in Figure 2. Figure 4 express Figure 3 Figure 2 is a flowchart of an efficient and lightweight mapping module for processing images. The efficient and lightweight mapping module executes steps S202 to S206, including two key steps: Sigmoid mapping in step S204 and weight fusion in step S206. First, the characteristics of the Sigmoid function enable it to map any real number into the range of 0-1, and this probability value can be interpreted as the possibility of a certain attribute of a given input sample, that is, the larger the weight value, the more important the feature at the corresponding position. Secondly, the deep features are mapped through the Sigmoid function to obtain the feature weight distribution at each position, and this distribution can more effectively capture the semantic information of different positions. Finally, these weights are mapped to the shallow features to strengthen the part of the shallow features that corresponds to the important semantic information in the deep features, thereby generating optimized edge information. The specific process is as follows:

[0061] (1) The deep features are reduced in dimension through a 1x1 convolution to obtain a feature vector with the same channel dimension as the shallow features.

[0062] (2) The nonlinear transformation is performed through the Sigmoid function to obtain the feature weight distribution between 0 and 1.

[0063] (3) Multiply the feature weight distribution by the shallow features element by element to achieve adaptive channel weighting.

[0064] (4) Feature fusion is then performed through a 1x1 convolution to output weighted shallow features.

[0065] The method of the prior art decouples the deep features into main features and edge features, and combines the shallow features to optimize the edge information. The method of the prior art mainly includes two branches: one is to obtain the main features by calculating the optical flow field and distorting and reconstructing it, and the other is to obtain the edge features by explicitly subtracting the main features from the semantic features. This implementation method has a large number of parameters and a high computational complexity. However, in the actual defect data set, such a complex operation is not required. The above three steps S202~S206 of this embodiment realize a simple and effective fusion from deep semantic features to shallow features, which is lighter and more efficient than the previous method. While reducing the parameters and calculation amount of the model, it ensures the regulatory effect of semantic information on shallow features and improves the response to key areas. Therefore, this embodiment simplifies the fusion process from semantic features to shallow features, and is more suitable for industrial detection applications with high real-time requirements.

[0066] Specifically, step S106 specifically includes steps S302 to S310:

[0067] Step S302, learning the attention weights of each channel of the second image.

[0068] Step S304: using adaptive global average pooling to obtain compressed features.

[0069] Step S306: Use a convolutional layer to calculate the weights of each channel of the second image.

[0070] Step S308, normalize using the Hard Sigmoid activation function, and multiply the result by the attention weight to obtain the channel attention.

[0071] Step S310: Use concatenate to combine the channel attention with the spatial details of the third image to obtain the fourth image.

[0072] In this embodiment, if Figure 3 and 5 As shown, Figure 3The structural block diagram of the industrial surface defect multi-scale detection system is shown in Figure 2. Figure 5 express Figure 3 Flowchart of image processing by adaptive attention module in . In order to solve the problem of feature extraction of targets of different scales, an adaptive attention module AAM is proposed to execute steps S302~S310. In image processing, compared with the entire image, the defect area in the industrial data set often accounts for a small proportion, which makes it easy for the model to pay too much attention to irrelevant areas. Through the adaptive feature extraction method, the attention to each channel can be dynamically adjusted, so that the network can pay more attention to the feature channels that are critical to the current task, while suppressing the attention to the channels that are less relevant to the task, thereby improving the overall model performance.

[0073] Specifically, the adaptive attention module AAM mainly consists of two parts:

[0074] (1) Adaptive Channel Weight (ACW)

[0075] The adaptive channel weight unit performs the method of step S302 to learn the attention weights of each channel. The adaptive channel weight unit contains a learnable parameter weight tensor, and the feature dimension matches the attention module, which is 1×1×C, where C represents the number of input feature channels. The weight is continuously updated during the training process according to the back propagation algorithm, so as to focus on the feature channels that are more important to the current task.

[0076] (2) Efficient Squeeze-and-Excitation (eSE)

[0077] The efficient squeeze excitation unit executes the method of steps S304~S308. The existing squeeze-and-excitation (SE) module uses two fully connected layers to learn channel attention. In order to reduce the amount of calculation, the fully connected layer is used to reduce the dimension first and then increase the dimension, but this easily leads to the loss of channel information. The efficient squeeze excitation unit only uses one convolutional layer to predict the weight of each channel, avoiding the loss of channel information. Its calculation process includes: first, using adaptive global average pooling to obtain 1×1×C compressed features; then calculating the weights of each channel through the convolutional layer; and finally using the HardSigmoid activation function for normalization. In this way, more accurate channel attention can be obtained more efficiently.

[0078] Finally, the attention weights of each channel learned by the adaptive channel weight unit are multiplied by the attention features output by the efficient squeeze excitation unit, thus realizing the adaptive channel attention mechanism. This mechanism enhances the model's perception of effective features, enabling it to dynamically focus on semantic information that is more relevant to the current defect detection task, thereby improving the robustness and generalization performance of the model, and is particularly suitable for industrial surface defect detection of multi-scale targets.

[0079] In the second embodiment, a multi-scale detection method for industrial surface defects is proposed, which includes all the steps and sub-steps in the first embodiment. After step S106, image loss detection is also required, which specifically includes the following steps S402 to S408:

[0080] Step S402, obtaining an output loss generated by outputting the fourth image, and obtaining a semantic loss generated by the second image and the third image.

[0081] Step S404: Obtain edge detail loss of the first image.

[0082] Step S406: Obtain the mask perception loss generated by the fourth image.

[0083] Step S408: combining the output loss, the semantic loss, the edge detail loss and the mask perception loss to obtain an image loss.

[0084] In the field of defect detection, the design of the loss function is crucial to the performance of the model, as it directly affects the model's ability to locate and identify targets. The prior art model uses a basic combined loss function, which includes the output loss and semantic loss given in step S402, to meet the model's needs for losses at different stages. However, in the face of defect targets of various scales in the industrial surface defect dataset, this standard loss function may not be able to fully capture the characteristics of defects of various scales, so the model's detection capability is limited. In order to solve this problem, this embodiment introduces edge detail loss and mask perception loss on the basis of the original loss function. The specific calculation formula is as follows:

[0085]

[0086]

[0087] Among them, λ0, λ1, λ2 and λ3 represent the parameters of training loss; l 0 represents the cross entropy loss of online hard case mining, that is, the output loss generated by outputting the fourth image; l 1 and l 2 represents the semantic loss of the second image and the third image respectively;l edge Indicates the loss of edge details; L 1 represents the total loss obtained by integrating the edge loss into the model as a regular term; L 2 represents the mask-perceptual loss; α represents the weighting parameter for defect category segmentation; β represents the weighting parameter of the mask-perceptual loss; L final Represents the joint loss function given in this embodiment.

[0088] Therefore, this embodiment constructs a new joint loss function L final ,Its advantage is that it can focus on both global information and local features at the same time, and improve the detection capability of multi-scale targets.

[0089] Specifically, the method for obtaining edge detail loss in step S404 specifically includes steps S502 to S506:

[0090] Step S504: performing a convolution operation on the real label of the first image by using a Laplacian operator to extract edge information.

[0091] Step S506: weightedly fuse the edge information of different scales into a single edge target image, and calculate the edge detail loss.

[0092] Step S508 , learning the edge region of the first image, and integrating the Dice loss of the edge region into the edge detail loss as a regularization term.

[0093] In this embodiment, edge detail loss is introduced to focus on edge information. First, the real label is convolved by Laplacian operator to extract edge information. Then the boundary information of different scales is weighted and fused into a single edge target map for loss calculation. In order to further strengthen the model's learning of edge areas, Dice loss is calculated at the same time, which measures the similarity between the predicted boundary and the true boundary. It is then integrated into the total loss of the model as a regularization term, and the result is L 1. Ensure that the model not only focuses on the overall pixel-level classification during training, but also pays enough attention to edge details. This method can significantly improve the model's ability to recognize the edges of objects, especially when dealing with small or fuzzy boundaries in images.

[0094] Specifically, the method for obtaining the mask perception loss in step S406 specifically includes steps S602 to S606:

[0095] Step S602, constructing a binary mask map using the real label.

[0096] Step S604 , separating background features and non-background features of the fourth image according to the binary mask image; the non-background features are the sum of defect categories.

[0097] Step S606: stack the background features and the non-background features in the channel dimension, calculate the binary classification cross entropy loss in combination with the binary mask image, and obtain the mask perception loss.

[0098] In this embodiment, a mask-aware loss is proposed to suppress background interference. First, a mask map is constructed, and all pixels belonging to the background category in the true label are set to 0, and all pixels belonging to the defect category are set to 1, to obtain the corresponding binary mask map. The background category and multiple defect categories output by the network are separated, where the background features are retained and the sum of the defect category features is represented as non-background features. Then, the background features and non-background features are stacked in the channel dimension, and the binary cross entropy loss is calculated in combination with the annotated binary mask map as the mask loss. By introducing mask-aware loss, the model's ability to distinguish between foreground defect areas and background areas can be significantly improved, and the network's understanding of the defect layout distribution in the image can be strengthened, thereby improving the detection effect, especially the recognition of small-scale defects.

[0099] In the third embodiment, the specific experimental details of the above two embodiments are given, which are divided into two processes: ablation experiment and experimental analysis. The experiment adopts Ubuntu 18.04 server configuration, equipped with Inter(R) Core(TM) i9-10900X model CPU, NVIDIA GeForce RTX 3090 model GPU, 24GB memory, etc. The software environment implements the model and performs training and testing based on frameworks such as Python 3.9 and Pytorch 1.10. In terms of data sets, the public standard data set NEU-Seg and the self-built industrial data set leather defect data set are used. The public data set can ensure the comparability between the experimental results and other methods, and the self-built data set can evaluate the adaptability of the model to the actual industrial scene. The combination of the two can comprehensively examine the model performance. The batch size in the parameter configuration is 16, the optimizer is Adam, and the initial learning rate is 0.0001. The evaluation indicators use two indicators, the average cross-class mIoU and the frame rate per second FPS, to evaluate the accuracy and speed of the model from multiple angles. Combining these two evaluation indicators, the improvement of the model in industrial defect detection accuracy and detection speed can be comprehensively evaluated.

[0100] 1. Ablation experiment

[0101] In order to verify the effectiveness of the efficient and lightweight mapping module, adaptive attention module and joint loss function of the above embodiment, this embodiment conducted an ablation experiment on the NEU-Seg dataset. The NEU-Seg dataset contains six typical surface defects of strip steel. This experiment selected three defect types: inclusions, patches and scratches. By removing or replacing relevant modules in a targeted manner, the impact of each module on the model detection performance was analyzed.

[0102] First, the efficient and lightweight mapping module is evaluated, and the results are shown in Table 1. It can be seen that the Baseline directly outputs features without a mapping module and does not adopt the strategy of mutual optimization between shallow and deep layers, achieving 85.1% mIoU and 128.6FPS.

[0103] DM is a main feature and edge feature decoupling mapping module in the prior art (CN116452813A). Compared with the Baseline, mIoU is increased to 90.1%, but the inference speed is reduced from 128.6 FPS to 117.6 FPS. Using the efficient and lightweight mapping module ELMM proposed in the above embodiment, mIoU reaches 91.3% and the speed is increased to 126.7 FPS. Therefore, compared with the existing mapping module, ELMM improves the segmentation speed by 9.1 FPS while improving the accuracy by 1.2% mIoU, proving that the module can achieve a better balance between efficiency and performance.

[0104] Table 1 Ablation experiment of efficient and lightweight mapping module

[0105]

[0106] Next, the effect of the adaptive attention module AAM is verified, and the results are shown in Table 2. AAM uses eSE plus adaptive channel weight ACW. Therefore, it is compared with the SE module and the eSE module respectively. The results show that compared with the SE module, the detection accuracy after the introduction of the AAM module is improved by 0.6% mIoU, and the FPS is increased from 122.9 to 126.9, indicating that the AAM module can not only better focus on the key feature areas, but also accelerate the model operation, making the overall performance of the model lighter and more efficient. Compared with using only the eSE module, the mIoU indicator of the model has also achieved a slight increase after the introduction of the AAM module. This shows that the adaptive attention mechanism can autonomously adjust the degree of attention to the feature channel according to different goals, thereby improving the model's ability to detect multi-scale defects.

[0107] In addition, to explore the optimal insertion position of the AAM module, this embodiment tried the effects of single and double insertions. The results show that inserting AAM twice can achieve the best detection effect, 91.3% mIoU and 126.7 FPS. In summary, the introduction of the adaptive attention module AAM can significantly improve the defect detection performance of the model.

[0108] Finally, the joint loss function is evaluated, and the results are shown in Table 3. Based on the existing loss function, the edge detail loss is added, and the mIoU is increased to 90.7%; the mask-aware loss is added, and the mIoU reaches 90.8%. Finally, using the joint loss function proposed in the above embodiment, the mIoU reaches 91.3%, which is significantly higher than the model using only the basic loss function, and improves the model's ability to learn edges and key areas. These improvements help the model better learn defect edges and foreground features, and detect small targets more accurately.

[0109] Table 2 Ablation experiment of adaptive attention module

[0110]

[0111] Table 3 Joint loss function ablation experiment

[0112]

[0113] Through the above ablation experiments, the effectiveness of the proposed modules and strategies has been verified. After integrating all the improved modules, a segmentation accuracy of 91.3% mIoU and an inference speed of 126.7 FPS were achieved on the NEU-Seg dataset, which greatly improved the detection accuracy while maintaining real-time performance, especially the recognition effect of small targets was significantly enhanced.

[0114] 2. Experimental analysis

[0115] This example comprehensively evaluates the performance of the proposed A2MS-DSMONet model in the multi-scale defect detection task through experimental results on NEU-Seg and self-built leather defect datasets.

[0116] First, from the perspective of quantitative analysis, as shown in Tables 4 and 5, the evaluation indicators of different models on the leather and NEU-Seg defect datasets are listed respectively. It can be seen that A2MS-DSMONet significantly improves the detection speed while maintaining high-precision segmentation compared with the DSMONet model of the existing technology (CN116452813A). Taking the ResNet-50 backbone network as an example, the mIoU of A2MS-DSMONet-B on the leather dataset reaches 91.0%, which is 1.8% higher than that of DSMONet-B; the detection speed is also increased from 67.3 FPS to 71.6 FPS, an increase of about 4.3. On the NEU-Seg dataset, the mIoU of A2MS-DSMONet-B is as high as 91.3%, and the speed reaches 127.9 FPS. This result not only surpasses DSMONet, but also achieves the best balance between speed and accuracy compared with the models Sub-region UNet, FDSNet and PGA-Net that specialize in industrial defect detection in recent years.

[0117] Table 4 Quantitative experimental results of leather dataset

[0118]

[0119] Table 5 Quantitative experimental results of NEU-Seg dataset

[0120]

[0121] Secondly, compared with semantic segmentation models for more general scenarios, A2MS-DSMONet also shows obvious advantages. Taking the leather dataset as an example, compared with classic models such as BiSeNet and STDC-Seg, A2MS-DSMONet-B is at least 3.9 percentage points ahead in accuracy, and the reasoning speed is also very fast. This fully proves that without sacrificing detection accuracy, the adaptive attention mechanism, efficient mapping module, and joint loss function strategies of the above embodiments effectively improve the model's ability to detect multi-scale defects.

[0122] Next, from the perspective of qualitative analysis, the segmentation visualization comparison results on the NEU-Seg dataset are shown. It can be seen that there are fewer types of defect shapes in this dataset, but the image background is complex, which puts a certain test on the robustness of the model. For some long scratch defects that run through the entire image, other models have failed to fully detect all defects, while A2MS-DSMONet can be closer to the true label. Some images contain multiple small point defects, which are missed by DDRNet and STDC2-Seg, while A2MS-DSMONet can detect the location and shape of all small defects. Some are irregularly shaped defects, including DSMONet, which have intermittent segmentation results, while A2MS-DSMONet can continuously and completely outline the contours of the defects.

[0123] In general, on datasets with complex backgrounds such as NEU-Seg, A2MS-DSMONet still performs well, and can accurately detect defects of various shapes and sizes. The segmentation edges are clear and smooth, and adjacent defects can be well separated, verifying the robustness and applicability of the model.

[0124] From the comprehensive experimental data and visualization results, it can be seen that the A2MS-DSMONet model of the above embodiment achieves an excellent balance between speed and accuracy in multi-scale industrial defect detection tasks by constructing an adaptive attention mechanism to dynamically focus on key features, designing a simple and efficient shallow mapping module to accelerate reasoning, and introducing a joint loss function that constrains global and local features. This provides a new solution for building a real-time and accurate industrial inspection system.

[0125] In the fourth embodiment, Figure 2 , 3 , 4 and 5, a multi-scale detection system for industrial surface defects is proposed, and the multi-scale detection system for industrial surface defects includes:

[0126] A backbone network is used to obtain a first image, perform semantic extraction processing on the first image, and obtain a second image;

[0127] An efficient and lightweight mapping module, used for fusing shallow features of the first image and deep features of the second image to obtain a third image;

[0128] An adaptive attention module, configured to process the second image using an adaptive channel attention mechanism to obtain channel attention, and combine the spatial details of the third image to obtain a fourth image;

[0129] An image loss calculation module is used to combine the output loss generated by outputting the fourth image, the semantic loss generated by obtaining the second image and the third image, the edge detail loss of the first image, and the mask perception loss generated by obtaining the fourth image to obtain the image loss.

[0130] In this embodiment, the backbone network has been described in detail in the first embodiment, the efficient and lightweight mapping module performs the above steps S202 to S206, the adaptive attention module performs the above steps S302 to S310, and the image loss calculation module performs the steps S402 to S408 and their sub-steps in the second embodiment. The beneficial effects of this system are consistent with the above, and will not be repeated here.

[0131] Figure 6 The internal structure diagram of a computer device in one embodiment is shown. The computer device includes a processor, a memory, a network interface and an input device connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a multi-scale detection method for industrial surface defects. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute a multi-scale detection method for industrial surface defects.

[0132] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In a fifth embodiment, a computer device is provided, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0134] Acquire a first image, and perform semantic extraction processing on the first image to obtain a second image;

[0135] fusing shallow features of the first image and deep features of the second image to obtain a third image;

[0136] Processing the second image using an adaptive channel attention mechanism to obtain channel attention, and combining the spatial details of the third image to obtain a fourth image;

[0137] The output loss generated by outputting the fourth image, the semantic loss generated by obtaining the second image and the third image, the edge detail loss of the first image, and the mask perception loss generated by obtaining the fourth image are combined to obtain the image loss.

[0138] In a sixth embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following steps:

[0139] Acquire a first image, and perform semantic extraction processing on the first image to obtain a second image;

[0140] fusing shallow features of the first image and deep features of the second image to obtain a third image;

[0141] Processing the second image using an adaptive channel attention mechanism to obtain channel attention, and combining the spatial details of the third image to obtain a fourth image;

[0142] The output loss generated by outputting the fourth image, the semantic loss generated by obtaining the second image and the third image, the edge detail loss of the first image, and the mask perception loss generated by obtaining the fourth image are combined to obtain the image loss.

[0143] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-scale detection method for industrial surface defects, characterized in that: The multi-scale detection method for industrial surface defects comprises: Acquire a first image, and perform semantic extraction processing on the first image to obtain a second image; fusing shallow features of the first image and deep features of the second image to obtain a third image; Processing the second image using an adaptive channel attention mechanism to obtain channel attention, and combining the spatial details of the third image to obtain a fourth image; The fusing of the shallow features of the first image and the deep features of the second image to obtain the third image specifically includes the following steps: The deep features are reduced in dimension through a 1x1 convolution to obtain a feature vector with the same channel dimension as the shallow features; Mapping the feature vector using the Sigmoid function to obtain feature weight distribution at each position; Mapping the feature weights to the shallow features, strengthening the semantic information portion of the shallow features corresponding to the deep features, and obtaining the third image; The step of processing the second image using the adaptive channel attention mechanism to obtain channel attention specifically includes the following steps: Learning attention weights for each channel of the second image; Using adaptive global average pooling, obtaining compression features of the second image; Using a convolutional layer to calculate the attention weights of each channel of the second image; The compressed features are normalized using a Hard Sigmoid activation function, and the result is multiplied by the attention weight to obtain the channel attention.

2. The multi-scale detection method for industrial surface defects according to claim 1, characterized in that: The combining the spatial details of the third image to obtain the fourth image specifically comprises: combining the channel attention with the spatial details of the third image using concatenate to obtain the fourth image.

3. The multi-scale detection method for industrial surface defects according to claim 1, characterized in that: After obtaining the fourth image, it is necessary to detect image loss, which specifically includes the following steps: Obtaining an output loss generated by outputting the fourth image, and obtaining a semantic loss generated by obtaining the second image and the third image; Obtaining edge detail loss of the first image; Obtaining a mask perception loss generated by the fourth image; The output loss, the semantic loss, the edge detail loss and the mask-aware loss are combined to obtain the image loss.

4. The multi-scale detection method for industrial surface defects according to claim 3, characterized in that: The method for obtaining the edge detail loss specifically comprises the following steps: Performing a convolution operation on the true label of the first image through a Laplacian operator to extract edge information; Weightedly fusing the edge information of different scales into a single edge target map, and calculating the edge detail loss; The edge region of the first image is learned, and the Dice loss of the edge region is integrated into the image loss as a regularization term.

5. The multi-scale detection method for industrial surface defects according to claim 3, characterized in that: The method for obtaining the mask perception loss specifically comprises the following steps: Constructing a binary mask map using the true label of the first image; Separating background features and non-background features of the fourth image according to the binary mask image; the non-background features are the sum of defect categories; The background features and the non-background features are stacked in the channel dimension, and the binary classification cross entropy loss is calculated in combination with the binary mask map to obtain the mask-perceived loss.

6. An industrial surface defect multi-scale detection system, characterized in that: The industrial surface defect multi-scale detection system comprises: A backbone network is used to obtain a first image, perform semantic extraction processing on the first image, and obtain a second image; An efficient and lightweight mapping module, used for fusing the shallow features of the first image and the deep features of the second image to obtain a third image; the fusing of the shallow features of the first image and the deep features of the second image to obtain the third image specifically comprises the following steps: reducing the dimension of the deep features through a 1x1 convolution to obtain a feature vector with the same channel dimension as the shallow feature dimension; mapping the feature vector using a Sigmoid function to obtain a feature weight distribution at each position; mapping the feature weights to the shallow features to strengthen the semantic information portion corresponding to the shallow features and the deep features to obtain the third image; An adaptive attention module, configured to process the second image using an adaptive channel attention mechanism to obtain channel attention, and combine the spatial details of the third image to obtain a fourth image; The adaptive attention module includes an adaptive channel weight unit and an efficient squeeze excitation unit; The adaptive channel weight unit is used to learn the attention weights of each channel of the second image; The efficient squeeze excitation unit uses adaptive global average pooling to obtain a compression feature of the second image, uses a convolutional layer to calculate the attention weight of each channel of the second image, uses a Hard Sigmoid activation function to normalize the compression feature, and multiplies the result by the attention weight to obtain the channel attention; An image loss calculation module is used to combine the output loss, semantic loss, edge detail loss and mask perception loss to obtain the image loss; the output loss is obtained from the fourth image, the semantic loss is obtained from the second image and the third image, the edge detail loss is obtained from the first image, and the mask perception loss is obtained from the fourth image.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the multi-scale detection method for industrial surface defects as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the multi-scale detection method for industrial surface defects according to any one of claims 1 to 5.

Images