Surface defect detection method and device based on spinning adjacency attention

Abstract

The invention relates to the technical field of surface defect detection, in particular to a surface defect detection method and device based on spinning adjacency attention, and the method comprises the steps: obtaining image data of a surface target area, and extracting a first feature map; the first feature map is input into a spinning adjacency attention network, a second feature map with spinning adjacency attention is obtained, the spinning adjacency attention network at least comprises a spinning self-adaptive attention module and an adjacency routing attention module, and the spinning self-adaptive attention module comprises a spinning self-adaptive attention module and an adjacency routing attention module; the spin adaptive attention module is used for capturing multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module is used for distinguishing similar defect areas and defect-free areas through inter-class modeling fine-grained marks by a spatial adjacency matrix; detecting a defect area of the target area based on the second feature map; the surface defect detection method provided by the invention has both accuracy and precision.

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular, to a surface defect detection method and device based on spin adjacent attention. Background Art

[0002] In industrial production, defect detection usually relies on experienced workers for manual monitoring. However, small defects randomly appearing on the surface of an object (as shown by the square in Figure 1 ) are easily overlooked by workers. Deep learning-based methods can detect small defects from complex environments through feature extraction, but deep neural networks do not fully utilize the multiple global information of feature maps, so some tiny defects are still ignored.

[0003] At the same time, because there are many high similarities between the surface defect area and the normal area (as shown by the square in Figure 2 ), it is difficult for neural networks to distinguish these similarities, resulting in missed defect areas or false detections. Capturing more detailed features by modeling context dependencies can distinguish defect areas and normal areas, but most current attention-based methods tend to model context dependencies by calculating the pixel correlations of all dimensions of the feature map, with a large amount of calculation and poor practicability. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present invention is to provide a surface defect detection solution based on spin adjacent attention to solve the problem that the existing neural network-based surface defect detection solution has low detection accuracy because it is easy to ignore small defects and difficult to distinguish between normal areas and similar defect areas.

[0005] In a first aspect, an embodiment of the present invention provides a surface defect detection method based on spin adjacent attention, including: Obtaining image data of a surface target area and extracting a first feature map; Inputting the first feature map into a spin adjacent attention network to obtain a second feature map with spin adjacent attention, where the spin adjacent attention network at least includes a spin adaptive attention module and an adjacent routing attention module. The spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module models fine-grained labels between classes through a spatial adjacency matrix to distinguish between similar defect areas and defect-free areas; Detecting a defect area of the target area based on the second feature map.

[0006] Optionally, the obtaining image data of a surface target area and extracting a first feature map includes: Obtaining the image data of the surface target area; Perform group normalization on the image data and extract the first feature map based on the Sigmoid linear unit.

[0007] Optionally, the capturing multi-view spatial information from the first feature map to search for small defects on the surface includes: Convert the first feature map into third feature maps at a preset number of preset angles through spin transformation; Capture small target features from the third feature maps corresponding to each preset angle based on the adaptive attention mechanism.

[0008] Optionally, the preset angles include the directions of front, back, left, right, up, and down, and the converting the first feature map into third feature maps at a preset number of preset angles through spin transformation includes: Rotate the first feature map clockwise by 90°, 180°, and 270° along the z-axis to obtain the third feature maps in the right, back, and left directions respectively; and Rotate the first feature map counterclockwise by 90° and 270° along the x-axis to obtain the third feature maps in the up and down directions respectively.

[0009] Optionally, the capturing small target features from the third feature maps corresponding to each preset angle based on the adaptive attention mechanism includes: Perform 1×1 convolution on the third feature map to obtain a feature space, and generate an attention mask using the argmax function after a Boolean function based on the feature space; Multiply the third feature map by the attention mask to generate a fourth feature map; Sum the fourth feature maps corresponding to each preset angle element-wise to obtain a fifth feature map, and the fifth feature map is the feature map that enhances the small target features.

[0010] Optionally, after generating the fourth feature map from the third feature map, it further includes: Perform dimensional conversion on the third feature map with the corresponding third feature maps in the left, right, up, and down directions to ensure that the first feature map and the third feature maps at 6 preset angles have equal lengths in the three-dimensional coordinate system.

[0011] Optionally, the differentiating similar defect regions and defect-free regions by modeling fine-grained labels between classes through the spatial adjacency matrix includes: Divide the first feature map into non-overlapping feature map blocks of a preset size according to a preset rule; Generate an adjacent attention map for each feature map block based on the adjacent feature map blocks; Generate a sixth feature map based on the adjacent attention maps corresponding to each of the feature patches.

[0012] Optionally, generating an adjacent attention map for each of the feature patches based on the adjacent feature patches includes: Perform attention calculation on the feature patch and each adjacent feature patch to respectively obtain a first fine-grained attention map between the query and the adjacent key, and a second fine-grained attention map between the query and the adjacent value; Multiply the transposed first fine-grained attention map by the second fine-grained attention map and process it with the Max function to obtain the adjacent attention map generated by the feature patch.

[0013] Optionally, generating a sixth feature map based on the adjacent attention maps corresponding to each of the feature patches includes: Traverse each of the feature patches and obtain the adjacent attention map of each of the feature patches, splice all the adjacent attention maps together, and output the sixth feature map at the original position.

[0014] In a second aspect, an embodiment of the present invention provides a surface defect detection device based on spin adjacent attention, including: A feature extraction module, configured to obtain image data of a surface target area and extract a first feature map; A spin adjacent attention generation module, configured to input the first feature map into a spin adjacent attention network to obtain a second feature map with spin adjacent attention, where the spin adjacent attention network at least includes a spin adaptive attention module and an adjacent routing attention module, and the spin adaptive attention module is configured to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module models fine-grained labels between classes through a spatial adjacency matrix to distinguish between similar defect areas and defect-free areas; A detection module, configured to detect a defect area of the target area based on the second feature map.

[0015] The embodiments of the present invention have the following beneficial effects: The embodiments of the present invention develop a spin adaptive attention module to locate small defects on the material surface by learning multi-view spatial information. First, different variants are generated by rotating from six angles through rotation transformation, and then an adaptive attention mechanism is developed to enhance small target features through detail capture. Therefore, it avoids the problem that existing detection schemes based on deep neural networks do not fully utilize the multiple global information of feature maps and still ignore some tiny defects. At the same time, the embodiments of the present invention model fuzzy features based on the correlation between adjacent blocks, so as to distinguish similar defect regions from defect-free regions. Since only the dependence relationship between adjacent blocks is modeled instead of learning the global correlation between all pixels, the calculation amount is greatly reduced compared with existing neural network-based schemes, making it more practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is an example diagram of a small defect area on the surface; Figure 2 It is an example diagram of a defect area on the surface that is highly similar to the normal area; Figure 3 It is a schematic diagram of the architecture of the spin adjacent attention network proposed in some embodiments of the present invention; Figure 4 It is a flowchart of a surface defect detection method based on spin adjacent attention provided by the embodiments of the present invention; Figure 5 It is a schematic diagram of the spin adaptive attention module in some embodiments of the present invention; Figure 6 It is a schematic diagram of the adaptive attention in some embodiments of the present invention; Figure 7 It is a schematic diagram of the adjacent routing attention module in some embodiments of the present invention; Figure 8 It is a schematic diagram of a surface defect detection device based on spin adjacent attention shown in some embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] It should be noted that although functional module division is performed in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described may be executed in a different module division in the device or a different sequence in the flowchart. Terms such as "first" and "second" in the specification, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0021] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of this application. However, those skilled in the art will realize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0022] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware charging modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the drawings are only illustrative and do not necessarily include all the content and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0024] In industrial production, defect detection usually relies on experienced workers for manual monitoring. However, some defect areas are very small and randomly appear on the surface of the object, such as Figure 1As shown in the boxes in, these small defects are easily overlooked by workers, resulting in detection failures and significant losses. To address these issues, deep learning-based methods detect small defects from complex environments through feature extraction. Although performing well, deep neural networks still overlook some tiny defects because they do not fully utilize the multiple global information of feature maps. In addition, there are many high similarities between the surface defect regions and normal regions, such as Figure 2 shown in the boxes in. These similarities are largely difficult to distinguish by neural networks, leading to the omission or misdetection of defect regions. Therefore, it is crucial to capture more fine-grained features to distinguish normal regions from similar defect regions. Due to limited feature representation, it is a challenging task for ordinary convolutional neural networks (CNNs) to learn subtle features from similar regions, so an attention mechanism for capturing detailed features by modeling context dependencies is proposed. Among them, most attention-based methods tend to distinguish targets by calculating the pixel correlations of all dimensions of the feature map, which is very computationally resource-consuming.

[0025] Aiming at the above defects of the existing neural network-based surface defect detection schemes, an embodiment of the present invention proposes a spin adjacent attention network, which at least includes a spin adaptive attention module and an adjacent routing attention module. The spin adaptive attention module is used to capture multi-view spatial information to search for small defects on the surface, and the adjacent routing attention module models fine-grained labels between classes through a spatial adjacency matrix to distinguish similar defect regions and defect-free regions. In some embodiments of the present invention, the network structure of the spin adjacent attention network is as Figure 3 shown.

[0026] The present invention also proposes a spin adjacent attention-based surface defect detection method based on the spin adjacent attention network. As Figure 4 shown, the method includes the following steps: S410, obtaining image data of the surface target region and extracting a first feature map.

[0027] In some embodiments of the present invention, the image data of the surface target region is obtained, and then the image data is subjected to group normalization and a first feature map is extracted based on the Sigmoid linear unit.

[0028] S420, inputting the first feature map into the spin adjacent attention network to obtain a second feature map with spin adjacent attention, where the spin adjacent attention network at least includes a spin adaptive attention module and an adjacent routing attention module. The spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module models fine-grained labels between classes through a spatial adjacency matrix to distinguish similar defect regions and defect-free regions.

[0029] The present invention provides multi-view feature maps for a neural network based on multi-view learning to search for small targets from the global region by utilizing the multiple global information of the feature maps. Among them, the multi-view feature maps refer to six directions of feature mapping: up, down, left, right, front, and back. Extracting features based on the multi-view feature maps is similar to humans observing a three-dimensional figure from multiple angles, which can provide comprehensive multi-view spatial information of small defects for the neural network, helping to explore target features and understand the entire scene to adapt to different geometries.

[0030] Some embodiments of the present invention learn multi-view spatial information through a spin adaptive attention module to locate small defects on the surface of a material. Among them, the feature map is first rotated into different variants from six angles through a rotation transformation, and then an adaptive attention mechanism is developed to enhance the small target features through detail capture. The schematic diagram of the spin adaptive attention module is as Figure 5 shown and defined as: ; where X represents the input first feature map, SAA(X) represents the spin adaptive attention transformation of the first feature map, and SC, AA, and SP represent sum concatenation, adaptive attention, and spin respectively.

[0031] The description of the spin adaptive attention module is as follows: First, in order to enable the neural network to learn multi-view feature maps, the input feature map is converted into different variants at six angles through a spin (SP) transformation, that is, the third feature maps corresponding to the directions of front, back, left, right, up, and down respectively. Specifically, the original feature map is rotated clockwise by 90°, 180°, and 270° along the z-axis to obtain the third feature maps in the right, back, and left directions, defined as: ; ; and Left: ; In addition, the original feature map is rotated counterclockwise by 90° and 270° along the x-axis to obtain the third feature maps corresponding to the up and down directions (also known as the top and bottom directions), defined as: Top:

[0032] and Bottom:

[0033] Then, based on the adaptive attention (AA), the small target features are focused. The adaptive attention is as Figure 6 shown and defined as: ; Among them, AA(X) represents performing an adaptive attention transformation on the input first feature map X, and B, A, and F represent a Boolean function, an argmax function, and a 1×1 convolution respectively. The specific description is as follows: First, a feature space (F) is obtained by performing a 1×1 convolution on the input feature map (I). Second, an attention mask (G) is generated using argmax(·) after the Boolean function based on the feature space (F). Third, the attention mask (G) is multiplied with the feature map (I) to output a fourth feature map. Finally, the left, right, top, and bottom feature maps are permuted to ensure that these six feature maps have equal lengths in three dimensions (x, y, z), and then these feature maps are element-wise summed to obtain the final output fifth feature map.

[0034] From the visual features of the original image, the correlation between adjacent blocks is also the most similar. As Figure 7 shown in the box, the Q region has the same defect features as the right region, so they are highly similar. Inspired by this, the present invention designs an adjacency routing attention (ARA) module to distinguish similar defect regions from defect-free regions by modeling the blurred features. The adjacency routing attention module is as Figure 7 shown. Among them, an adjacency routing mechanism is designed to model the dependence relationship between adjacent blocks, rather than learning the correlation between all global pixels.

[0035] The expression of the adjacency routing attention module is: ; Among them, ARA(X) represents performing an adjacency routing attention transformation on the input first feature map, ARA represents adjacency routing attention, and CC, PT, ARK, and ARV represent concatenation, block division, adjacency routing between key (K) and query (Q), and adjacency routing between value (V) and query (Q) respectively.

[0036] The specific description is as follows: First, the input image is divided into smaller and non-overlapping blocks, and then they are used as separate elements for subsequent attention-based calculations. For example, the input image is segmented into 5×5 non-overlapping blocks of size [W / 5, H / 5], as Figure 7 shown.

[0037] Second, the fine-grained attention map between the query (Q) and the adjacent key (K) is modeled by the adjacency routing mechanism (ARK), which establishes the correlation between the image patch Q and its eight surrounding adjacent image patches. Similarly, the fine-grained attention map between the query (Q) and the adjacent value (V) is obtained from the adjacency routing mechanism. Specifically, the adjacency routing represents the attention calculation between the central image patch and its eight surrounding adjacent image patches.

[0038] Third, transpose the concatenated attention map QK, and then multiply it by the matrix of the concatenated attention map QV to obtain the adjacency attention feature map, which will be processed by Max(·) to finally output the sixth attention map.

[0039] Fourth, use the same operation to traverse all the feature patches to obtain the feature maps of all the feature patches, then stitch them together, and finally output the feature map according to the original positions.

[0040] S330, detect the defective area of the target area based on the second feature map.

[0041] One embodiment of the present invention tests Figure 4 the method (abbreviated as saa-net) described in S410 - S430 in

[0042] Comparison with advanced methods on the NEU-DET dataset: To verify the performance of SAA-Net, eight classic state-of-the-art methods, namely U-Net, AIS Net, CADN, DEA RetinaNet, PGA Net, MF-GAN, PAN, and PS-CNN, were tested on NEU-DET. The experimental results are shown in Table 1: Table 1: Quantitative results of different methods on NEU-DET

[0043] First, as can be seen from Table 1, the proposed SAA-Net shows the best defect detection performance on the NEUDET dataset, with an accuracy of 98.67%, a precision of 95.31%, a recall of 94.87%, an F1 score of 95.07%, and an IoU of 91.98%, demonstrating the effectiveness of SAA-Net for surface defect detection. In addition, compared with multi-scale feature methods such as PAN, PGA-Net, PS-CNN, DEA-RetinaNet, and AISNet, SAA-Net can achieve better performance than these methods, indicating that the proposed SAA-Net is more effective in multi-scale feature fusion. Moreover, the performance of the proposed SAA-Net is better than the MF-GAN method, with improvements of 1.65%, 2.39%, 5.18%, 3.79%, and 2.71% in terms of accuracy, precision, recall, F1 score, and IoU respectively, indicating that the spin adjacency strategy is more sensitive to small defect detection. In summary, these results show that saa-net can be used as an effective tool for surface defect detection.

[0044] Comparison with related methods on the MAGNETIC-TILE dataset: Similarly, these eight methods were carried out on the MAGNETIC-TILE database to verify the superiority of the proposed SAA-Net in surface defect detection, and the results are shown in Table 2.

[0045] Table 2: Quantitative results of different methods on the MAGNETIC-TILE dataset

[0046] As can be seen from Table 2, the proposed SAA-Net has better performance in surface defect detection on the MAGNETIC-TILE database than other methods, with an accuracy of 99.21%, a precision of 91.12%, a recall of 88.19%, an F1 score of 90.07%, and an IoU of 90.33%. Specifically, most methods are based on multi-scale information fusion and can learn features of different scales from the feature map. After introducing the attention mechanism into the network, the performance of defect detection can be further improved. For example, the performance of AISNet is better than multi-scale information fusion strategies such as PGANet, PS-CNN, and DEARetinaNet. However, it only adaptively captures multi-scale features during the learning stage and does not bring multi-view spatial information and adjacent ffne-grained tokens into the network, which may lead to the loss of ffne-grained features of small defects.

[0047] In contrast, the proposed SAA-Net adopts a multi-view spatial information fusion and adjacent fine-grained feature modeling strategy for surface defect detection. Compared with the AIS network, the SAA network can improve the accuracy. In the MAGNETICTILE database, the precision, recall, F1 score, and IoU value are improved by 0.48%, 3.08%, 2.87%, 3.41%, and 1.64% respectively. These excellent results verify the superiority of multi-view information fusion and adjacent fine-grained feature modeling in surface defect detection. In addition, the performance of the proposed SAA-Net is better than the MF-GAN method, with improvements of 2.57%, 4.79%, 5.03%, 5.35%, and 4.92% in terms of accuracy, precision, recall, F1 score, and IoU respectively, indicating that SAA-Net can effectively learn rich and discriminative fine-grained features for surface defect detection.

[0048] Figure 8 is a schematic diagram of a surface defect detection device based on spin adjacent attention according to some embodiments of the present invention. As Figure 8 shown, the surface defect detection device 800 based on spin adjacent attention includes a feature extraction module 810, a spin adjacent attention generation module 820, and a detection module 830. Among them: The feature extraction module 810 is configured to obtain image data of a surface target area and extract a first feature map; The spin adjacent attention generation module 820 is configured to input the first feature map into a spin adjacent attention network to obtain a second feature map with spin adjacent attention, where the spin adjacent attention network at least includes a spin adaptive attention module and an adjacent routing attention module. The spin adaptive attention module is configured to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module models fine-grained labels between classes through a spatial adjacency matrix to distinguish similar defect areas and defect-free areas; The detection module 830 is configured to detect a defect area of the target area based on the second feature map..

[0049] In summary, the surface defect detection method and device based on spin adjacency attention provided by the embodiments of the present invention locate small defects on the material surface by learning multi-view spatial information. First, different variants are generated by rotating from six angles through rotation transformation, and then an adaptive attention mechanism is developed to enhance small target features through detail capture. Therefore, it avoids the problem that the existing detection scheme based on deep neural network does not fully utilize the multiple global information of the feature map and still ignores some tiny defects. At the same time, the embodiments of the present invention model fuzzy features based on the correlation between adjacent blocks, so as to distinguish similar defect regions from defect-free regions. Since only the dependence relationship between adjacent blocks is modeled, rather than learning the global correlation between all pixels, the calculation amount is greatly reduced compared with the existing neural network-based scheme, and it is more practical.

[0050] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the charging modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0051] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional charging modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0052] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0053] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item) of the following" or its similar expressions refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0054] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0055] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0056] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0057] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media that can store programs, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0058] The preferred embodiments of the embodiments of this application have been described above with reference to the accompanying drawings, but this does not limit the scope of the rights of the embodiments of this application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of this application shall be within the scope of the rights of the embodiments of this application.

Claims

1. A surface defect detection method based on spin adjacency attention, characterized in that: include: Acquire image data of a surface target area and extract a first feature map; Inputting the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacent routing attention module, the spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module distinguishes similar defect areas from defect-free areas by modeling fine-grained labels between classes through a spatial adjacency matrix; A defect area of ​​the target area is detected based on the second feature map.

2. The method according to claim 1, characterized in that: The acquiring image data of the surface target area and extracting the first feature map comprises: Acquiring image data of the surface target area; The image data is group normalized and the first feature map is extracted based on a Sigmoid linear unit.

3. The method according to claim 1, characterized in that: The capturing of multi-view spatial information from the first feature map to search for small surface defects comprises: Converting the first characteristic map into a third characteristic map of a preset number of preset angles by spin transformation; Based on the adaptive attention mechanism, small target features are captured from the third feature map corresponding to each preset angle.

4. The method according to claim 3, characterized in that: The preset angles include the directions of front, back, left, right, top and bottom, and the converting of the first feature map into a third feature map of a preset number of preset angles by spin transformation includes: Rotating the first feature map along the z-axis clockwise by 90°, 180°, and 270° to obtain the third feature maps in the right, rear, and left directions, respectively; and The first feature map is rotated counterclockwise by 90° and 270° along the x-axis to obtain the third feature map in the upper and lower directions, respectively.

5. The method according to claim 3, characterized in that: The capturing of small target features from the third feature map corresponding to each preset angle based on the adaptive attention mechanism includes: Performing a 1×1 convolution on the third feature map to obtain a feature space, and generating an attention mask using an argmax function after a Boolean function based on the feature space; Multiplying the third feature map by the attention mask to generate a fourth feature map; The fourth feature map corresponding to each preset angle is element-summed to obtain a fifth feature map, and the fifth feature map is a feature map for strengthening the small target feature.

6. The method according to claim 5, characterized in that: After generating the fourth feature map from the third feature map, the method further includes: The third characteristic graph and the corresponding third characteristic graphs in the left, right, upper and lower directions are dimensionally transformed to ensure that the third characteristic graphs of the first characteristic graph at 6 preset angles have equal lengths in the three-dimensional coordinate system.

7. The method according to claim 1, characterized in that: The method of modeling fine-grained labels between classes by using a spatial adjacency matrix to distinguish similar defect areas from defect-free areas includes: Dividing the first feature map into feature map blocks of preset sizes and non-overlapping according to preset rules; Based on the adjacent feature blocks, generating an adjacent attention map for each feature block; Generate a sixth feature map based on the adjacent attention maps corresponding to each of the feature map blocks.

8. The method according to claim 7, characterized in that: The step of generating an adjacent attention map for each of the feature blocks based on the adjacent feature blocks includes: Performing attention calculation on the feature block and each adjacent feature block to obtain a first fine-grained attention map between the query and the adjacent keys, and a second fine-grained attention map between the query and the adjacent values; The transposed first fine-grained attention map is multiplied by the second fine-grained attention map and processed by the Max function to obtain a neighboring attention map generated by the feature block.

9. The method according to claim 7, characterized in that: Generating the sixth feature map based on the adjacent attention map corresponding to each of the feature map blocks includes: Traverse each of the feature blocks and obtain the adjacent attention maps of each of the feature blocks, splice all of the adjacent attention maps together, and output the sixth feature map at the original position.

10. A surface defect detection device based on spin adjacency attention, characterized in that: include: A feature extraction module, used to obtain image data of a surface target area and extract a first feature map; A spin neighbor attention generation module, used for inputting the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacent routing attention module, the spin adaptive attention module is used for capturing multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacent routing attention module distinguishes between similar defect areas and defect-free areas by modeling fine-grained labels between classes through a spatial adjacency matrix; A detection module is used to detect a defect area of ​​the target area based on the second feature map.

Images