A surface defect detection method and device based on spin adjacency attention

Abstract

The present invention relates to the technical field of surface defect detection, and in particular to a surface defect detection method and device based on spin neighbor attention. The method comprises: acquiring image data of a surface target area and extracting a first feature map; inputting the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network comprises at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish similar defect areas from defect-free areas; detecting defect areas in the target area based on the second feature map; the surface defect detection method provided by the present invention has both accuracy and precision.

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular to a surface defect detection method and device based on spin adjacency attention. Background Art

[0002] In industrial production, defect detection usually relies on manual monitoring by experienced workers. However, small defects that randomly appear on the surface of objects (such as Figure 1 The small defects (shown as boxes in the figure) can be easily overlooked by workers. Deep learning-based methods can detect small defects in complex environments through feature extraction, but deep neural networks do not fully utilize the multiple global information of feature maps and thus still overlook some tiny defects.

[0003] At the same time, because there are many high similarities between surface defect areas and normal areas (e.g. Figure 2 Neural networks struggle to distinguish these similarities, leading to missed defect areas or incorrect detections. Capturing more detailed features by modeling contextual dependencies can distinguish defective and normal areas. However, most current attention-based methods tend to model contextual dependencies by calculating pixel correlations across all dimensions of the feature map, which is computationally intensive and impractical. Summary of the Invention

[0004] In view of this, the purpose of an embodiment of the present invention is to provide a surface defect detection solution based on spin adjacency attention to solve the problem that the existing surface defect detection solution based on neural network has low detection accuracy because it easily ignores small defects and has difficulty in distinguishing normal areas from similar defect areas.

[0005] In a first aspect, an embodiment of the present invention provides a surface defect detection method based on spin adjacency attention, comprising:

[0006] Acquiring image data of a surface target area and extracting a first feature map;

[0007] Inputting the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish similar defect areas from defect-free areas;

[0008] A defect area of ​​the target area is detected based on the second feature map.

[0009] Optionally, acquiring image data of the surface target area and extracting the first feature map includes:

[0010] acquiring image data of the surface target area;

[0011] Group normalization is performed on the image data and the first feature map is extracted based on a Sigmoid linear unit.

[0012] Optionally, capturing multi-view spatial information from the first feature map to search for small surface defects includes:

[0013] Converting the first characteristic map into a third characteristic map of a preset number of preset angles by spin transformation;

[0014] Based on the adaptive attention mechanism, small target features are captured from the third feature map corresponding to each preset angle.

[0015] Optionally, the preset angles include front, back, left, right, top, and bottom directions, and converting the first feature map into a third feature map having a preset number of preset angles by spin transformation includes:

[0016] Rotating the first feature map by 90°, 180°, and 270° clockwise along the z-axis to obtain the third feature maps in the right, rear, and left directions, respectively; and

[0017] The first feature map is rotated counterclockwise by 90° and 270° along the x-axis to obtain the third feature maps in the upper and lower directions, respectively.

[0018] Optionally, capturing small target features from the third feature map corresponding to each preset angle based on the adaptive attention mechanism includes:

[0019] Performing a 1×1 convolution on the third feature map to obtain a feature space, and generating an attention mask using an argmax function after a Boolean function based on the feature space;

[0020] Multiplying the third feature map by the attention mask to generate a fourth feature map;

[0021] The fourth feature map corresponding to each preset angle is element-summed to obtain a fifth feature map, which is a feature map for strengthening small target features.

[0022] Optionally, after generating the fourth feature map from the third feature map, the method further includes:

[0023] The third characteristic graph is dimensionally transformed with the corresponding third characteristic graphs in the left, right, up, and down directions to ensure that the third characteristic graphs of the first characteristic graph at the six preset angles have equal lengths in a three-dimensional coordinate system.

[0024] Optionally, the modeling of fine-grained labels between classes using a spatial adjacency matrix to distinguish similar defect areas from non-defect areas includes:

[0025] Dividing the first feature map into feature blocks of preset size and non-overlapping according to preset rules;

[0026] Based on the adjacent feature blocks, generating an adjacent attention map for each feature block;

[0027] A sixth feature map is generated based on the adjacent attention maps corresponding to each of the feature map blocks.

[0028] Optionally, generating an adjacent attention map for each feature block based on the adjacent feature blocks includes:

[0029] Performing attention calculation on the feature block and each adjacent feature block to obtain a first fine-grained attention map between the query and the adjacent keys, and a second fine-grained attention map between the query and the adjacent values;

[0030] The transposed first fine-grained attention map is multiplied by the second fine-grained attention map and processed with the Max function to obtain a neighboring attention map generated by the feature block.

[0031] Optionally, generating a sixth feature map based on the adjacent attention map corresponding to each feature map block includes:

[0032] Traverse each of the feature blocks and obtain the adjacent attention maps of each of the feature blocks, splice all of the adjacent attention maps together, and output the sixth feature map at the original position.

[0033] In a second aspect, an embodiment of the present invention provides a surface defect detection device based on spin adjacency attention, comprising:

[0034] a feature extraction module, configured to acquire image data of a surface target area and extract a first feature map;

[0035] a spin neighbor attention generation module, configured to input the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is configured to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish between similar defect areas and defect-free areas;

[0036] A detection module is used to detect a defect area in the target area based on the second feature map.

[0037] The embodiments of the present invention include the following beneficial effects: the embodiments of the present invention develop a spin-adaptive attention module, which locates small defects on the surface of the material by learning multi-view spatial information. First, different variants are generated from six angles through rotation transformation, and then an adaptive attention mechanism is developed to enhance small target features by capturing details, thereby avoiding the problem that the existing detection scheme based on deep neural networks does not fully utilize the multiple global information of the feature map and still ignores some tiny defects; at the same time, the embodiments of the present invention model fuzzy features based on the correlation between adjacent blocks, thereby distinguishing similar defective areas from non-defective areas. Because only the dependency relationship between adjacent blocks is modeled instead of learning the global correlation between all pixels, the amount of computation is greatly reduced compared to the existing neural network-based scheme, and it is more practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is an example diagram of a small surface defect area;

[0040] Figure 2 This is an example image of a defective area with a surface that is highly similar to a normal area;

[0041] Figure 3 Schematic diagram of the architecture of the spin-adjacent attention network proposed in some embodiments of the present invention;

[0042] Figure 4 is a flow chart of a surface defect detection method based on spin adjacency attention provided by an embodiment of the present invention;

[0043] Figure 5 is a schematic diagram of a spin-adaptive attention module in some embodiments of the present invention;

[0044] Figure 6 is a schematic diagram of adaptive attention in some embodiments of the present invention;

[0045] Figure 7 is a schematic diagram of an adjacency routing attention module in some embodiments of the present invention;

[0046] Figure 8is a schematic diagram of a surface defect detection device based on spin adjacency attention according to some embodiments of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0050] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0051] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. In other words, these functional entities may be implemented in software, in one or more hardware charging modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0052] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0053] In industrial production, defect detection usually relies on manual monitoring by experienced workers. However, some defect areas are small and appear randomly on the surface of objects, such as Figure 1 As shown in the boxes in the figure, these small defects can easily be overlooked by workers, leading to detection failures and significant losses. To address these issues, deep learning-based methods detect small defects in complex environments through feature extraction. Despite their good performance, deep neural networks still overlook some tiny defects because they do not fully utilize the multiple global information of feature maps. In addition, there are many high similarities between surface defect areas and normal areas, such as Figure 2 These similarities are largely indistinguishable to neural networks, leading to missed or incorrect detection of defective regions. Therefore, capturing more fine-grained features to distinguish normal regions from similar defective regions is crucial. Due to limited feature representation, learning subtle features from similar regions is a difficult task for ordinary convolutional neural networks (CNNs). Therefore, attention mechanisms have been proposed to capture detailed features by modeling contextual dependencies. Most attention-based methods tend to distinguish objects by computing pixel correlations across all dimensions of the feature map, which is computationally expensive.

[0054] In response to the above-mentioned shortcomings of existing neural network-based surface defect detection solutions, embodiments of the present invention propose a spin adjacency attention network, which includes at least a spin adaptive attention module and an adjacency routing attention module. The spin adaptive attention module is used to capture multi-view spatial information to search for small surface defects, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish similar defect areas from non-defective areas. In some embodiments of the present invention, the network structure of the spin adjacency attention network is as follows: Figure 3 shown.

[0055] The present invention also proposes a surface defect detection method based on spin neighbor attention network. Figure 4 As shown, the method includes the following steps:

[0056] S410, acquiring image data of a surface target area and extracting a first feature map.

[0057] In some embodiments of the present invention, image data of a surface target area is acquired, group normalization is performed on the image data, and a first feature map is extracted based on a Sigmoid linear unit.

[0058] S420, input the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish similar defect areas from defect-free areas.

[0059] This invention uses multi-view learning to provide a neural network with a multi-view feature map. This map leverages the multiple global information in the feature map to search for small objects in the global region. The multi-view feature map refers to the six directions of feature mapping: up, down, left, right, front, and back. Extracting features based on the multi-view feature map is similar to how humans observe a three-dimensional figure from multiple angles. This provides the neural network with comprehensive multi-view spatial information about small defects, helping it explore target features and understand the entire scene to adapt to different geometries.

[0060] Some embodiments of the present invention learn multi-view spatial information through a spin-adaptive attention module to locate small defects on the surface of a material, where the feature map is first rotated into different variants from six angles through a rotation transformation, and then an adaptive attention mechanism is developed to enhance the features of small targets by capturing details. The schematic diagram of the spin-adaptive attention module is shown in Figure 2. Figure 5 As shown, it is defined as:

[0061] ;

[0062] Where X represents the first feature map of the input, SAA(X) represents the spin-adaptive attention transformation on the first feature map, SC, AA, and SP represent summation cascade, adaptive attention, and spin, respectively.

[0063] The description of the spin-adaptive attention module is as follows:

[0064] First, in order to enable the neural network to learn multi-view feature maps, the input feature map is converted into six different angle variations through spin (SP) transformation, namely the third feature maps corresponding to the front, back, left, right, top and bottom directions respectively. Specifically, the original feature map is rotated 90°, 180° and 270° clockwise along the z-axis to obtain the third feature maps in the right, back and left directions, which are defined as:

[0065] ;

[0066] ;

[0067] and

[0068] Left: ;

[0069] In addition, the original feature map is rotated 90° and 270° counterclockwise along the x-axis to obtain a third feature map corresponding to the up and down directions (also known as the top and bottom directions), which is defined as:

[0070] top:

[0071] and

[0072] end:

[0073] Then, we focus on small target features based on adaptive attention (AA). Figure 6 As shown, it is defined as:

[0074] ;

[0075] Among them, AA(X) represents the adaptive attention transformation of the first feature map X of the input, B, A and F represent Boolean function, argmax function and 1×1 convolution respectively.

[0076] The specific instructions are as follows:

[0077] First, the feature space (F) is obtained by performing a 1×1 convolution on the input feature map (I). Second, the attention mask (G) is generated after the Boolean function based on the feature space (F) using argmax (·). Thirdly, the attention mask (G) is multiplied with the feature map (I) to output the fourth feature map. Finally, the left, right, top, and bottom feature maps are permuted to ensure that the six feature maps have equal lengths in three dimensions (x, y, z), and then these feature maps are element-wise summed to obtain the final output fifth feature map.

[0078] From the perspective of the visual features of the original image, the correlation between adjacent blocks is also the most similar. Figure 7 As shown in the box, region Q has the same defect features as the region on the right, so they are highly similar. Inspired by this, the present invention designs an adjacency routing attention (ARA) module to distinguish similar defect regions from non-defective regions by modeling fuzzy features. Figure 7 As shown in Figure 2, an adjacency routing mechanism is designed to model the dependencies between neighboring patches instead of learning the correlations between all global pixels.

[0079] The expression of the adjacency routing attention module is:

[0080] ;

[0081] Among them, ARA(X) represents the adjacency routing attention transformation on the first feature map of the input, ARA represents adjacency routing attention, CC, PT, ARK and ARV represent concatenation, chunking, adjacency routing between key (K) and query (Q), and adjacency routing between value (V) and query (Q), respectively.

[0082] The specific instructions are as follows:

[0083] First, the input image is divided into smaller and non-overlapping blocks, which are then treated as separate elements for subsequent attention-based computations. For example, the input image is split into 5×5 non-overlapping blocks of size [W / 5, H / 5], as Figure 7 shown.

[0084] Second, the fine-grained attention map between the query (Q) and the neighboring keys (K) is modeled by the adjacency routing mechanism (ARK), which establishes the correlation between the image patch Q and its eight surrounding neighboring image patches. Similarly, the fine-grained attention map between the query (Q) and the neighboring values ​​(V) is obtained from the adjacency routing mechanism. Specifically, adjacency routing represents the attention calculation between the central image patch and its eight surrounding neighboring image patches.

[0085] Third, the cascaded attention map QK is transposed and then multiplied by the matrix of the cascaded attention map QV to obtain the adjacent attention feature map, which will be processed by Max(·) to finally output the sixth attention map.

[0086] Fourth, use the same operation to traverse all feature blocks to obtain the feature maps of all feature blocks, then splice them together, and then output the final feature map based on the original position.

[0087] S330: Detect a defect area in the target area based on the second feature map.

[0088] One embodiment of the present invention is Figure 4 The method described in S410-S430 (abbreviated as saa-net) was tested, and the evaluation indicators included accuracy, precision, recall, F1-score, IoU and mAP.

[0089] Comparison with advanced methods on the NEU-DET dataset:

[0090] In order to verify the performance of SAA-Net, eight classic state-of-the-art methods, namely U-Net, AIS Net, CADN, DEA RetinaNet, PGA Net, MF-GAN, PAN and PS-CNN, were tested on NEU-DET. Experimental results Table 1:

[0091] Table 1: Quantitative results of different methods in NEU-DET

[0092]

[0093] First, as can be seen in Table 1, the proposed SAA-Net demonstrates the best defect detection performance on the NEUDET dataset, with an accuracy of 98.67%, precision of 95.31%, recall of 94.87%, F1 score of 95.07%, and IoU of 91.98%, demonstrating the effectiveness of SAA-Net for surface defect detection. Furthermore, compared to multi-scale feature methods such as PAN, PGA-Net, PS-CNN, DEA-RetinaNet, and AISNet, SAA-Net achieves superior performance, demonstrating that the proposed SAA-Net is more effective at fusing multi-scale features. Furthermore, the proposed SAA-Net outperforms the MF-GAN method, achieving improvements of 1.65%, 2.39%, 5.18%, 3.79%, and 2.71% in accuracy, precision, recall, F1 score, and IoU, respectively, demonstrating that the spin-neighbor strategy is more sensitive to small defect detection. In summary, these results demonstrate that SAA-Net can be used as an effective tool for surface defect detection.

[0094] Comparison with related methods on the MAGNETIC-TILE dataset:

[0095] Likewise, these eight methods are conducted on the MAGNETIC-TILE database to verify the superiority of the proposed SAA-Net in surface defect detection, and the results are shown in Table 2.

[0096] Table 2: Quantitative results of different methods on the MAGNETIC-TILE dataset

[0097]

[0098] As shown in Table 2, the proposed SAA-Net outperforms other methods in surface defect detection on the MAGNETIC-TILE database, achieving 99.21% accuracy, 91.12% precision, 88.19% recall, 90.07% F1 score, and 90.33% IoU. Specifically, most methods are based on multi-scale information fusion, learning features at different scales from feature maps. Introducing an attention mechanism into the network can further improve defect detection performance. For example, AISNet outperforms multi-scale information fusion strategies such as PGANet, PS-CNN, and DEARetinaNet. However, it only adaptively captures multi-scale features during the learning phase, without incorporating multi-view spatial information and adjacent FFNE granularity tokens into the network, which may result in the loss of FFNE fine-grained features for small defects.

[0099] In contrast, the proposed SAA-Net employs a multi-view spatial information fusion and adjacency fine-grained feature modeling strategy for surface defect detection. Compared to the AIS network, the SAA network achieves improved accuracy. On the MAGNETICTILE database, precision, recall, F1 score, and IoU value are improved by 0.48%, 3.08%, 2.87%, 3.41%, and 1.64%, respectively. These outstanding results validate the superiority of multi-view information fusion and adjacency fine-grained feature modeling for surface defect detection. Furthermore, the proposed SAA-Net outperforms the MF-GAN method, achieving improvements of 2.57%, 4.79%, 5.03%, 5.35%, and 4.92% in accuracy, precision, recall, F1 score, and IoU, respectively, demonstrating that the SAA-Net can effectively learn rich and discriminative fine-grained features for surface defect detection.

[0100] Figure 8 FIG. 1 is a schematic diagram of a surface defect detection device based on spin adjacency attention according to some embodiments of the present invention. Figure 8 As shown, the surface defect detection device 800 based on spin adjacent attention includes a feature extraction module 810, a spin adjacent attention generation module 820, and a detection module 830.

[0101] A feature extraction module 810 is configured to obtain image data of a surface target area and extract a first feature map;

[0102] A spin neighbor attention generation module 820 is configured to input the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, wherein the spin adaptive attention module is configured to capture multi-view spatial information from the first feature map to search for small surface defects, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish between similar defect regions and defect-free regions;

[0103] The detection module 830 is configured to detect a defective area in the target area based on the second feature map.

[0104] In summary, the surface defect detection method and device based on spin adjacency attention provided by the embodiments of the present invention locate small defects on the surface of the material by learning multi-view spatial information. First, different variants are generated from six angles through rotation transformation, and then an adaptive attention mechanism is developed to enhance the features of small targets by capturing details. Therefore, the problem that the existing detection scheme based on deep neural networks does not fully utilize the multiple global information of the feature map and still ignores some minor defects is avoided. At the same time, the embodiment of the present invention models fuzzy features based on the correlation between adjacent blocks, thereby distinguishing similar defective areas from non-defective areas. Because only the dependency relationship between adjacent blocks is modeled instead of learning the global correlation between all pixels, the amount of computation is greatly reduced compared with the existing neural network-based scheme, and it is more practical.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one location or distributed across multiple network units. Some or all of the charging modules may be selected to achieve the objectives of this embodiment based on actual needs.

[0106] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional charging modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0107] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0108] It should be understood that in this application, "at least one (item)" means one or more, and "more" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store programs, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0113] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A surface defect detection method based on spin adjacency attention, characterized in that: include: Acquiring image data of a surface target area and extracting a first feature map; Inputting the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is used to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish similar defect areas from defect-free areas; detecting a defect area in the target area based on the second feature map; The capturing of multi-view spatial information from the first feature map to search for small surface defects comprises: Converting the first characteristic map into a third characteristic map of a preset number of preset angles by spin transformation; Capturing small target features from the third feature map corresponding to each preset angle based on an adaptive attention mechanism; The method of distinguishing similar defect areas from non-defect areas by modeling fine-grained labels between classes through a spatial adjacency matrix includes: Dividing the first feature map into feature blocks of preset size and non-overlapping according to preset rules; Based on the adjacent feature blocks, generating an adjacent attention map for each feature block; A sixth feature map is generated based on the adjacent attention maps corresponding to each of the feature map blocks.

2. The method according to claim 1, characterized in that The acquiring image data of the surface target area and extracting the first feature map comprises: acquiring image data of the surface target area; Group normalization is performed on the image data and the first feature map is extracted based on a Sigmoid linear unit.

3. The method according to claim 1, characterized in that The preset angles include front, back, left, right, up, and down directions, and the converting of the first feature map into a third feature map having a preset number of preset angles by spin transformation includes: Rotating the first feature map by 90°, 180°, and 270° clockwise along the z-axis to obtain the third feature maps in the right, rear, and left directions, respectively; and The first feature map is rotated counterclockwise by 90° and 270° along the x-axis to obtain the third feature maps in the upper and lower directions, respectively.

4. The method according to claim 1, characterized in that Capturing small target features from the third feature map corresponding to each preset angle based on the adaptive attention mechanism includes: Performing a 1×1 convolution on the third feature map to obtain a feature space, and generating an attention mask using an argmax function after a Boolean function based on the feature space; Multiplying the third feature map by the attention mask to generate a fourth feature map; The fourth feature map corresponding to each preset angle is element-summed to obtain a fifth feature map, which is a feature map for strengthening small target features.

5. The method according to claim 4, characterized in that: After generating the fourth feature map from the third feature map, the method further includes: The third characteristic graph is dimensionally transformed with the corresponding third characteristic graphs in the left, right, up, and down directions to ensure that the third characteristic graphs of the first characteristic graph at the six preset angles have equal lengths in a three-dimensional coordinate system.

6. The method according to claim 1, characterized in that Generating an adjacent attention map for each feature block based on the adjacent feature blocks includes: Performing attention calculation on the feature block and each adjacent feature block to obtain a first fine-grained attention map between the query and the adjacent keys, and a second fine-grained attention map between the query and the adjacent values; The transposed first fine-grained attention map is multiplied by the second fine-grained attention map and processed with the Max function to obtain a neighboring attention map generated by the feature block.

7. The method according to claim 1, characterized in that: Generating the sixth feature map based on the adjacent attention map corresponding to each feature map block includes: Traverse each of the feature blocks and obtain the adjacent attention maps of each of the feature blocks, splice all of the adjacent attention maps together, and output the sixth feature map at the original position.

8. A surface defect detection device based on spin adjacency attention, characterized in that: include: a feature extraction module, configured to acquire image data of a surface target area and extract a first feature map; a spin neighbor attention generation module, configured to input the first feature map into a spin neighbor attention network to obtain a second feature map with spin neighbor attention, wherein the spin neighbor attention network includes at least a spin adaptive attention module and an adjacency routing attention module, the spin adaptive attention module is configured to capture multi-view spatial information from the first feature map to search for small defects on the surface, and the adjacency routing attention module uses a spatial adjacency matrix to model fine-grained labels between classes to distinguish between similar defect areas and defect-free areas; a detection module, configured to detect a defective area in the target area based on the second feature map; The capturing of multi-view spatial information from the first feature map to search for small surface defects comprises: Converting the first characteristic map into a third characteristic map of a preset number of preset angles by spin transformation; Capturing small target features from the third feature map corresponding to each preset angle based on an adaptive attention mechanism; The method of distinguishing similar defect areas from non-defect areas by modeling fine-grained labels between classes through a spatial adjacency matrix includes: Dividing the first feature map into feature blocks of preset size and non-overlapping according to preset rules; Based on the adjacent feature blocks, generating an adjacent attention map for each feature block; A sixth feature map is generated based on the adjacent attention maps corresponding to each of the feature map blocks.

Images