Mamba enhanced industrial product surface defect detection method based on YOLOv11

Abstract

The invention discloses a Mama enhanced industrial product surface defect detection method based on YOLOv11, and belongs to the technical field of defect detection.The method comprises the steps that preprocessed industrial product surface image data are input into an industrial product surface defect detection model; according to industrial product surface image data, feature vectors are extracted through a plurality of C3k2 modules in sequence, and the feature vectors are input into an SPPF module to obtain the feature vectors; according to the SPPF module, obtaining a feature vector, splicing the feature vector with a feature vector in any C3k2 module so as to obtain a feature vector through the Mama module, splicing the feature vector obtained by the Mama module with any other feature vector so as to obtain a feature vector through the Mama module, and repeating the steps; and randomly selecting the feature vectors obtained by a plurality of Mamba modules, and inputting the feature vectors into a plurality of detection heads in a one-to-one correspondence manner for detecting surface defects. According to the method, both the detection precision and the reasoning efficiency are considered, and particularly, the detection rate and the positioning accuracy of surface defects of complex, tiny and long-strip-shaped industrial products are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, specifically relating to a method for detecting surface defects in industrial products based on YOLOv11 and Mamba enhancement. Background Technology

[0002] With the deepening of Industry 4.0 and intelligent manufacturing, quality control of industrial products is becoming increasingly critical in the production process. Among them, surface defects (such as scratches, dents, inclusions, etc.) are one of the core indicators affecting product quality. Traditional manual visual inspection methods suffer from problems such as low efficiency, strong subjectivity, high labor costs, and easy fatigue.

[0003] Therefore, automatic defect detection technology based on computer vision has gradually become a research hotspot in this field. In particular, single-stage target detection algorithms, represented by the YOLO series, are widely used in industrial surface defect detection tasks due to their excellent balance between detection speed and accuracy.

[0004] Models based on classic architectures such as YOLOv5 and YOLOv7 rely on the powerful local feature extraction capabilities of convolutional neural networks (CNNs) to effectively detect some defects with significant contrast and large size. However, the inherent local receptive field characteristics of CNNs make it difficult for them to capture long-range spatial dependencies between distant pixels in an image. When faced with small defects commonly found in industrial scenarios, such as those with complex textured backgrounds or weak contrast, these models are prone to both false positives and false negatives, and their edge localization of defects is often not precise enough.

[0005] To overcome the shortcomings of CNNs in global modeling, researchers began to introduce the Transformer module into the YOLO architecture. Through the Transformer's self-attention mechanism, the model can model global contextual information, thereby improving its ability to capture long-range dependencies and enhancing the detection of complex defects. However, the computational complexity of the self-attention mechanism is proportional to the square of the input image size. This results in enormous computational overhead when processing high-resolution industrial images, severely impacting real-time detection performance and making deployment difficult on resource-constrained industrial edge devices.

[0006] CNN-based models cannot effectively model long-range dependencies, while Transformer-based augmentation models sacrifice real-time performance due to high computational complexity. This contradiction makes it difficult for existing industrial surface defect detection solutions to balance detection accuracy, model efficiency, and inference speed, greatly limiting their large-scale application and deployment in real industrial production lines.

[0007] Therefore, there is an urgent need for a technical solution that can effectively model global context information while maintaining high detection efficiency, thereby enabling efficient, accurate, and real-time detection of surface defects in industrial products. Summary of the Invention

[0008] This invention provides a method for detecting surface defects in industrial products based on YOLOv11 and enhanced by Mamba. By innovatively introducing a Mamba module into the neck network of the model to replace the traditional CNN or Transformer structure, and designing a feature fusion path, it solves the contradiction between the insufficient long-range dependency modeling ability caused by the local receptive field limitation of CNN and the low efficiency of high-resolution image processing caused by the high computational complexity of the Transformer self-attention mechanism.

[0009] The technical solution adopted in this invention is as follows: A method for detecting surface defects in industrial products based on YOLOv11 and enhanced by Mamba includes: The preprocessed industrial product surface image data is input into the industrial product surface defect detection model, which includes a backbone network, a neck, and a head. The backbone network includes multiple C3k2 modules and one SPPF module. The neck includes multiple Mamba modules, and the head includes multiple detection heads. Based on the surface image data of the industrial product, feature vectors are extracted sequentially through multiple C3k2 modules and then input into the SPPF module to obtain feature vectors. The feature vector obtained from the SPPF module is concatenated with the feature vector from any of the C3k2 modules to obtain the feature vector through the Mamba module. The feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a feature vector through the Mamba module, and this process is repeated. Multiple feature vectors obtained from the Mamba modules can be arbitrarily selected and input into multiple detection heads in a one-to-one correspondence for detecting surface defects.

[0010] The Mamba-enhanced industrial product surface defect detection method based on YOLOv11 described in this invention also includes the following additional technical features: The C3k2 module extracts feature vectors as follows: The data input to the C3k2 module is expanded into a first feature part and a second feature part through a convolution channel; Based on the first feature portion, feature extraction is performed using the Bottleneck structure; The extracted features are concatenated with the second feature portion to extract the feature vector.

[0011] The feature vector is extracted sequentially through multiple C3k2 modules, specifically as follows: The surface image data of the industrial product is processed by the first C3k2 module to extract feature vectors. The feature vector obtained from the first C3k2 module is input into the second C3k2 module to extract the feature vector; The process is executed sequentially to extract feature vectors from the last C3k2 module, which are then used as input to the SPPF module.

[0012] The SPPF module obtains the feature vector as follows: Based on the feature vector of the input SPPF module, dimensionality reduction is performed through convolutional dimensionality reduction channels, followed by pooling operations through multiple pooling layers; The feature vector is obtained by concatenating the data obtained from multiple pooling layers.

[0013] The feature vector obtained from the SPPF module is concatenated with any feature vector from the C3k2 module, specifically as follows: The splicing module group is obtained by filtering according to the order of the multiple C3k2 modules, wherein the splicing module group does not include the first and last C3k2 modules; The feature vector obtained from the SPPF module is then concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group.

[0014] The feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a new feature vector through the Mamba module, specifically as follows: The feature vector obtained from the Mamba module is concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group; or... The feature vector obtained from the Mamba module is concatenated with the feature vector obtained from the SPPF module; or... The feature vector obtained by the Mamba module is concatenated with the feature vector obtained by another Mamba module.

[0015] Before inputting multiple detection heads in a one-to-one correspondence, the method further includes: Based on the concatenated feature vectors, the dimensions are adjusted through convolutional layers, and normalization and activation processes are performed. Based on the activated feature vector, it is input into the spatial information perception module for deep convolution and batch normalization processing, and then input into the intermediate convolutional layer with activation operation to further extract deep features. Based on the extracted deep features, the data is input into the 2D selective scanning module, which generates multiple series of sub-feature images along multiple different scanning directions. Based on the series, the sub-feature images are used to extract features through the corresponding S6 blocks and then merged to construct a feature map.

[0016] The number of times to repeat the execution is as follows: The number of times the operation is repeated is greater than or equal to the number of detection heads in the header.

[0017] The present invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method.

[0018] The present invention further provides a computer-readable storage medium having a computer program stored thereon. The computer program implements the method when executed by a processor.

[0019] Due to the adoption of the above technical solution, the beneficial effects achieved by this invention are as follows: 1. This invention introduces the Mamba module, the core of which lies in its 2D Selective Scanning (SS2D) mechanism. This mechanism can efficiently capture long-range spatial dependencies of feature maps in four directions—top-down, bottom-up, left-to-right, and right-to-left—with near-linear computational complexity. This allows the model to better understand the global structure and contextual information of the image, thus significantly reducing false negatives and improving localization accuracy for industrial defects such as complex backgrounds, low contrast, and irregular shapes (e.g., thin scratches, diffuse spots). Moreover, the near-linear computational complexity of the Mamba module overcomes the quadratic increase in computational burden with image resolution caused by the self-attention mechanism of Transformer-type models.

[0020] By embedding the Mamba module into the neck network, rather than simply replacing the entire backbone, the model can leverage Mamba for powerful global modeling while retaining the efficient local feature extraction capabilities of the YOLOv11 backbone. This allows the model to maintain high accuracy while ensuring inference efficiency, making it more suitable for real-time industrial detection scenarios.

[0021] Furthermore, through a dynamic, multi-layered feature fusion path, shallow features containing rich spatial details, mid-level features with moderate semantic information, and deep features rich in global semantics can be fused and enhanced multiple times and in a targeted manner using the Mamba module. This design ensures that the features input to the final detection head are strong representational features that have been fully understood by global context information and fused at multiple scales, thus enabling the model to maintain good detection performance for various defects with large size variations.

[0022] In summary, this invention balances detection accuracy and inference efficiency by deploying multiple Mamba modules in the neck network and constructing specific feature flow paths, and in particular significantly improves the detection rate and positioning accuracy of surface defects in complex, small, and elongated industrial products. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic flowchart of the YOLOv11-based Mamba-enhanced industrial product surface defect detection method according to one embodiment of the present invention. Figure 2 This is a schematic diagram of the industrial product surface defect detection model according to one embodiment of the present invention. Detailed Implementation

[0024] To more clearly illustrate the overall concept of the present invention, a detailed description will be provided below with reference to the accompanying drawings and examples.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0026] like Figure 1 and Figure 2 As shown, a method for detecting surface defects in industrial products based on YOLOv11 and enhanced by Mamba includes: S100: Input the preprocessed industrial product surface image data into the industrial product surface defect detection model, wherein the industrial product surface defect detection model includes a backbone network, a neck and a head, the backbone network includes multiple C3k2 modules and one SPPF module, the neck includes multiple Mamba modules, and the head includes multiple detection heads.

[0027] This step is the initialization and data input stage of the defect detection process. Its main purpose is to build and initialize a deep learning model designed specifically for industrial defect detection that integrates global perception capabilities, and to send the image data to be detected into the model for processing.

[0028] The industrial product surface defect detection model adopts a three-stage architecture: The backbone network is responsible for extracting multi-level features from the input image. It consists of multiple C3k2 modules and one SPPF module connected sequentially. The C3k2 modules, through their unique branching and concatenation structure, effectively extract and fuse local features while reducing computational cost; the SPPF module, through multi-scale pooling operations, further expands the receptive field of the feature map and captures richer contextual information.

[0029] Multiple Mamba modules are deployed in the neck region. Unlike existing technologies that use conventional CNN or Transformer modules in the neck region, the Mamba modules are based on a state-space model (SSM) and a selective scan mechanism (SS2D), enabling efficient modeling of long-range spatial dependencies of feature maps with near-linear computational complexity. This allows the model to understand the correlations between distant defect features in an image, such as the start and end points of a long scratch, and ensures high inference efficiency in real-time industrial inspection scenarios. While retaining YOLOv11's efficient local feature extraction capabilities, the model's global context modeling ability in the feature fusion stage is enhanced to balance local details and global correlations in industrial defect detection.

[0030] The head employs a multi-branch detection head, typically three, each responsible for predicting defects of different scales: small, medium, and large. Each detection head independently processes features from different levels in the neck that have been globally enhanced by the Mamba module, achieving decoupled output for classification and localization.

[0031] It should be noted that image data must be preprocessed before being input into the model to ensure that the data format meets the model's input requirements and to improve the model's robustness. Preprocessing operations include, but are not limited to: Image scaling and normalization scales the input image to a fixed size (e.g., 640x640 pixels) and normalizes the pixel values ​​to the [0,1] range to accelerate model convergence.

[0032] Data augmentation (during the training phase): To improve the model's generalization ability, online data augmentation techniques are used during training, including random rotation (±10°), random horizontal / vertical flipping, and random brightness and contrast adjustments, to simulate lighting changes, angle shifts, and other situations that may occur in industrial settings.

[0033] The preprocessed image data (usually in batches) is input into the pre-built and weighted industrial product surface defect detection model. The system first directs the data flow to the backbone network, initiating the entire feature extraction and defect identification process.

[0034] S200: Based on the surface image data of the industrial product, feature vectors are extracted sequentially through multiple C3k2 modules and then input into the SPPF module to obtain feature vectors.

[0035] The main purpose of this step is to perform multi-level, progressive local feature extraction on the input industrial product surface image and finally generate a strong semantic feature vector rich in multi-scale contextual information for subsequent global information fusion and enhancement of the neck network.

[0036] It should be noted that before the C3k2 module, the preprocessed image data first enters the initial part of the backbone network. This part typically consists of two consecutive convolutional layers with a stride of 2, padding of 1, and a kernel size of 3×3.

[0037] By using convolutions with a stride of 2, the height and width of the input image are rapidly reduced to half or even a quarter of their original size, quickly lowering the feature map resolution and significantly reducing subsequent computation. While reducing resolution, the number of channels in the feature map is increased (e.g., from 3 RGB channels to 64 or 128 feature channels) to accommodate richer feature information. Small convolutional kernels are used to capture the most basic local features in the image, such as edges and textures, achieving preliminary feature extraction.

[0038] After initial downsampling, the feature maps flow sequentially through multiple cascaded C3k2 modules. The core of each C3k2 module lies in efficient residual learning and feature reuse. Through the alternating stacking of multiple C3k2 modules (with possible staggered convolutions for further downsampling), the network can progressively extract rich features from low-level details to high-level semantics.

[0039] Through serialization processing using multiple C3k2 modules, the backbone network can gradually abstract local feature patterns (such as edges, corners, and textures) crucial for defect detection, starting from the pixel level, thus constructing a robust foundation of local features. Its internal segmentation and residual design effectively alleviate the vanishing gradient problem while deepening the network and ensuring the effective transmission of feature information.

[0040] The feature vector output from the last C3k2 module in the backbone network will be fed into the SPPF module. This module aims to efficiently aggregate multi-scale contextual information. The SPPF module enables the model to integrate contextual information from different regions of the image at deeper layers of the network, facilitating the understanding of the relationship between defects and the background and the judgment of the overall shape of large defects.

[0041] The feature vector output in this step contains both rich local details extracted by the C3k2 module and global context information aggregated by the SPPF module. This provides high-quality input for the Mamba module in the subsequent neck network to perform efficient long-range spatial dependency modeling.

[0042] S300: Obtain the feature vector from the SPPF module, and concatenate it with the feature vector from any of the C3k2 modules to obtain the feature vector through the Mamba module. The feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a feature vector through the Mamba module, and this process is repeated.

[0043] The main purpose of this step is to construct a dynamic, iterative, and multi-layered feature fusion and enhancement mechanism. This mechanism introduces the Mamba module to deeply fuse multi-scale features rich in local details extracted from the backbone network with deep semantic features with global context, thereby generating a series of enhanced features that combine local accuracy with global consistency.

[0044] The deep feature vectors output by the SPPF module are used as global context carriers. Simultaneously, a feature vector is selected from the outputs of multiple C3k2 modules in the backbone network, preserving relatively rich spatial details and intermediate-level semantic information.

[0045] The features are concatenated along the channel dimension to form the first fused feature. This fused feature is then fed into the first Mamba module for processing. The output of the Mamba module is an enhanced feature that incorporates robust global contextual information.

[0046] Iterative feature fusion and enhancement uses the enhanced features output by the previous Mamba module as a new and more powerful feature source, concatenates them with any other feature vector, and then inputs them back into the Mamba module.

[0047] Each concatenation and loop through the Mamba module is a re-fusion and enhancement of feature information, enabling global contextual information to permeate features at different scales and semantic levels.

[0048] This step effectively addresses the inherent limitations of traditional CNN models in long-range dependency modeling by repeatedly and directionally concatenating and processing the strong local features extracted by CNN with the strong global context modeled by Mamba. The generated series of features, each with a feature representation modulated by global image information, are used to determine whether a local region is a defect (especially defects strongly associated with the surrounding environment, such as thin scratches or regional pitting).

[0049] Through an iterative fusion strategy, feature maps at multiple enhancement stages are generated. Earlier outputs may focus more on the globalization of mid-level features, while later outputs fuse information at a richer scale. This provides the detection head with multi-granularity and multi-level globalization feature selection, enabling the model to more flexibly cope with defects of different sizes and shapes.

[0050] The near-linear computational complexity of the Mamba module allows this step to be repeated without causing an explosive increase in computational overhead. This enables the present invention to perform complex global interactions in the neck network of the model, while maintaining inference speed suitable for industrial real-time detection, thus balancing accuracy and efficiency.

[0051] Moreover, this dynamic multi-fusion mechanism can provide redundant contextual clues from global information. Even if local features are contaminated, the model can still make correct judgments based on the correlation information of other regions, making the model more robust to common industrial interference factors such as noise, changes in lighting, and partial occlusion.

[0052] S400: Arbitrarily select multiple feature vectors obtained from the Mamba modules and input them one-to-one into multiple detection heads for detecting surface defects.

[0053] The main purpose of this step is to efficiently and accurately convert the multi-scale features, which have been fused and globally enhanced in the previous steps, into specific defect detection results, including the defect category, location, and confidence level. This step achieves the final mapping from abstract features to specific detection boxes.

[0054] From the feature vectors output by multiple Mamba modules, a specific number of feature vectors are selected according to a predefined network structure. Typically, this number matches the number of detector heads; for example, three of the most representative feature vectors are selected.

[0055] The selected feature vectors should cover different spatial resolutions and semantic depths. For example: A Mamba output from the early stages of the fusion path has a high spatial resolution and contains more detailed information, making it suitable for detecting small defects (such as tiny scratches and spots). A Mamba output from the middle of the fusion path, which takes into account appropriate spatial details and semantic information, is suitable for detecting medium-sized defects; A Mamba output from the end of the fusion path has a lower spatial resolution but the richest semantic information and the largest receptive field, making it suitable for detecting large defects or defect regions with complex backgrounds.

[0056] The selected feature vectors are input into the corresponding detection heads in a fixed, one-to-one manner, in order from shallow to deep (i.e., from high resolution to low resolution). For example, the shallowest enhanced features are input into the head branch responsible for small target detection, and the deepest enhanced features are input into the head branch responsible for large target detection.

[0057] Each detection head, after receiving the corresponding feature vector enhanced by global context, employs a decoupled design to process the classification and localization tasks in parallel: The classification branch first uses depthwise separable convolutions to further process the input features. This operation significantly reduces the number of parameters and computational cost while maintaining the feature map size, which is beneficial for deploying the model on edge devices.

[0058] Then, the number of channels is adjusted to num_classes (the total number of defect categories) through a convolutional layer, and then the Softmax activation function is used to output the probability distribution of each spatial location belonging to each defect category.

[0059] Furthermore, the regression branch processes the input features using conventional convolutional layers. It employs the Distributed Focus Loss (DFL) mechanism to predict the bounding box coordinates. Instead of directly regressing the absolute coordinates of the box, DFL learns the probability distribution of the coordinate values, softening the one-hot encoding into a softmax distribution, which is particularly beneficial for improving the localization accuracy of blurred boundaries or small targets. This branch outputs the center point coordinates (x, y) of the bounding box, as well as its width and height (w, h).

[0060] Each detector head ultimately outputs a dense array of predictions. The system collects the outputs of all detector heads and then performs the following post-processing operations: Non-maximum suppression (NMS) iterates through all predicted bounding boxes, sorts them according to their classification confidence, and calculates the intersection-overall ratio (IoU) among all boxes. It suppresses (i.e., removes) redundant predicted bounding boxes whose IoU with higher-confidence predicted boxes exceeds a preset threshold (e.g., 0.5). This ensures that for the same defect, only the most reliable detection result is ultimately retained.

[0061] The results are formatted and filtered by NMS. The retained prediction boxes, their category labels, and confidence scores are assembled into a final list of detection results, which is then output to the user or downstream systems. These results can be directly used to trigger alarms, control robotic arms to sort defective products, or for visual annotation in the user interface.

[0062] By strategically assigning Mamba enhancement features at different levels to a dedicated detection head, an efficient multi-scale detection pipeline is constructed. High-resolution features ensure that small defects are not missed, while high-semantic features guarantee the accurate identification of large defects and defects in complex backgrounds, thereby significantly improving the model's overall recall and precision for defects of different sizes.

[0063] The features input to the detector head have undergone global context modulation by the Mamba module, resulting in more robust and discriminative feature representations. This makes the classification branch more accurate, while the regression branch, combined with the DFL mechanism, particularly optimizes the localization accuracy of the bounding boxes, especially addressing the edge blurring issue that traditional models struggle to handle.

[0064] In addition, each detection head only needs to focus on processing features of a specific scale, avoiding waste of resources and maintaining a high detection frame rate while ensuring accuracy, thus meeting the real-time requirements of industry.

[0065] In a preferred embodiment of the present invention, the C3k2 module extracts feature vectors as follows: The data input to the C3k2 module is expanded into a first feature part and a second feature part through a convolution channel; Based on the first feature portion, feature extraction is performed using the Bottleneck structure; The extracted features are concatenated with the second feature portion to extract the feature vector.

[0066] The C3k2 module is a component of the backbone network. Its main purpose is to provide high-quality multi-scale feature input by deepening feature extraction capabilities and retaining rich gradient information through an efficient feature segmentation and fusion strategy without excessively increasing computational complexity.

[0067] In the first two C3k2 modules of the backbone network, for the input features, a 1×1 convolution is first used to expand the channels, dividing the features into two parts. One part of the features is then extracted deeply using a Bottleneck structure and concatenated with the other part of the features along the channel dimension. This design significantly increases the effective depth of the network and improves the feature representation capability while cleverly controlling the growth of parameters and computational overhead, achieving a balance between performance and efficiency.

[0068] The main function of this 1×1 convolution is to expand and project the input features into a channel dimension, and its output channel count is usually greater than the input channel count C. This enhances the expressive power of the features and prepares the conditions for subsequent feature segmentation operations.

[0069] The segmentation strategy is not random or proportional, but strictly average, which ensures the balance of information carrying capacity of the two parts of features, and is used for subsequent differential processing and fusion.

[0070] The Bottleneck structure preferably employs the following sequence of operations: A 1×1 convolutional layer is used to reduce the number of channels and reduce computational cost; A 3×3 convolutional layer is used as the core convolution to effectively extract spatial features in a low-dimensional space. Another 1×1 convolutional layer is used to recover the number of channels, compared with the original input. The channel dimension is matched or adapted for subsequent splicing.

[0071] Throughout the Bottleneck structure, residual connections are typically introduced, which are the inputs of the Bottleneck... It is element-wise added to its output. This effectively alleviates the vanishing gradient problem in deep networks, stabilizes the training process, and helps the network learn identity mappings.

[0072] Specifically, for the input tensor ,in, H Indicates feature height, W Indicates the feature width. C The number of channels representing the features is used to split the tensor X into two parts after convolution. and , After passing through the Bottleneck structure, the output feature tensor is... , d This indicates the number of channels after passing through the Bottleneck structure; and After concatenation along the channel dimension, the number of channels in the output feature is adjusted using a 1×1 convolution, and the output feature is... , , This indicates the number of feature channels output by the C3k2 module.

[0073] Achieved The relatively shallow and detailed information retained Effective integration of the deep and abstract information contained therein.

[0074] The feature vector is extracted sequentially through multiple C3k2 modules, specifically as follows: The surface image data of the industrial product is processed by the first C3k2 module to extract feature vectors. The feature vector obtained from the first C3k2 module is input into the second C3k2 module to extract the feature vector; The process is executed sequentially to extract feature vectors from the last C3k2 module, which are then used as input to the SPPF module.

[0075] In addition, in the last two C3k2 modules of the backbone network, after channel expansion by a 1×1 convolution, the features are divided into two parts. One part of the features are extracted deeply by the C3k2 module and then concatenated with the other part of the features in the channel dimension.

[0076] Specifically, for the feature tensor input to the C3k2 module ,in, H Indicates feature height, W Indicates the feature width. C The number of channels representing the features is input into two 1×1 convolutional layers to obtain the results. and , and These represent the number of feature channels after convolution; the feature vectors are... The input is fed into a Bottleneck structure with large kernel grouped convolutions, and the resulting feature vector is then combined with... The data is concatenated, and a 1×1 convolutional layer is used to adjust the number of channels in the output features. The output features are... ,in , This represents the number of feature channels output by the C3k2 module.

[0077] It should be noted that the number of convolutional kernels corresponding to the Bottleneck structure increases according to the order of the multiple C3k2 modules.

[0078] The core of the Bottleneck structure in the last two C3k2 modules of the backbone network lies in replacing the traditional 3×3 convolution with grouped convolutions using large-size kernels (such as 5×5 or 7×7). The specific implementation is as follows: 1×1 convolution reduces the number of channels; Spatial feature extraction is performed using large kernel grouped convolutions. Grouped convolutions significantly reduce the number of parameters and computational cost introduced by the large convolution kernel; 1×1 convolution recovers the number of channels.

[0079] The advantage of large kernel convolutions lies in providing a larger receptive field, which is especially suitable for feature maps in deep networks that already have low resolution and need to capture more global feature information. It can integrate contextual information within a larger region, helping the model understand the macroscopic structure of defects and their relationship with the surrounding environment.

[0080] In a preferred embodiment of the present invention, the SPPF module obtains the feature vector as follows: Based on the feature vector of the input SPPF module, dimensionality reduction is performed through convolutional dimensionality reduction channels, followed by pooling operations through multiple pooling layers; The feature vector is obtained by concatenating the data obtained from multiple pooling layers.

[0081] The SPPF module is located at the end of the backbone network. Its main purpose is to efficiently aggregate multi-scale contextual information. By performing parallel multi-scale feature extraction and fusion on the input features, it significantly expands the receptive field of the feature map, thereby obtaining a strong representation feature vector rich in global semantic information.

[0082] eigenvectors The input is fed into the SPPF module, and after undergoing a 1×1 convolution to adjust the number of channels, the feature vector is obtained. The number of channels in the feature vector is reduced to half that of the input. Before entering the computationally intensive pooling operation, channel dimensionality reduction significantly reduces the number of feature channels, thereby effectively reducing the computational complexity and memory usage of all subsequent pooling branches and improving the overall efficiency of the module.

[0083] The input features are pooled multiple times using three pooling layers to obtain the output. , , ,in Mp This represents max pooling. All pooling layers use the same-sized square pooling kernel (e.g., 5×5), but through serial connection, it is equivalent to using progressively larger receptive fields (e.g., after three levels of 5×5 pooling, the equivalent receptive field is much larger than a single 5×5 pooling), greatly reducing computation and the number of parameters. With minimal computational overhead, it achieves multi-scale receptive field coverage similar to, or even more continuous than, using multiple large-sized parallel pooling kernels. This allows the network to effectively integrate local to global contextual information in images at deep layers, which is crucial for understanding the overall shape, size, and relationship of defects to their surroundings.

[0084] The features after each pooling are concatenated along the channel dimension and then passed through a 3×3 convolutional layer for output, resulting in a feature vector. ,in , This indicates the number of feature channels output by the SPPF module. The feature vector output by this module... It integrates spatial information at multiple scales, providing a broad field of view and rich semantic content. This information is then used in subsequent Mamba modules for long-range dependency modeling, establishing semantic associations between distant feature points in the image, thus enabling downstream detectors to adapt well to defects of different sizes.

[0085] In a preferred embodiment of the present invention, the feature vector obtained from the SPPF module is concatenated with the feature vector from any of the C3k2 modules, specifically as follows: The splicing module group is obtained by filtering according to the order of the multiple C3k2 modules, wherein the splicing module group does not include the first and last C3k2 modules; The feature vector obtained from the SPPF module is then concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group.

[0086] The feature vectors obtained from the SPPF module are concatenated with the feature vectors from the C3k2 module. This is the initialization step for feature fusion in the neck network. Its main purpose is to establish the first effective connection between local and global features in the backbone network. Through feature selection strategies, it ensures that the features participating in the first fusion are balanced in terms of semantic level and spatial details, resulting in high-quality initial fused features.

[0087] First, based on the order of multiple C3k2 modules in the backbone network, all C3k2 modules are sorted and filtered. The C3k2 module at the first and last position in the sequence is excluded, and the remaining intermediate layer C3k2 modules are used to form a candidate splicing module group.

[0088] For example, if the backbone network contains four C3k2 modules (referred to as C3k2_1, C3k2_2, C3k2_3, and C3k2_4 in sequence), then the selected splicing module group will include C3k2_2 and C3k2_3.

[0089] The feature vector output by the SPPF module (rich in global context information) is concatenated with the feature vector output by any C3k2 module (from an intermediate layer) selected from the concatenation module group. The concatenation operation is performed along the channel dimension to form the first fused feature, which will serve as the input to the first subsequent Mamba module.

[0090] By excluding the first and last C3k2 modules, this strategy cleverly avoids two extremes: Although the features of the shallowest (first) C3k2 module are rich in spatial details, their semantic level is too low. If they are directly concatenated with the SPPF features rich in high-level semantics, the semantic gap is too large, the fusion effect is poor and the training is unstable. The features of the deepest (last) C3k2 module are semantically similar to those of SPPF. However, because it is adjacent to the SPPF module, the feature information extracted by the two may have a large degree of redundancy. Direct splicing has limited value and cannot introduce new, complementary intermediate layer information.

[0091] Choosing the C3k2 module in the middle layer preserves relatively rich spatial details and has a certain semantic level, which is more compatible with the semantic level of SPPF features, enabling more effective and balanced fusion.

[0092] SPPF features are stronger than global context, but their spatial details are relatively coarse; the intermediate layer C3k2 features are stronger than local details and medium-scale semantics. Concatenating the two essentially combines global "environment" information with local "target" details in a powerful complementary way, providing a high-quality input basis containing multi-level information for the Mamba module to perform accurate long-range dependency modeling.

[0093] As one embodiment of this implementation, the feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a feature vector through the Mamba module, specifically as follows: The feature vector obtained from the Mamba module is concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group; or... The feature vector obtained from the Mamba module is concatenated with the feature vector obtained from the SPPF module; or... The feature vector obtained by the Mamba module is concatenated with the feature vector obtained by another Mamba module.

[0094] The purpose of iteratively concatenating the feature vector obtained by the Mamba module with any other feature vector is to provide a flexible and powerful feature enhancement mechanism. Through the iterative fusion of multi-path and multi-source information, the features processed by the Mamba module that already contain global context information can continuously interact and complement features from other sources, thereby generating multi-level enhanced features with strong expressive power.

[0095] Path 1: Concatenate the feature vector output by the current Mamba module with the feature vector output by any C3k2 module in the concatenation module group, and then input the concatenation result into the next Mamba module.

[0096] This approach aims to combine the features modulated by global context with the rich spatial details and moderate semantic features retained in the intermediate layers of the backbone network that were not initially fused. This is equivalent to supplementing the features with a global perspective with more precise local location information or semantic information from different intermediate levels, which helps to further refine the feature representation, especially improving the detection accuracy of small targets and defects with high edge sharpness requirements.

[0097] Path 2: Concatenate the feature vector output by the current Mamba module with the original feature vector obtained by the SPPF module, and then input the concatenation result into the next Mamba module.

[0098] This approach implements a kind of global context information injection or enhancement. Although it already contains global information, the original feature vector obtained by the SPPF module, as the most primitive global feature source without complex Mamba transformations, may have different information distribution and emphasis. Combining the two can prevent the global information decay that may occur during subsequent Mamba processing and provide another perspective on the global context, making the final enhanced features more robust and capable of understanding complex scenarios and large-scale defects.

[0099] Path 3: Concatenate the feature vector output by the current Mamba module with the feature vector output by another Mamba module, and then input the concatenation result into the next Mamba module.

[0100] This approach enables direct interaction between global augmentation features at different stages or fusion levels. For example, it concatenates Mamba outputs from earlier stages with those from later stages. Different stages may emphasize global dependencies at different granularities or in different regions. This approach allows for cross-dialogue and information complementarity between these high-level, globalized semantic features, enabling the fusion of multi-stage global understanding and potentially capturing more complex and abstract defect patterns and associations.

[0101] Each concatenation path specifically addresses different problems that may be encountered during feature evolution: the detail enhancement path supplements spatial information, the global semantic enhancement path consolidates the context, and the high-level semantic interaction path promotes the fusion of abstract information. This multi-target enhancement strategy ensures that the final feature map used for detection is a high-quality feature that has been refined comprehensively and in multiple rounds.

[0102] Specifically, such as Figure 2 As shown, the feature vector output by the C2PSA module is upsampled and then concatenated with the feature vector output by the third C2K3 module in the channel dimension. The concatenated feature vector is then input into the Mamba module.

[0103] The feature vector output by the Mamba module is upsampled and then concatenated with the feature vector output by the second C2K3 module along the channel dimension. The concatenated feature vector is then input into the Mamba module.

[0104] The feature vector output by the Mamba module is concatenated with the feature vector output by the second Mamba module along the channel dimension, and the concatenated feature vector is then input into the Mamba module.

[0105] The feature vector output by the Mamba module is concatenated with the feature vector output by the first Mamba module along the channel dimension, and the concatenated feature vector is then input into the Mamba module.

[0106] In a preferred embodiment of the present invention, before inputting the plurality of detection heads in a one-to-one correspondence, the method further includes: Based on the concatenated feature vectors, the dimensions are adjusted through convolutional layers, and normalization and activation processes are performed. Based on the activated feature vector, it is input into the spatial information perception module for deep convolution and batch normalization processing, and then input into the intermediate convolutional layer with activation operation to further extract deep features. Based on the extracted deep features, the data is input into the 2D selective scanning module, which generates multiple series of sub-feature images along multiple different scanning directions. Based on the series, the sub-feature images are used to extract features through the corresponding S6 blocks and then merged to construct a feature map.

[0107] Before inputting the feature vectors one-to-one into multiple detection heads, the processing of the spliced ​​feature vectors is mainly aimed at performing in-depth local feature enhancement and global dependency modeling on the fused features. Through a series of carefully designed sub-modules, the fused features containing multi-source information are transformed into strong representational features that have both local accuracy and global consistency, so as to achieve high-precision defect detection.

[0108] For the feature vector of the input Mamba module The feature vector is obtained after adjusting the channels of the feature vector through a 1×1 convolutional layer, and then undergoing normalization and activation processing. ,in, d This indicates the number of channels in the adjusted feature vector, which is then input into the spatial information perception module to enhance the capture of local features.

[0109] The feature vectors of the input spatial information perception module are first subjected to 3×3 depthwise convolution and batch normalization, i.e. ,in This represents the input feature vector. This indicates a depthwise convolution operation with a kernel size of 3×3. BN ( ) indicates batch normalization. This step extracts local spatial features through depthwise convolution while reducing the number of parameters.

[0110] The output feature vector The input is fed into two 1×1 convolutional layers with activation operations in between to further extract deeper features, i.e. , in This represents the input features from the previous layer, the first one. This indicates a 1×1 convolution operation used to adjust the number of channels in the input features. Gelu Indicates activation operation, the second one. A 1×1 convolution operation is used to adjust the number of channels of a feature to the size of the input; ⊕ indicates that the output feature is multiplied by the input feature. The splicing is performed along the channel dimension. This represents the characteristics output by the spatial information perception module.

[0111] The output features of the spatial information sensing module are processed by layer normalization and then input into the 2D selective scanning module. , This represents the features output from the spatial information perception module. The representation layer is normalized, and SS2D represents the 2D selective scanning module.

[0112] The 2D selective scanning module mainly consists of three algorithm steps: scan expansion, S6 block feature extraction, and scan merging.

[0113] During the scan expansion phase, the input feature sequence is traversed along four different scan directions: top-down, bottom-up, left-to-right, and right-to-left, forming four different series of sub-feature images. Each series of sub-images is processed independently by different S6 blocks. The processed results are then merged to construct a two-dimensional feature map as the final output. For the four different series of sub-feature images representing the input, This represents block S6, which processes images of different series of sub-features. This indicates a cross-merge operation.

[0114] By enhancing local feature capture through a spatial information perception module and modeling global dependencies through a 2D selective scanning module, the output features retain detailed information and possess context awareness, significantly improving the ability to represent complex defects.

[0115] The combination of depthwise convolution and selective scanning mechanisms maintains high accuracy while controlling computational complexity, ensuring a balance between computational efficiency and accuracy, and meeting the needs of real-time industrial detection.

[0116] In a preferred embodiment of the present invention, the number of times the operation is repeated is specifically as follows: The number of times the operation is repeated is greater than or equal to the number of detection heads in the header.

[0117] The main objective of this implementation is to ensure that the model can provide sufficiently optimized feature representations for each scale of detection task by quantizing and controlling the depth of feature enhancement, thereby achieving the optimal match between detection resources and feature quality.

[0118] In a typical embodiment of the present invention, M detection heads are usually set, specifically 3, which are responsible for detecting defects in small, medium and large targets respectively.

[0119] The stitching process in the neck network—via the Mamba module—should be repeated at least three times. For example, it can be set to N times, specifically three, four, or more, but the minimum must equal the number of detection heads.

[0120] The model is capable of generating feature maps for N different enhancement stages. When assigning features to the detection head, the most representative M features can be selected from these N enhancement features, such as the features with the deepest enhancement and the richest semantic information, and then sent to the detection head at different scales.

[0121] This principle fundamentally ensures that the model provides at least one specially enhanced feature map as input for each detection head (corresponding to a scale range). These features have gradient differences in semantic level and globalization, with higher information density and broader semantic coverage, thus ensuring that defects at all scales can be supported by high-quality features.

[0122] The present invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method.

[0123] Therefore, it can achieve any effect of the Mamba-enhanced industrial product surface defect detection method based on YOLOv11, which will not be elaborated here.

[0124] The present invention further provides a computer-readable storage medium having a computer program stored thereon. The computer program implements the method when executed by a processor.

[0125] Therefore, it can achieve any effect of the Mamba-enhanced industrial product surface defect detection method based on YOLOv11, which will not be elaborated here.

[0126] For any parts not mentioned in this invention, existing technologies can be used or referenced.

[0127] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0128] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for detecting surface defects in industrial products based on YOLOv11 and enhanced by Mamba, characterized in that, include: The preprocessed industrial product surface image data is input into the industrial product surface defect detection model, which includes a backbone network, a neck, and a head. The backbone network includes multiple C3k2 modules and one SPPF module. The neck includes multiple Mamba modules, and the head includes multiple detection heads. Based on the surface image data of the industrial product, feature vectors are extracted sequentially through multiple C3k2 modules and then input into the SPPF module to obtain feature vectors. The feature vector obtained from the SPPF module is concatenated with the feature vector from any of the C3k2 modules to obtain the feature vector through the Mamba module. The feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a feature vector through the Mamba module, and this process is repeated. Multiple feature vectors obtained from the Mamba modules can be arbitrarily selected and input into multiple detection heads in a one-to-one correspondence for detecting surface defects.

2. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 1, characterized in that, The C3k2 module extracts feature vectors as follows: The data input to the C3k2 module is expanded into a first feature part and a second feature part through a convolution channel; Based on the first feature portion, feature extraction is performed using the Bottleneck structure; The extracted features are concatenated with the second feature portion to extract the feature vector.

3. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 2, characterized in that, The feature vector is extracted sequentially through multiple C3k2 modules, specifically as follows: The surface image data of the industrial product is processed by the first C3k2 module to extract feature vectors. The feature vector obtained from the first C3k2 module is input into the second C3k2 module to extract the feature vector; The process is executed sequentially to extract feature vectors from the last C3k2 module, which are then used as input to the SPPF module.

4. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 1, characterized in that, The SPPF module obtains the feature vector as follows: Based on the feature vector of the input SPPF module, dimensionality reduction is performed through convolutional dimensionality reduction channels, followed by pooling operations through multiple pooling layers; The feature vector is obtained by concatenating the data obtained from multiple pooling layers.

5. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 1, characterized in that, The feature vector obtained from the SPPF module is concatenated with any feature vector from the C3k2 module, specifically as follows: The splicing module group is obtained by filtering according to the order of the multiple C3k2 modules, wherein the splicing module group does not include the first and last C3k2 modules; The feature vector obtained from the SPPF module is then concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group.

6. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 5, characterized in that, The feature vector obtained by the Mamba module is concatenated with any other feature vector to obtain a new feature vector through the Mamba module, specifically as follows: The feature vector obtained from the Mamba module is concatenated with the feature vector of any one of the C3k2 modules in the concatenation module group; or... The feature vector obtained from the Mamba module is concatenated with the feature vector obtained from the SPPF module; or... The feature vector obtained by the Mamba module is concatenated with the feature vector obtained by another Mamba module.

7. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 1, characterized in that, Before inputting multiple detection heads in a one-to-one correspondence, the method further includes: Based on the concatenated feature vectors, the dimensions are adjusted through convolutional layers, and normalization and activation processes are performed. Based on the activated feature vector, it is input into the spatial information perception module for deep convolution and batch normalization processing, and then input into the intermediate convolutional layer with activation operation to further extract deep features. Based on the extracted deep features, the data is input into the 2D selective scanning module, which generates multiple series of sub-feature images along multiple different scanning directions. Based on the series, the sub-feature images are used to extract features through the corresponding S6 blocks and then merged to construct a feature map.

8. The method for detecting surface defects in industrial products based on YOLOv11 using Mamba enhancement as described in claim 1, characterized in that, The number of times to repeat the execution is as follows: The number of times the operation is repeated is greater than or equal to the number of detection heads in the header.

9. An electronic device, characterized in that, Including memory and processor, The memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It contains computer programs. When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images