Multi-scale attention-based few-sample leather defect real-time detection system and method

Abstract

The invention discloses a few-sample leather defect real-time detection system and method based on multi-scale attention, and belongs to the technical field of image processing and artificial intelligence. The system comprises an image acquisition module, a preprocessing module, a deep learning detection module, a multi-scale feature fusion module, a prototype classification and positioning module, a post-processing module and an accelerated reasoning module. The method comprises the following steps: acquiring a leather image through an industrial camera, preprocessing the leather image, inputting the leather image into a double-branch network, respectively carrying out ResCBAM attention enhancement and SCConv feature reconstruction, fusing multi-scale features through a cross-scale attention mechanism, carrying out few-sample classification and CIoU accurate positioning by using a ProtoNet prototype network, and finally carrying out post-processing and TensorRT acceleration to realize real-time output. According to the method, the problems of leather texture interference, multi-scale defect leak detection and few-sample training are effectively solved, the real-time requirement of an industrial production line is met while high detection precision is ensured, and the automation and intelligence level of leather quality detection is remarkably improved.

Description

Technical Field

[0001] This invention relates to a real-time detection system and method for few-sample leather defects based on multi-scale attention, belonging to the fields of image processing and artificial intelligence technology, and is particularly suitable for real-time detection of few-sample leather defects based on multi-scale attention. Background Technology

[0002] Leather, a crucial material widely used in industries such as clothing, footwear, furniture, and automobile manufacturing, directly impacts the appearance, performance, and commercial value of products due to its surface quality. However, during production and processing, defects such as black spots, white spots, discoloration, cracks, and seams often appear on the leather surface, severely affecting product grade and market competitiveness. Therefore, efficient and accurate defect detection technology is of paramount importance in leather manufacturing quality control.

[0003] Currently, the industry still primarily relies on manual visual inspection for identifying surface defects in leather. While experienced inspectors possess certain identification abilities, this method has significant drawbacks: high labor intensity, low efficiency, subjective and inconsistent standards, and susceptibility to missed detections and misjudgments due to fatigue and differences in experience. This inspection method not only limits the level of production automation but also struggles to meet the demands of large-scale, highly consistent quality inspection. With the development of computer vision and deep learning technologies, convolutional neural networks (CNNs) have achieved significant results in surface defect detection tasks in fields such as steel, electronics, and textiles. Although CNNs perform excellently in general object detection, leather defect detection often requires massive amounts of labeled data. However, in actual production, the number of samples for certain specific defects is extremely small (the few-sample problem), making it impossible to effectively train conventional models. The following are the main technical challenges in leather defect detection: real-world leather defect samples are scarce and class imbalanced, making it difficult to train deep learning models and resulting in poor generalization ability; defects vary greatly in size in images, and many tiny defects (such as pinholes and dents) occupy only a very small area of ​​the image and are often missed; the natural texture of leather itself is highly irregular, and some defects (pinholes and scratches) are similar to the background in shape, color, and texture, which can easily lead to false detection.

[0004] Therefore, there is an urgent need for a detection method that can accurately identify multi-scale defects and effectively suppress background interference even under sparse sample conditions. This method should be able to significantly improve the accuracy and robustness of defect identification while maintaining real-time performance, so as to meet the growing demand for automated and intelligent quality inspection in the leather manufacturing industry. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a real-time detection system and method for leather defects based on multi-scale attention. It aims to solve the problems of difficulty in taking into account multi-scale defects, low recognition rate of small-sample defects, and serious interference from complex texture backgrounds in existing leather surface defect detection technologies. At the same time, it overcomes the shortcomings of traditional deep learning models, such as high inference latency and difficulty in meeting the real-time requirements of industrial production lines.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A real-time few-sample leather defect detection system based on multi-scale attention, combined with Figure 1 Its features include an image acquisition module, a preprocessing module, a deep learning detection module, a multi-scale feature fusion module, a prototype classification and localization module, a post-processing module, and an accelerated inference and deployment module; specifically,

[0008] The image acquisition module is responsible for acquiring images of the leather surface in real time. It obtains high-resolution raw images through an industrial line scan camera and transmits them to the preprocessing module. This module supports image stream input and is suitable for continuous production line environments.

[0009] After receiving the acquired leather surface image, the preprocessing module performs operations such as global contrast enhancement, smoothing and noise reduction, and local feature enhancement, and finally outputs an image with uniform size and standardized format; it is directly connected to the output end of the image acquisition module and to the input end of the deep learning detection module.

[0010] The deep learning detection module is connected to the output of the preprocessing module and the input of the multi-scale feature fusion module. It includes a parallel dual-branch backbone network, which adopts the ResCBAM (Residual Convolutional Block Attention Module) attention mechanism and the Spatial and Channel Reconstruction Convolution (SCConv) respectively. The features of the two branches are concatenated at the end to extract highly identifiable defect features, which are then output to the multi-scale feature fusion module.

[0011] The multi-scale feature fusion module is connected to the output of the deep learning detection module and the input of the prototype classification and localization module; it receives feature maps from the deep learning detection module and achieves adaptive fusion of low-resolution semantic features and high-resolution detail features through a cross-scale attention mechanism; the module outputs the fused multi-scale feature map and transmits it to the prototype classification and localization module.

[0012] The prototype classification and localization module is connected to the output of the multi-scale feature fusion module and the input of the post-processing module. Based on the ProtoNet prototype network, the distance between the input features and various defect prototypes is calculated to achieve high-precision classification under conditions of few samples. At the same time, the CIoU (Complete Intersection over Union) loss function is used to perform bounding box regression and output the defect category, confidence and location information to the post-processing module.

[0013] The post-processing module performs confidence filtering, non-maximum suppression, coordinate restoration, and result formatting on the detection results, and finally outputs the detection image with labeled boxes and structured detection data. This module is directly connected to the output of the prototype classification and localization module and provides output to the user interface or control system.

[0014] The accelerated inference and deployment module enables the system to operate efficiently in real time in an industrial environment through model quantization, memory optimization, and concurrent inference technologies. This module supports the real-time operation of the entire system and works in conjunction with the preprocessing module and the deep learning detection module.

[0015] Furthermore, the ResCBAM attention mechanism is specifically composed of channel attention and spatial attention connected in series through residual connections. It focuses on the defect region and suppresses the interference of background texture, thereby enhancing the feature response.

[0016] Furthermore, the working principle of the channel attention in the ResCBAM attention mechanism is as follows:

[0017] ;

[0018] in, This is a channel attention map, whose dimension is the same as the number of channels in the input feature map; The input feature map for the module. and They are respectively for The global average pooling and global max pooling operations are used to aggregate spatial information; they are used to aggregate spatial information of feature maps from different perspectives. A shared multilayer perceptron is used to learn channel weights from two pooling results; The Sigmoid activation function is used to normalize the weight values ​​to 0. between.

[0019] Furthermore, the spatial attention mechanism of the ResCBAM attention mechanism works as follows:

[0020] ;

[0021] in Spatial attention map; Channel attention map; for Convolution operation of convolution kernel; This indicates a channel splicing operation.

[0022] Furthermore, the residual connections of the ResCBAM attention mechanism are as follows:

[0023] ;

[0024] in, For hyperparameters, For element-wise multiplication, For attention-enhanced feature maps.

[0025] Furthermore, the SCConv channel reconstruction convolution specifically employs grouped convolution, spatial reconstruction, and channel reconstruction sequentially to separate and reconstruct spatial and channel information in the features, eliminate redundancy in high-level feature representations, optimize feature transmission, and thereby extract more distinctive and compact feature representations.

[0026] SCConv consists of two units: SRU (Spatial Reconstruction Unit): reduces spatial redundancy through a split-reconstruction operation and uses a gating mechanism to filter important features; CRU (Channel Reconstruction Unit): reduces channel redundancy and optimizes feature representation using a segmentation-transformation-fusion strategy. Specifically, the input feature map... First, the feature map is refined using the SRU space, then refined using the CRU channel, and finally output with redundancy eliminated. .

[0027] A real-time detection method for few-sample leather defects based on multi-scale attention, combined with Figure 4 It includes the following steps:

[0028] S1: The image acquisition module acquires images of the leather surface in real time, and the preprocessing module preprocesses the leather surface images and outputs standardized and enhanced images;

[0029] S2: Extract leather defect features through a parallel dual-branch backbone network of the deep learning detection module, and output a feature map that integrates attention enhancement and redundancy elimination;

[0030] S3: The multi-scale feature fusion module downsamples and fuses feature information at different scales in the feature map to obtain a multi-scale feature map;

[0031] S4: The prototype classification and localization module is based on the ProtoNet prototype network and combines CIoU loss to calculate the distance between multi-scale feature maps and various defect prototypes, thereby realizing defect classification of leather surface images.

[0032] S5: The post-processing module performs quality control and formatting on the defect classification results, and outputs the final inspection results;

[0033] S6: The accelerated inference and deployment module achieves real-time inference for steps S1 to S6 through engineering optimization, ensuring application in industrial production lines.

[0034] Furthermore, step S3 specifically includes:

[0035] S301: Assemble the attention-enhanced feature map Feature maps for redundancy elimination Perform downsampling twice to obtain high-resolution feature maps that retain their original size. Low-resolution feature map after 2 downsampling ;

[0036] S302: Employs a cross-scale attention (CSA) mechanism for feature fusion to obtain multi-scale feature maps. ;in, This is an upsampling operation used to enable... Size and Size alignment; The fusion weights for adaptive learning have values ​​located at... Between these, contributions from low-resolution and high-resolution feature maps are dynamically balanced based on feature content to achieve optimal cross-scale information integration.

[0037] Furthermore, step S4 specifically includes:

[0038] S401: Build a ProtoNet prototype network and for each defect category prototype Calculations were performed to obtain ;in, For category The number of samples, For the first Categories Sample images, For feature extraction functions;

[0039] S402: For any eigenvector in the multi-scale feature map Calculate its relationship with the defect category prototype Distance metric ;

[0040] S403: Transform distance metrics into defect classification probabilities for leather surface images using the Softmax function:

[0041] ;

[0042] in, Represents the input feature vector Belongs to the defect category The probability of; This represents the total number of defect categories;

[0043] S404: The CIoU loss function is used as the bounding box regression loss to locate the bounding boxes corresponding to the defect categories, and outputs the category, confidence score and bounding box coordinates of each detected defect.

[0044] Furthermore, step S5 specifically includes:

[0045] S501: Set a confidence threshold to filter out bounding boxes with a confidence level below the threshold;

[0046] S502: The non-maximum suppression (NMS) algorithm is used to eliminate redundant boxes to ensure that each defective region corresponds to only one bounding box;

[0047] S503: Converts the bounding box coordinates of the sub-image detection to the original large image coordinates and filters out the category detection results of defects with too small an area.

[0048] Furthermore, to ensure the real-time performance and engineering feasibility of the method of this invention, step S6 specifically includes: converting the trained model to ONNX format and building a TensorRT inference engine; enabling FP16 half-precision quantization mode to reduce memory usage and accelerate computation; in a C++ deployment environment, using cudaMallocHost to allocate page lock memory to accelerate data transmission between the host and the device; simultaneously, employing a std::async asynchronous multi-threaded concurrency strategy to divide the high-resolution leather image into multiple subgraphs for parallel inference, and using a mutex lock mechanism to ensure thread safety during result merging, thereby significantly reducing inference latency.

[0049] An electronic device includes at least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0050] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the aforementioned multi-scale attention-based real-time detection method for few-sample leather defects.

[0051] Finally, the present invention also discloses a computer-readable storage medium storing computer instructions for causing a processor to execute the above-described method for real-time detection of few-sample leather defects based on multi-scale attention.

[0052] The beneficial effects of this invention are as follows: It provides a real-time detection system and method for leather defects based on multi-scale attention with few samples, including the following beneficial effects:

[0053] Strong anti-interference ability: By introducing the ResCBAM attention mechanism and SCConv channel reconstruction convolution, the interference of natural textures on the leather surface (such as pores and growth lines) on defect detection is effectively solved, and the purity of feature extraction is significantly improved.

[0054] Good multi-scale adaptability: By utilizing the CSA cross-scale attention mechanism, adaptive feature fusion is achieved, which effectively solves the problem of feature disappearance of small defects (such as pinholes) in deep networks and improves multi-scale detection performance.

[0055] High robustness with few samples: Based on the ProtoNet prototype network strategy, even when defective samples are scarce, a robust prototype space can still be built through metric learning, achieving high-precision classification and recognition, and reducing the dependence on massive labeled data.

[0056] Superior real-time performance: By combining the TensorRT engine, FP16 quantization, and C++ concurrency technology, the algorithm achieves engineering-level acceleration, ensuring real-time operation under high-resolution image streams in industrial production lines and effectively improving the automation level of leather production. Attached Figure Description

[0057] To make the objectives and technical solutions of this invention clearer, the following figures are provided for illustration:

[0058] Figure 1 This is an architecture diagram of the real-time detection system for few-sample leather defects based on multi-scale attention in this invention; where the solid line represents the data flow direction and the dashed line represents the collaborative accelerated inference connection.

[0059] Figure 2 This is a diagram of the ResCBAM attention mechanism architecture in Embodiment 1 of the present invention;

[0060] Figure 3 This is a diagram of the SCConv channel reconstruction convolutional architecture in Embodiment 1 of the present invention;

[0061] Figure 4 This is a flowchart of the real-time detection method for few-sample leather defects based on multi-scale attention in this invention;

[0062] Figure 5 This is a grayscale schematic diagram of the defects in the leather portion in Embodiment 2 of the present invention;

[0063] Figure 6 This is a schematic diagram of the grayscale results of the leather black spot defect system monitoring in Embodiment 2 of the present invention;

[0064] Figure 7 This is a schematic diagram of the electronic device in Embodiment 3 of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0066] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0067] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] Example 1: In actual production, leather surfaces often exhibit various defects such as black spots, white spots, pinholes, scratches, and discoloration. Furthermore, some defect samples are extremely rare (e.g., certain rare discolorations or special scratches). Traditional manual inspection or conventional deep learning models struggle to meet the requirements for high-precision and high-efficiency quality inspection. To achieve high-speed, automated surface defect detection of leather products (such as shoe uppers, handbags, and automotive interiors) on the production line, this invention provides a "real-time detection system for few-sample leather defects based on multi-scale attention."

[0070] Combination Figure 1 The system includes an image acquisition module (1), a preprocessing module (2), a deep learning detection module (3), a multi-scale feature fusion module (4), a prototype classification and localization module (5), a post-processing module (6), and an accelerated inference and deployment module (7).

[0071] The image acquisition module (1) uses an industrial line scan camera (such as DALSA Piranha4) to acquire images of the leather surface in real time during production line operation. It supports a resolution of up to 8192×2048 pixels and a frame rate of no less than 200fps, and outputs RAW format image streams to the preprocessing module (2).

[0072] After receiving the acquired leather surface image, the preprocessing module (2) performs operations such as global contrast enhancement, smoothing and noise reduction, and local feature enhancement, and finally outputs an image with uniform size and standardized format; it is directly connected to the output end of the image acquisition module (1) and to the input end of the deep learning detection module (3). The specific preprocessing process is as follows:

[0073] 1) Uniform size: Scale or crop all input images to a fixed size of 2048×2048 pixels;

[0074] 2) Global contrast enhancement: Histogram equalization is used to improve overall contrast;

[0075] 3) Gaussian filtering for noise reduction: A Gaussian kernel with σ=1.5 is used for smoothing.

[0076] 4) Local feature enhancement: Edge enhancement is performed using the Sobel operator, and the CLAHE algorithm (Clip Limit=3.0, Tile Grid Size=8×8) is applied to enhance local texture contrast;

[0077] 5) Image formatting: Convert the image to Tensor format and normalize it (normalize pixel values ​​to the [0,1] range).

[0078] The deep learning detection module (3) is connected to the output of the preprocessing module (2) and the input of the multi-scale feature fusion module (4); it includes a parallel dual-branch backbone network:

[0079] 1) First branch: Combination Figure 2 Based on the ResCBAM attention mechanism, channel attention and spatial attention weighting are performed sequentially to enhance the feature response of defect areas and suppress background texture interference;

[0080] 2) Second branch: Combination Figure 3 Based on the SCConv feature reconstruction module, feature redundancy is eliminated through grouped convolution, spatial reconstruction unit (SRU) and channel reconstruction unit (CRU) to extract compact features;

[0081] 3) After the outputs of the two branches are concatenated in the channel dimension, they are fused by 1×1 convolution to output the defect enhancement feature map to the multi-scale feature fusion module (4).

[0082] The multi-scale feature fusion module (4) is connected to the output of the deep learning detection module (3) and the input of the prototype classification and localization module (5); it receives feature maps from the deep learning detection module (3) and achieves adaptive fusion of low-resolution semantic features and high-resolution detail features through a cross-scale attention mechanism; the module outputs the fused multi-scale feature map and transmits it to the prototype classification and localization module (5).

[0083] Specifically, the input feature map is downsampled twice by convolution with a stride of 2 to obtain three feature maps with different resolutions (2048×2048, 1024×1024, and 512×512), and adaptive fusion is performed using a cross-scale attention mechanism (CSA).

[0084] The prototype classification and localization module (5) is connected to the output end of the multi-scale feature fusion module (4) and the input end of the post-processing module (6). Based on the ProtoNet prototype network, the distance between the input features and various defect prototypes is calculated to achieve high-precision classification under the condition of few samples. At the same time, the CIoU (Complete Intersection over Union) loss function is used to perform bounding box regression and output the defect category, confidence and location information to the post-processing module.

[0085] Specifically, this module achieves high-precision defect classification and localization under conditions of limited samples based on the ProtoNet prototype network. It constructs a learnable defect prototype representation space, calculates the Euclidean distance between the input feature vector and each prototype, and uses the Softmax function to output a classification probability feature map. Simultaneously, it employs the CIoU loss function for bounding box regression, comprehensively considering overlap area, center point distance, and aspect ratio differences. Finally, it outputs the defect category, confidence score, and precise bounding box coordinates for each detection location, achieving robust recognition and localization in scenarios with scarce samples.

[0086] The post-processing module (6) performs confidence filtering, non-maximum suppression, coordinate restoration and result formatting on the detection results, and finally outputs the detection image with labeled boxes and structured detection data; this module is directly connected to the output end of the prototype classification and positioning module (5) and provides output for the user interface or control system.

[0087] Specifically, this module is responsible for quality control and standardization of the raw detection results. By setting a confidence threshold of 0.3, low-confidence detection boxes are filtered out. The non-maximum suppression algorithm (NMS) is used, with an IOU threshold of 0.4, to eliminate redundant detection boxes. The sub-image coordinate system is then restored to the original large-image coordinate system. Finally, structured defect information (including category, coordinates, and confidence) and detection images with visually labeled boxes are output, providing a standardized data interface for subsequent quality control and production decisions.

[0088] The accelerated inference and deployment module (7) enables the system to run efficiently in real time in an industrial environment through model quantization, memory optimization and concurrent inference technology. This module supports the real-time operation of the entire system and works in conjunction with the preprocessing module (2) and the deep learning detection module (3).

[0089] Specifically, this module ensures the real-time performance of the system in industrial environments through a series of engineering optimization techniques. These include converting the training model to ONNX format and building an inference engine based on TensorRT; enabling FP16 half-precision quantization to reduce memory usage and computational latency; utilizing CUDA page-locked memory to accelerate data transfer between the host and the device; and employing a C++ multi-threaded concurrency strategy to segment high-resolution input images into sub-images for parallel inference, using a mutex lock mechanism to ensure thread safety during result merging, thereby achieving high throughput and low latency industrial-grade real-time detection capabilities.

[0090] Example 2: In response to the industrial challenges of complex leather texture, scarce defect samples, and high real-time detection requirements in Example 1, this invention also proposes a "real-time detection method for few-sample leather defects based on multi-scale attention" in conjunction with the system proposed in Example 1.

[0091] Combination Figure 4 The method includes the following steps:

[0092] S1. Acquire the image to be inspected and perform preprocessing; Specifically, firstly, the original leather surface image to be inspected is acquired in real time using an industrial line scan camera image acquisition device, the image is sharpened and cropped, and preliminary screening is performed. The screened image is further enhanced, and finally the acquired image is converted into a standardized format.

[0093] Furthermore, it should be noted that in some embodiments, step S100 may include:

[0094] S101. Obtain images of the leather surface using an industrial line scan camera image acquisition device;

[0095] Unified input dimensions ( )for Pixel;

[0096] Memory allocation functions use The allocation size is Page-locked memory for bytes; using Pinned Memory: describes how images are loaded into Pinned Memory in preparation for CUDA preprocessing.

[0097] S102. Perform global contrast enhancement and smoothing noise reduction on the original leather surface image;

[0098] Specifically, the original image is input into the preprocessing module. First, histogram equalization is used to adjust the global grayscale distribution of the image, improving the overall contrast and clarifying details in both bright and dark areas. Then, Gaussian filtering is used to smooth the image, filtering out high-frequency noise introduced during image acquisition and preserving edge information of defects.

[0099] S103. Perform local feature enhancement on the pre-processed image;

[0100] In some specific embodiments, local enhancement processing is performed to further highlight defects that are similar in texture to the background or are small in size. First, the Sobel operator is used to locate the potential defect contours, and then contrast-limited adaptive histogram equalization using the CLAHE algorithm is applied. This technique performs histogram equalization on each local region of the image separately, enhancing the local contrast of the image, distinguishing the features of the defect region from its surrounding background, and ultimately obtaining an enhanced image with significant features that is easily recognized by the model. Typically, the clip limit of the CLAHE algorithm is set to... The tile grid size is set to... This is to enhance contrast while suppressing noise amplification.

[0101] S104. The image after formatting and feature enhancement is used to obtain a preprocessed image of the leather surface.

[0102] S2. Extract leather defect features from the standardized enhanced image through the parallel dual-branch backbone network of the deep learning detection module (3), and output a feature map that integrates attention enhancement and redundancy elimination.

[0103] The parallel dual-branch backbone network employs two branches: ResCBAM attention mechanism and spatial and channel reconstruction convolution, and is spliced ​​at the end.

[0104] Furthermore, the ResCBAM attention mechanism is specifically formed by concatenating channel attention and spatial attention through residual connections, focusing on the defect region and suppressing the interference of background texture, thereby enhancing the feature response;

[0105] Furthermore, the working principle of the channel attention in the ResCBAM attention mechanism is as follows:

[0106] ;

[0107] in, This is a channel attention map, whose dimension is the same as the number of channels in the input feature map; The input feature map for the module. and They are respectively for The global average pooling and global max pooling operations are used to aggregate spatial information; they are used to aggregate spatial information of feature maps from different perspectives. A shared multilayer perceptron is used to learn channel weights from two pooling results; The Sigmoid activation function is used to normalize the weight values ​​to 0. between.

[0108] Furthermore, the spatial attention mechanism of the ResCBAM attention mechanism works as follows:

[0109] ;

[0110] in Spatial attention map; Channel attention map; for Convolution operation of convolution kernel; This indicates a channel splicing operation.

[0111] Furthermore, the residual connections of the ResCBAM attention mechanism are as follows:

[0112] ;

[0113] in, For hyperparameters, For element-wise multiplication, For attention-enhanced feature maps.

[0114] Furthermore, the SCConv channel reconstruction convolution specifically employs grouped convolution, spatial reconstruction, and channel reconstruction sequentially to separate and reconstruct spatial and channel information in the features, eliminate redundancy in high-level feature representations, optimize feature transmission, and thereby extract more distinctive and compact feature representations.

[0115] SCConv consists of two units: SRU (Spatial Reconstruction Unit): reduces spatial redundancy through a split-reconstruction operation and uses a gating mechanism to filter important features; CRU (Channel Reconstruction Unit): reduces channel redundancy and optimizes feature representation using a segmentation-transformation-fusion strategy. Specifically, the input feature map... First, the feature map is refined using the SRU space, then refined using the CRU channel, and finally output with redundancy eliminated. .

[0116] In this embodiment, the parallel dual-branch backbone network is based on the YOLO architecture and trained using the AdamW optimizer. The preferred training hyperparameter configuration is as follows:

[0117] Total training epochs: Batch Size: Initial learning rate ( ): Input image size (imgsz): ;

[0118] The weights of the loss function are assigned as follows: bounding box loss weights Classification loss weights Distributed focus loss weights .

[0119] S3, the multi-scale feature fusion module (4) downsamples and fuses feature information at different scales in the feature map to obtain a multi-scale feature map. Specifically,

[0120] S301. Assemble the attention-enhanced feature map Feature maps for redundancy elimination Perform downsampling twice to obtain high-resolution feature maps that retain their original size. Low-resolution feature map after 2 downsampling ;

[0121] S302. Employ a cross-scale attention mechanism for feature fusion to obtain multi-scale feature maps. ;in, This is an upsampling operation used to enable... Size and Size alignment; The fusion weights for adaptive learning have values ​​located at... Between these, contributions from low-resolution and high-resolution feature maps are dynamically balanced based on feature content to achieve optimal cross-scale information integration.

[0122] S4, Prototype Classification and Localization Module (5) Based on the ProtoNet prototype network and combined with CIoU loss, it calculates the distance between multi-scale feature maps and various defect prototypes, realizing defect classification of leather surface images. Specifically,

[0123] S401. Establish the ProtoNet prototype network and for each defect category... prototype Calculations were performed to obtain ;in, For category The number of samples, For the first Categories Sample images, For feature extraction functions;

[0124] S402, For any feature vector in the multi-scale feature map Calculate its relationship with the defect category prototype Distance metric ;

[0125] S403. The distance metric is converted into the defect classification probability of the leather surface image using the Softmax function:

[0126] ;

[0127] in, Represents the input feature vector Belongs to the defect category The probability of; This represents the total number of defect categories;

[0128] S404. The CIoU loss function is used as the bounding box regression loss to locate the bounding boxes corresponding to the defect categories, outputting the category, confidence score, and bounding box coordinates for each detected defect. Specifically, the CIoU loss function is used for bounding box regression; to improve the accuracy of defect localization, this invention uses the CIoU loss function as the bounding box regression loss. This loss function simultaneously considers the differences in overlap area, center point distance, and aspect ratio. The main CIoU formula is as follows:

[0129]

[0130] The This represents the total loss for bounding box regression. This is the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box. The center point b of the predicted bounding box and the center point of the ground truth bounding box The square of the Euclidean distance between them; It is the diagonal length of the smallest closure region that simultaneously contains both the predicted bounding box and the ground truth bounding box; These are the weighting coefficients; This is the aspect ratio penalty term. In the explanation of the CIoU master formula above, it is defined as follows: When the aspect ratio is a penalty term:

[0131]

[0132] The Used to measure the consistency of aspect ratio; and These are the width and height of the prediction box, respectively; and These are the width and height of the actual bounding box, respectively; It is the arctangent function.

[0133] In the above explanation of the CIoU master formula, the definition is... When it is a weighting coefficient:

[0134]

[0135] The for The weight coefficient of the item, based on the current Value and Value dynamically adjusted The contribution to the total loss is used to balance positioning accuracy and aspect ratio consistency.

[0136] The final localization result is decoded and output; the feature map after the discrimination result is input into the YOLOv11 detection head network. The head network decodes these features and outputs the category, confidence score and precise bounding box coordinates of each detected defect, which are then sent to the post-processing module (6) of S5 for final screening.

[0137] S5, the post-processing module (6) performs quality control and formatting on the defect classification results, and outputs the final detection results. Specifically,

[0138] S501. Set the confidence threshold to 0.3 and filter out bounding boxes with a confidence level lower than the threshold.

[0139] S502. Use the non-maximum suppression (NMS) algorithm to eliminate redundant boxes, and set the threshold to 0.4 to ensure that each defective region corresponds to only one bounding box.

[0140] S503: Convert the coordinates of the sub-image detection bounding box to the coordinates of the original large image, and filter the category detection results of defects with too small an area; output the final defect category, coordinates, confidence level and other information to the front-end display page to realize automated and visual annotation of leather surface defects, and support further shutdown feedback processing.

[0141] The S6, GPU deployment, concurrent inference, and system feedback (real-time engineering implementation) modules are used to carry out the system's real-time inference tasks. By combining three major strategies—deep optimization, memory acceleration, and C++ concurrency—high performance is ensured under high-resolution image input.

[0142] S601, Model Deployment and Accuracy Optimization (Static Acceleration): This step focuses on one-time optimization configuration before model deployment, aiming to maximize the running efficiency of the TensorRT engine. ONNX Conversion and Version: First, export the trained model to ONNX format, setting the opset version to 16 during export. Precision Quantization: Subsequently, use TensorRT to build the inference engine and enable FP16 (half-precision floating-point) quantization mode to reduce memory usage and accelerate computation while ensuring that accuracy loss is within an acceptable range.

[0143] S602, CUDA Preprocessing and Memory Allocation Acceleration; This step details the key memory technologies that ensure GPU data transfer efficiency; Pinned Memory Allocation: During the CUDA preprocessing stage, the cudaMallocHost function is used to allocate page-locked memory (Pinned Memory) as a buffer for the host CPU's image input data. Buffer Size: The fixed size of the allocated memory is related to the image size. Hook, exact size 2048 2048 3 bytes to enable the fastest zero-copy or asynchronous transfer between the host and the device.

[0144] S603, C++ Concurrent Image Slicing and Inference Strategy (Dynamic Acceleration): This step describes the core concurrency strategy for processing large industrial images during actual runtime. Large Image Slicing: The ModelManager module is responsible for slicing the original large leather image into 2048... The subgraph is divided into sections of 2048 pixels. Asynchronous concurrent execution: For each subgraph's inference task, an independent parallel inference task is created using C++'s `std::async` function, fully utilizing the GPU's concurrent computing capabilities. Thread safety: When concurrently merging the detection results of subgraphs, a `std::mutex` mutex mechanism is used, with `std::lock_guard` locking critical code regions to ensure thread safety for coordinate adjustment and result merging operations.

[0145] This method was tested in a real leather production line. Targeting typical defects such as black spots, white spots, pinholes, and scratches, it achieved an average detection accuracy (mAP) of ≥95% and a single-frame inference time of <100ms under the condition of only 5-10 samples per category, meeting the requirements of real-time industrial detection.

[0146] Example 3: For the scenario in Example 1, Figure 7 A schematic diagram of an electronic device (90) that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0147] Electronic devices can also refer to various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0148] like Figure 7As shown, the electronic device (90) includes at least one processor (91) and a memory, such as a read-only memory (ROM) (92) or a random access memory (RAM) (93), which is communicatively connected to the at least one processor (91). The memory stores computer programs executable by the at least one processor. The processor (91) can perform various appropriate actions and processes based on the computer programs stored in the ROM (92) or loaded from storage units (98) into the RAM (93). The RAM (43) may also store various programs and data required for the operation of the electronic device (90). The processor (91), ROM (42), and RAM (43) are interconnected via a bus (94). An input / output (I / O) interface (95) is also connected to the bus (94).

[0149] Multiple components in the electronic device (90) are connected to the I / O interface (95), including: input units (96), such as keyboards, mice, etc.; output units (97), such as various types of displays, speakers, etc.; storage units (98), such as disks, optical disks, etc.; and communication units (99), such as network cards, modems, wireless transceivers, etc. The communication unit (99) allows the electronic device (90) to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0150] The processor (91) can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processors (91) include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor (91) performs the various methods and processes described above, such as a real-time detection method for few-sample leather defects based on multi-scale attention.

[0151] In some embodiments, the multi-scale attention-based real-time detection method for few-sample leather defects can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit (98). In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device (90) via a ROM (92) and / or a communication unit (99). When the computer program is loaded into RAM (93) and executed by a processor (91), one or more steps of the multi-scale attention-based real-time detection method for few-sample leather defects described above can be performed. Alternatively, in other embodiments, the processor (91) can be configured to perform the multi-scale attention-based real-time detection method for few-sample leather defects by any other suitable means (e.g., by means of firmware).

[0152] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0156] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0157] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0158] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A system for real-time detection of defects in leather based on few-shot learning using multi-scale attention, characterized in that, The image acquisition module (1), the preprocessing module (2), the deep learning detection module (3), the multi-scale feature fusion module (4), the prototype classification and positioning module (5), the post-processing module (6) and the acceleration inference and deployment module (7) are connected to constitute the system. The image acquisition module (1) is responsible for real-time acquisition of leather surface images, acquires high-resolution original images through an industrial line scanning camera, and transmits the images to the preprocessing module (2); the module supports image stream input and is suitable for continuous production line environment; The preprocessing module (2) receives the collected leather surface images, performs global contrast enhancement, smoothing denoising, local feature enhancement and other operations, and finally outputs images with uniform size and standardized format; the output end of the image acquisition module (1) is directly connected to the input end of the deep learning detection module (3); The deep learning detection module (3) is connected to the output end of the preprocessing module (2) and the input end of the multi-scale feature fusion module (4); it includes a double-branch backbone network in parallel, which respectively adopts ResCBAM (Residual Convolutional Block Attention Module) attention mechanism and spatial and channel reconstruction convolution (SCConv), and the features of the two branches are spliced at the end to extract high-recognizability defect features and output to the multi-scale feature fusion module (4); The multi-scale feature fusion module (4) is connected to the output end of the deep learning detection module (3) and the input end of the prototype classification and positioning module (5); it receives feature maps from the deep learning detection module (3), realizes adaptive fusion of low-resolution semantic features and high-resolution detail features through cross-scale attention mechanism; the module outputs the fused multi-scale feature maps and transmits them to the prototype classification and positioning module (5); The prototype classification and positioning module (5) is connected to the output end of the multi-scale feature fusion module (4) and the input end of the post-processing module (6); based on the ProtoNet prototype network, it calculates the distance between the input features and each class of defect prototype, realizes high-precision classification under the condition of few samples, and uses CIoU (Complete Intersection over Union) loss function for boundary box regression, and outputs the defect category, confidence and position information to the post-processing module; The post-processing module (6) performs confidence filtering, non-maximum suppression, coordinate restoration and result formatting on the detection results, and finally outputs the detection image with labeled frame and structured detection data; the module is directly connected to the output end of the prototype classification and positioning module (5) and provides output for user interface or control system. The acceleration reasoning and deployment module (7) realizes efficient real-time operation of the system in an industrial environment through model quantization, memory optimization and concurrent reasoning technology; the module supports real-time operation of the entire system and works in cooperation with the preprocessing module (2) and the deep learning detection module (3).

2. The multi-scale attention based few-shot real-time leather defect detection system according to claim 1, wherein, The ResCBAM attention mechanism specifically comprises: a residual connection channel attention (ChannelAttention) and a spatial attention (Spatial Attention) connected in series, which focuses on the defect area and suppresses the interference of background texture, and enhances the feature response.

3. The multi-scale attention based few-shot real-time leather defect detection system according to claim 2, wherein, The working principle of the channel attention of the ResCBAM attention mechanism is: ； in, This is a channel attention map, whose dimension is the same as the number of channels in the input feature map; The input feature map for the module. and They are respectively for The global average pooling and global max pooling operations are used to aggregate spatial information; they are used to aggregate spatial information of feature maps from different perspectives. A shared multilayer perceptron is used to learn channel weights from two pooling results; The Sigmoid activation function is used to normalize the weight values ​​to 0. between; The working principle of the spatial attention of the ResCBAM attention mechanism is: ； wherein for spatial attention map; for channel attention map; for convolution operation of convolution kernel; denotes channel concatenation operation; The residual connection of the ResCBAM attention mechanism is: ； wherein, is a hyperparameter, is an element-wise multiplication, is an attention enhanced feature map.

4. The multi-scale attention based few-shot real-time leather defect detection system according to claim 1, wherein, The SCConv channel reconstruction convolution is composed of two units: SRU (space reconstruction unit): reducing spatial redundancy through separation-reconstruction operation, using a gating mechanism to filter important features; CRU (channel reconstruction unit): using a segmentation-conversion-fusion strategy to reduce channel redundancy and optimize feature representation; specifically, the input feature map is first refined in space by SRU, then refined in channel by CRU, and finally outputted as a feature map with redundancy eliminated .

5. A real-time detection method for leather defects with few samples based on multi-scale attention, characterized in that, The method comprises the following steps: S1: The image acquisition module (1) acquires real-time leather surface images, and the preprocessing module (2) preprocesses the leather surface images and outputs standardized enhanced images; S2: The dual-branch backbone network of the deep learning detection module (3) extracts leather defect features, and outputs feature maps fused with attention enhancement and redundancy elimination; S3: The multi-scale feature fusion module (4) performs down-sampling and fusion of feature information of different scales on the feature maps to obtain multi-scale feature maps; S4: The prototype classification and positioning module (5) realizes distance calculation between the multi-scale feature maps and the prototypes of various defects based on the ProtoNet prototype network and in combination with the CIoU loss, and realizes defect classification of the leather surface images; S5: The post-processing module (6) performs quality control and formatting on the defect classification results, and outputs the final detection results; S6: The acceleration reasoning and deployment module (7) realizes real-time reasoning of steps S1-S6 through engineering optimization, and ensures industrial production line application.

6. The multi-scale attention based few-shot real-time leather defect detection method according to claim 5, characterized in that, The step S3 specifically comprises: S301: the spliced attention enhanced feature map and the feature map of redundancy elimination Two times of downsampling are performed to obtain a high-resolution feature map retaining the original size , a low-resolution feature map after 2 times of downsampling ; S302: feature fusion is performed by using a cross-scale attention (CSA) mechanism to obtain a multi-scale feature map ; wherein, is an up-sampling operation for aligning the size of with the size of ; is an adaptive learning fusion weight, whose value is between , for dynamically balancing the contributions from low-resolution feature maps and high-resolution feature maps according to feature content to achieve optimal cross-scale information integration.

7. The multi-scale attention based few-shot real-time leather defect detection method according to claim 5, characterized in that, The step S4 specifically comprises: S401: Build a ProtoNet prototype network and for each defect category prototype Calculations were performed to obtain ;in, For category The number of samples, For the first Categories Sample images, For feature extraction functions; S402: calculating a distance metric of any feature vector in the multi-scale feature map to a prototype of a defect class ;​​​ S403: The distance metric is converted into a defect classification probability of the leather surface image through a Softmax function: ； wherein, represents an input feature vector belongs to a defect class probability; is the total number of defect classes. S404: The CIoU loss function is used as a boundary box regression loss to position the boundary box corresponding to the defect category, and the class, confidence score and boundary box coordinates of each detected defect are output.

8. The multi-scale attention based few-shot real-time leather defect detection method according to claim 5, characterized in that, The step S5 specifically comprises: S501: A confidence threshold is set to filter out boundary boxes with a confidence lower than the threshold; S502: A non-maximum suppression (NMS) algorithm is used to eliminate redundant boxes to ensure that each defect area corresponds to only one boundary box; S503: The sub-image detection boundary box coordinates are converted into original large image coordinates, and the class detection results of defects with small areas are filtered out.

9. The multi-scale attention based few-shot real-time leather defect detection method according to claim 5, characterized in that, The step S6 specifically comprises: converting the trained model into an ONNX format and building a TensorRT inference engine, starting an FP16 semi-precision quantization mode to reduce the memory occupation and accelerate the calculation; in a C++ deployment environment, using cudaMallocHost to allocate page-locked memory to accelerate the data transmission between the host and the device; at the same time, using an std::async asynchronous multi-thread concurrent strategy, cutting the high-resolution leather large image into multiple sub-images for parallel inference, and using a mutex mechanism to ensure the thread safety of the result merging, so as to greatly reduce the inference delay.

10. An electronic device and a computer readable storage medium, characterized by The electronic device comprises at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the multi-scale attention-based real-time leather defect detection method of any one of claims 5-9; the computer readable storage medium stores computer instructions for enabling the processor to execute the multi-scale attention-based real-time leather defect detection method of any one of claims 5-9 when executed.

Images