Lightweight belt tearing detection method and system based on feature information collaborative simplification

Abstract

The invention provides a lightweight belt tearing detection method and system based on feature information collaborative simplification, and relates to the technical field of belt tearing detection.The method comprises the steps that S1, a to-be-detected belt surface image in the belt running process is obtained; s2, inputting the surface image of the belt to be detected into a feature extraction module of a Mama block based on redundant feature compression convolution and parameter simplification for feature extraction, and generating three feature maps with different scales; s3, inputting the three feature maps with different scales into a feature fusion module based on a lightweight adaptive upsampler for feature fusion, and outputting a tearing prediction result including frame prediction, category prediction and segmentation mask prediction; and S4, inputting the tear prediction result into a post-processing module for post-processing, and outputting a final tear detection result of the to-be-detected belt surface image. The belt tearing detection device can perform lightweight detection on belt tearing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of belt tear detection, in particular to a lightweight belt tear detection method and system based on feature information coordination simplification. BACKGROUND

[0002] As an important part of modern industrial production, the belt conveying system undertakes a large number of material transportation tasks. Due to long-term high-intensity working conditions and complex operating environments, the belt surface is prone to different types of tears such as large-area damage, patch scratches, belt stacking, and edge damage, which not only causes production interruption but also may cause safety accidents. As one of the key issues of mine belt safety production, belt tear detection technology directly affects the reliability and safety of the production process. This technology accurately identifies the type and location of the belt tear by analyzing the belt surface image collected by an industrial camera, ensuring that the producer can respond effectively to different tear types in a timely manner. Therefore, there is an increasing demand for the accuracy and real-time performance of belt system monitoring in industrial production.

[0003] Most existing belt tear detection methods use target detection technology based on the YOLO algorithm to generate detection bounding boxes to locate the tear area and its category. However, in practical applications, the tear area of the belt often has complex shapes and small areas, and the target detection box usually cannot closely match the tear area, resulting in inaccurate coverage. This can easily cause false positives and false negatives, leading to low detection accuracy. To address this issue, semantic segmentation algorithms can classify each pixel in the image, regardless of the shape and size of the tear. However, when there are multiple tears in an image, semantic segmentation cannot distinguish them as independent instances, which limits its ability to distinguish different objects in the same tear category and can still cause false positives and false negatives, resulting in decreased detection accuracy. To address the above challenges, instance segmentation algorithms can not only accurately depict the shape and size of the tear area through pixel-level analysis but also distinguish multiple tears in an image as different tear instances, allowing for precise detection of their shapes and boundaries, and outputting more detailed and accurate detection results. Therefore, compared to target detection and semantic segmentation algorithms, instance segmentation algorithms can better meet the accuracy requirements of multi-target detection in complex scenarios in belt tear detection.

[0004] However, existing instance segmentation algorithms have complex network structures and require high computational costs for feature extraction and upsampling operations without distinguishing between different areas of the image. In the belt tear detection task, the tear area usually occupies a small area in the entire image, and indiscriminate processing of the entire image can waste a lot of computing resources. Therefore, it is necessary to simplify the computational overhead caused by redundant features in non-tear areas.

[0005] In addition, in multi-class detection tasks including belt tear detection, the increase in the number of classes further increases the computational load of the model. In particular, when the similarity between classes is high, due to the low sharing of features in the instance segmentation algorithm, the model has difficulty in distinguishing between the common features and the detailed features of each class that exist in similar classes at the same time, and is prone to confusion in the classification process. Therefore, the model needs to extract the unique features of each class by deepening the network structure or setting a more complex activation function to distinguish the more subtle differences between classes, which results in the model consuming more computing resources. The above limitations of large computational load and slow processing speed are difficult to meet the real-time detection requirements while ensuring detection accuracy, especially in industrial scenarios that require fast response. SUMMARY

[0006] In order to solve the technical problems existing in the prior art, the present application provides a lightweight belt tear detection method based on feature information cooperative simplification, and the technical solution is as follows:

[0007] On the one hand, a lightweight belt tear detection method based on feature information cooperative simplification is provided, which comprises:

[0008] S1, obtaining a to-be-detected belt surface image in a belt running process;

[0009] S2, inputting the to-be-detected belt surface image into a feature extraction module based on a redundant feature compression convolution and a parameter simplified Mamba block for feature extraction, to generate three different scale feature maps;

[0010] S3, inputting the three different scale feature maps into a feature fusion module based on a lightweight adaptive up-sampler for feature fusion, to output a tear prediction result including a bounding box prediction, a class prediction and a segmentation mask prediction;

[0011] S4, inputting the tear prediction result into a post-processing module for post-processing, to output a final tear detection result of the to-be-detected belt surface image.

[0012] Optionally, the feature extraction module, after the input of the pre-processed original image with a size of HxWx3, H and W being the image height and width respectively, is subjected to two redundant feature compression convolution operations, the pixel is halved while the redundant feature information is reduced, and the image size is changed to C is the number of channels, and then the feature representation is further refined by a parameter simplified Mamba block, and after repeating the process of redundant feature compression convolution and parameter simplified Mamba block twice, a shallow feature map X1 with a size of is generated.

[0013] The redundant feature compression convolution and the Mamba block process of parameter simplification are repeated once to generate a middle layer feature map X2 with a size of ;

[0014] After the redundant feature compression convolution is performed once again, a spatial pyramid pooling layer SPPF block is further used to extract detailed feature information and generate a deep layer feature map X3 with a size of ;

[0015] The three feature maps with different sizes provide rich feature representations for a subsequent instance segmentation task.

[0016] Optionally, the redundant feature compression convolution is performed on the input image through two convolution operations, a first convolution operation using a large convolution kernel to halve the height and width of the image, and a second convolution operation using a small convolution kernel to extract preliminary feature information.

[0017] Then, the feature information is subjected to a spatial feature simplification operation by a redundant spatial compression module, and spatially simplified feature information is output. Figure X W The feature information is further subjected to a channel number simplification operation by a redundant channel compression module, and a channel-simplified feature map Y' is output.

[0018] Finally, the feature map Y' is subjected to a 1x1 convolution to complete feature map dimension matching, and a redundant feature-simplified feature map Y is output.

[0019] Optionally, the redundant spatial compression module uses a trainable variable θ in group normalization to evaluate the content of key information in different feature maps, and generates a weight W based on θ from the normalized feature map. i The formula expressions of group normalization and weight calculation are as follows:

[0020]

[0021] Where X represents an input feature map, X GN represents a group-normalized feature map output, W i represents a corresponding feature map weight, μ and σ represent a mean and a standard deviation, θ and ε are trainable variables, and β is a very small constant used to ensure stability. A larger θ represents a greater change between pixels and richer spatial information.

[0022] Subsequently, the feature map weight W i is subjected to a nonlinear transformation by a sigmoid activation function to limit its range to 0 to 1, and a threshold is distinguished by a gate function Gate to distinguish W1 above the threshold and W2 below the threshold. After being multiplied by the corresponding feature maps, a feature map with more feature information and a feature map with less feature information Will With Randomly divided into And Four spatial feature subsets, and combine the subsets with large information content with the subsets with small information content through multiplication operation;

[0023] Finally, the feature splicing avoids excessive small information feature map calculation, and outputs the spatially simplified features Figure X W , reducing the redundant features in spatial dimension.

[0024] Optionally, the redundant channel compression module, the spatially simplified features Figure X W are divided into two groups according to the number of channels: one group has a number of channels αC, and the other group has a number of channels (1-α)C, where α is a hyperparameter and 0≤α≤1;

[0025] Then, 1×1 convolution is used on the number of channels of the two groups of feature maps to obtain X a and X b , respectively. This design groups the compressed features by channel number, effectively reducing the number of channels in the calculation process while ensuring feature richness, and realizes channel simplification;

[0026] Take the input X a as the input of channel key feature extraction, and perform global convolution and point convolution operations respectively, and add the two to obtain the output Y 1， Capture the long-distance dependence between features;

[0027] Take the input X b as a supplement to channel key feature extraction, and perform point convolution operation, and the obtained result is spliced with the original input feature to obtain the output Y2;

[0028] Then, Y1 and Y2 are respectively subjected to global pooling, and the global spatial information and channel information are combined together;

[0029] Then, the two results after pooling are calculated by the Softmax function to obtain the feature weight vectors γ1 and γ2;

[0030] Finally, based on the feature weight vectors and Y1, Y2, the linear sum calculation is performed to obtain the output Y' = γ1Y1+γ2Y2, Y' is the final output of the channel-simplified feature map.

[0031] Optionally, the parameter simplified Mamba block, the output feature map Y of the redundant feature compression convolution is first subjected to a 1×1 convolution dimension reduction processing, thereby reducing the calculation complexity without significantly losing information, and obtaining a feature tensor Y inThe four groups are divided according to the number of channels, and a feature directional scanning module is applied to each group of feature maps with a size of ;

[0032] The feature directional scanning module performs directional scanning along four directions from left to right, from right to left, from top to bottom and from bottom to top along the marker sequence, splices the features extracted from the scanned positions according to the scanning order to form a one-dimensional feature sequence, reduces the data amount when all features are processed synchronously, and only needs to consider the front and back relationship between sequences when calculating the one-dimensional feature sequence, instead of the relationship in the entire two-dimensional plane, thereby reducing the calculation overhead while maintaining high precision;

[0033] After directional scanning, layer normalization processing is performed to stabilize the feature distribution, then the feature sequence after layer normalization is taken as the input of the Mamba sub-block, the feature map dimension is adjusted through linear transformation operation, and then the deep separable convolution is used to reduce the calculation amount of the Mamba sub-block while maintaining the feature extraction effect, then the one-dimensional self-attention mechanism 1D-SS in the Mamba sub-block is used to enhance the attention of the key feature area, and the feature sequence is subjected to layer normalization and linear transformation again and taken as the output of the Mamba sub-block, and this process relies on the lightweight advantage of the one-dimensional attention mechanism calculation process to efficiently and accurately identify deep details;

[0034] Then the feature sequence output by the Mamba sub-block is combined with the layer normalized feature sequence through residual connection to ensure effective information transmission and prevent the common gradient vanishing or explosion problem in deep networks;

[0035] The combined feature sequence is subjected to layer normalization processing again to ensure the stability of the data distribution and make the model converge faster, and a feedforward network FFN is used to extract and convert deeper features from the layer normalized feature map, the output of the FFN is added to the residual connection result, and a flattened sequence combined with the input features and high-level semantic information is generated, and the dimension of the flattened sequence is , where B is the batch size, and N is the number of markers in the sequence, N = H' x W';

[0036] The flattened sequence is subjected to channel interaction calculation to increase the cross-channel information exchange.

[0037] Optionally, the channel interaction calculation on the flattened sequence to increase the cross-channel information exchange specifically includes:

[0038] First, the feature tensor map Y in is subjected to average pooling AvgPool to calculate channel statistical information and extract global feature information;

[0039] Next, the channel interaction coefficient Y channeland the calculation of the target feature map Y out is shown in the following formula:

[0040] Y channel = δ (P2σ (P1·AvgPool (Y in )))

[0041]

[0042] where δ and σ represent two different nonlinear functions, which are used to assign an importance weight to each channel to calculate the feature information correlation of the channel, P1 and P2 are two learnable weights, Y contact is the result after flattening and concatenating the four feature directional scanning modules, and the symbol represents multiplying the elements in the sequence with the channel interaction coefficient respectively and merging and concatenating the results as the target feature map Y out to recalibrate the feature sequence output of the Mamba sub-block. Through the nonlinear function and the weight adjustment, the information interaction and the importance weight distribution between channels are realized, and the sharing of general features between channels is enhanced.

[0043] Finally, the target feature map Y out is normalized layer by layer and subjected to a feedforward network FFN operation, further enhancing the correlation between features, and the feature tensor Y in is concatenated and used as the feature extraction output of the parameter simplified Mamba block.

[0044] Optionally, the feature fusion module integrates the feature information of different levels by concatenating the deep feature map X3, the middle layer feature map X2, and the shallow layer feature map X1, as follows:

[0045] The deep feature map X3 is first subjected to up-sampling processing by a lightweight adaptive up-sampler. After updating the size of the high-level feature information with a small amount of calculation, it is positioned to a higher resolution and concatenated with the middle layer feature map X2. Then, the concatenated feature map is passed to the C2f block of the YOLOV8 model, and key information is further extracted from the fused feature map to generate a feature F1. The F1 is then concatenated with the shallow layer feature map X1 after being processed by a lightweight adaptive up-sampler to generate a feature F2.

[0046] X3 containing rich abstract local information is input into detection head 1, and X3 is adjusted in feature space layout after 3x3 convolution structure and then concatenated with F1 and input into detection head 2. The concatenation result is input into detection head 3 after being processed by C2f block and 3x3 convolution and concatenated with F2 containing more global information.

[0047] Finally, the tearing prediction results including the bounding box prediction, the category prediction and the segmentation mask prediction are output based on the three detection heads respectively.

[0048] Optionally, the lightweight adaptive up-sampler realizes dynamic up-sampling by generating an offset, and reduces the calculation complexity of the up-sampling process, and the process is as follows:

[0049] First, the input feature Figure X Two different linear transformations are applied, and the result of one of the linear transformations is calculated using a 0.5 times sigmoid function to obtain a reasonable pixel value range to ensure the stability and controllability of the pixel value.

[0050] Then, the two results are multiplied and then rearranged through a pixel shuffle operation to convert the low-resolution feature map into a high-resolution feature map with extremely low calculation, thereby increasing the diversity and complexity of the sampling set and improving the up-sampling efficiency.

[0051] The pixel shuffle obtains an up-sampling offset O, which can adaptively adjust the position of the sampling point, further enhancing the flexibility and accuracy of the sampling set.

[0052] Then, the up-sampling offset O and the original up-sampling grid are combined to form a new sampling set S, and the input feature map is accurately matched with the sampling set S based on the built-in grid_sample function of Pytorch, and the powerful calculation capability of the grid_sample function is used to complete the spatial transformation of the feature map. The grid_sample function takes the feature map in the original up-sampling grid and the sampling offset O as input, uses bilinear interpolation to calculate the pixel value of the new position after the sampling offset, and then controls the alignment mode of the boundary pixels to accurately adjust the pixel positions to obtain the correct mapping of the feature map, finally realizes the dynamic correspondence between the feature map and the sampling set, and generates the up-sampled feature Figure X .

[0053] On the other hand, a lightweight belt tearing detection system based on feature information cooperative simplification is provided, and the system comprises:

[0054] An acquisition module is configured to acquire a to-be-detected belt surface image in a belt running process.

[0055] A feature extraction module is configured to input the to-be-detected belt surface image into a feature extraction module of a Mamba block based on redundant feature compression convolution and parameter simplification to perform feature extraction and generate three feature maps of different scales.

[0056] The feature fusion module is configured to input the three different scale feature maps into a feature fusion module based on a lightweight adaptive up-sampler for feature fusion, and output a tearing prediction result including a bounding box prediction, a category prediction, and a segmentation mask prediction.

[0057] The post-processing module is configured to input the tearing prediction result into a post-processing module for post-processing, and output a final tearing detection result of the to-be-detected belt surface image.

[0058] In another aspect, an electronic device is provided, which includes a processor and a memory having at least one instruction stored therein, the at least one instruction being loaded and executed by the processor to implement the above-mentioned feature information collaborative simplification-based lightweight belt tearing detection method.

[0059] In another aspect, a computer-readable storage medium is provided, which has at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement the above-mentioned feature information collaborative simplification-based lightweight belt tearing detection method.

[0060] The technical solution provided by the present application brings at least the following beneficial effects:

[0061] 1) For the multi-class detection based on the instance segmentation model in the existing scheme, a feature extraction network with a parameter simplified Mamba block is designed, the input channels are divided into four groups for independent scanning in different directions through directional scanning design, the parameter amount of the overall module is simplified by grouping processing, and similar features are shared by different tearing categories relying on the interaction design of each channel, which overcomes the problem of excessive calculation caused by capturing too many detailed features when the number of categories is large and the similarity between categories is high, greatly improving the segmentation and classification efficiency of the tearing area.

[0062] 2) For the calculation burden caused by the redundant features of the tearing area during feature extraction, the redundant feature compression convolution is replaced with the traditional convolution structure, the image regions are separated into sub-feature sets in the spatial dimension, the important features after spatial refinement are generated after weight reconstruction, and then the channel separation is also performed in the channel dimension, and the final spatial channel refined features are formed through convolution transformation, thereby avoiding the consumption of too much calculation resource caused by redundant features, reducing the calculation complexity of the feature extraction link, and improving the model training speed.

[0063] 3) In order to solve the problem of too large calculation amount of up-sampling caused by high resolution of existing segmentation model, a lightweight adaptive up-sampler is added, the idea of point sampling is used, the up-sampling points are located through dynamic adjustment of offset, and the sampling ratio can be adaptively adjusted according to different target sizes, the multi-scale feature sampling of different tearing categories is adapted, high-resolution features are not needed as input, the dynamic adjustment of up-sampling calculation amount for different regions is realized, the learning efficiency of the model is greatly improved, and the problems of large up-sampling calculation amount and slow model training caused by high feature resolution are overcome. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Figure 1 is a lightweight belt tearing detection method flow chart based on feature information cooperative simplification provided by the embodiment of the present application;

[0066] Figure 2 is a lightweight belt tearing detection method overall block diagram based on feature information cooperative simplification provided by the embodiment of the present application;

[0067] Figure 3 is a feature extraction module structure block diagram provided by the embodiment of the present application;

[0068] Figure 4 is a redundant feature compression convolution structure block diagram provided by the embodiment of the present application;

[0069] Figure 5 is a redundant space compression module structure block diagram provided by the embodiment of the present application;

[0070] Figure 6 is a redundant channel compression module structure block diagram provided by the embodiment of the present application;

[0071] Figure 7 is a parameter simplified Mamba block structure block diagram provided by the embodiment of the present application;

[0072] Figure 8 is a feature directional scanning module structure block diagram provided by the embodiment of the present application;

[0073] Figure 9 is a feature fusion module structure block diagram provided by the embodiment of the present application;

[0074] Figure 10A structure block diagram of a lightweight adaptive up-sampler provided by an embodiment of the present application is shown in the figure.

[0075] Figure 11 A lightweight belt tearing detection system based on feature information cooperative simplification provided by an embodiment of the present application is shown in the figure.

[0076] Figure 12 A structure schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0077] To make the technical problems, technical solutions and advantages of the present application clearer, specific embodiments will be described in detail below with reference to the accompanying drawings.

[0078] An embodiment of the present application provides a lightweight belt tearing detection method based on feature information cooperative simplification, which can be implemented by an electronic device, which can be a terminal or a server. Figure 1 A flowchart of the method is shown in the figure, Figure 2 An overall block diagram of the method is shown in the figure, and the processing flow can include the following steps:

[0079] S1, obtaining a to-be-detected belt surface image in a belt running process;

[0080] An embodiment of the present application uses an industrial camera to shoot a 2D belt surface image in a belt running process under a complex industrial scene, and the image pixel size is 2048x2000.

[0081] S2, inputting the to-be-detected belt surface image into a feature extraction module based on a redundant feature compression convolution and a parameter simplified Mamba block for feature extraction, to generate three different scale feature maps;

[0082] Optionally, as shown in the figure, Figure 3 the feature extraction module performs a redundant feature compression convolution operation twice on the input original image with a preprocessed size of HxWx3, H and W are the image height and width respectively, reduces the redundant feature information while halving the pixels, and changes the image size to C is the number of channels, then a parameter simplified Mamba block is used to further refine the feature representation, and after repeating the redundant feature compression convolution and the parameter simplified Mamba block process twice, a shallow feature map X1 with a size of is generated.

[0083] After repeating the redundant feature compression convolution and the parameter simplified Mamba block process once, a middle layer feature map X2 with a size of is generated.

[0084] After another redundant feature compression convolution, further extract detailed feature information and generate a deep feature map X3 with a size of

[0085] The three different sizes of feature maps provide rich feature representations for subsequent instance segmentation tasks.

[0086] Optionally, as shown in Figure 4 , the redundant feature compression convolution, the input image is subjected to two convolution operations, the first convolution uses a large convolution kernel (convolution kernel (k=3), stride (s=2) and sets zero padding (p=1)) to halve the height and width of the image, and the second convolution uses a small convolution kernel (convolution kernel (k=1), stride (s=1) and sets zero padding (p=0)) to extract preliminary feature information;

[0087] Then the feature information is subjected to spatial feature refinement operation by the redundant spatial compression module, and the spatially refined feature Figure X W is output.

[0088] Finally, the feature map Y' is subjected to a 1x1 convolution to complete feature map dimension matching, and the redundant feature refined feature map Y is output.

[0089] Optionally, as shown in Figure 5 , the redundant spatial compression module uses trainable variables θ in group normalization to evaluate the content of key information in different feature maps, and generates weights W i from the normalized feature map based on θ.

[0090]

[0091] Where X represents the input feature map, X GN represents the group normalized feature map output, W i represents the corresponding feature map weight, μ and σ represent the mean and standard deviation, θ and ε are trainable variables, and β is a small constant to ensure stability. Greater θ represents greater variation between pixels and richer spatial information.

[0092] Then the feature map weight W i is subjected to nonlinear transformation by the sigmoid activation function to limit its range to 0 to 1, and the threshold is distinguished by the gate function Gate to distinguish W1 above the threshold and W2 below the threshold, which are multiplied by their corresponding feature maps to obtain feature maps with more feature information ​The feature map with less feature information Will be With Randomly divided into And Four spatial feature subsets, and the subsets with large information amount and the subsets with small information amount are combined with each other through multiplication operation;

[0093] Finally, the spatially simplified features are output by avoiding excessive small information amount feature map calculation Figure X W , reducing the redundant features in spatial dimension.

[0094] Optionally, as shown in Figure 6 , the redundant channel compression module, the spatially simplified features Figure X W are divided into two groups according to the number of channels: one group has αC channels, and the other group has (1-α)C channels, where α is a hyperparameter and 0≤α≤1;

[0095] Then, 1×1 convolution is used on the number of channels of the two groups of feature maps to obtain X a and X b , respectively. This design groups the compressed features according to the number of channels, effectively reduces the number of channels in the calculation process while ensuring the richness of the features, and realizes channel simplification;

[0096] Take the input X a as the input of channel key feature extraction, and perform global convolution and point convolution operations respectively, and add the two to obtain the output Y 1， to capture the long-distance dependency relationship between features;

[0097] Take the input X b as a supplement to channel key feature extraction, and perform point convolution operation, and the obtained result is spliced with the original input feature to obtain the output Y2;

[0098] Then, Y1 and Y2 are respectively subjected to global pooling, and the global spatial information and channel information are combined together;

[0099] Then, the two results after pooling are calculated by Softmax function to obtain feature weight vectors γ1 and γ2;

[0100] Finally, based on the feature weight vectors and Y1, Y2, linear addition calculation is performed to obtain the output Y' = γ1Y1+γ2Y2, Y' is the final output of the channel-simplified feature map.

[0101] Optionally, as shown in Figure 7As shown, the parameter-reduced Mamba block compresses the output feature map Y of the redundant feature convolution, first reduces the dimension through a 1x1 convolution to reduce the computational complexity without significant loss of information, and obtains a feature tensor Y with a size of H'xW'xC' in The feature maps are then divided into four groups according to the number of channels, and a feature directional scanning module is applied to each group of feature maps with a size of

[0102] The feature directional scanning module performs directional scanning along four directions from left to right, from right to left, from top to bottom, and from bottom to top, respectively, concatenates the features extracted at the scanned positions in the scanning order to form a one-dimensional feature sequence, reduces the data amount when all features are processed synchronously, and only needs to consider the relationship between the front and back of the sequence when calculating the one-dimensional feature sequence, rather than the relationship on the entire two-dimensional plane, thereby reducing the computational overhead while maintaining high accuracy;

[0103] After directional scanning, layer normalization processing is performed to stabilize the feature distribution, and then the layer-normalized feature sequence is taken as the input of the Mamba sub-block, as shown in Figure 8 After the linear transformation operation adjusts the dimension of the feature map, the deep separable convolution is used to reduce the computational amount of the Mamba sub-block while maintaining the feature extraction effect, and then the one-dimensional self-attention mechanism 1D-SS in the Mamba sub-block is used to enhance the attention of the key feature area, and the layer normalization and linear transformation are performed again on the feature sequence to obtain the output of the Mamba sub-block, which relies on the lightweight advantage of the one-dimensional attention mechanism calculation process to efficiently and accurately identify deep details;

[0104] Then, the feature sequence output by the Mamba sub-block and the layer-normalized feature sequence are combined through residual connection to ensure effective information transmission and prevent the common gradient vanishing or explosion problem in deep networks;

[0105] The combined feature sequence is again subjected to layer normalization processing to ensure the stability of the data distribution and make the model converge faster, and the feedforward network FFN is used to perform deeper feature extraction and conversion on the layer-normalized feature map, and the output of the FFN is added to the residual connection result to generate a flattened sequence that combines the input features and high-level semantic information, with a dimension of where B is the batch size, and N is the number of tokens in the sequence, N = H'xW';

[0106] The flattened sequence is subjected to channel interaction calculation to increase the cross-channel information exchange.

[0107] Optionally, as shown in Figure 7 ​As shown, the channel interaction calculation is performed on the flattened sequence to increase the cross-channel information exchange, specifically including:

[0108] First, the feature tensor graph Y in is averaged pooled AvgPool to calculate channel statistics and extract global feature information;

[0109] Next, the channel interaction coefficient Y channel is calculated with the target feature graph Y out , whose formula is shown as follows:

[0110] Y channel = δ (P2σ (P1·AvgPool (Y in )))

[0111]

[0112] Where δ and σ represent two different nonlinear functions, which are used to assign an importance weight to each channel to calculate the feature information correlation of the channel, P1 and P2 are two learnable weights, Y contact is the result of concatenating the flattened sequence of the four feature directional scanning modules, and the symbol represents multiplying the elements in the sequence with the channel interaction coefficient respectively and merging and concatenating the results into the target feature graph Y out to recalibrate the feature sequence output of the Mamba sub-block, and through the nonlinear function and weight adjustment, the information interaction between channels and the importance weight distribution are realized, enhancing the sharing of general features between channels;

[0113] Finally, the target feature graph Y out is normalized layer by layer and fed forward network FFN operation, further enhancing the correlation between features, and the feature tensor Y in after concatenation is used as the feature extraction output of the parameter simplified Mamba block.

[0114] This structure enables general similar features to be shared by different detection categories, greatly reducing the computational complexity of the feature extraction link, and the accurate recognition of shared features also helps to distinguish the detailed features of each category, without the need to deepen the network depth to ensure classification accuracy.

[0115] S3, input the three different scale feature graphs into the feature fusion module based on the lightweight adaptive up-sampler for feature fusion, and output the tearing prediction results including bounding box prediction, category prediction and segmentation mask prediction;

[0116] Optionally, as Figure 9As shown, the feature fusion module integrates the deep feature map X3, the middle feature map X2, and the shallow feature map X1 to integrate feature information at different levels, as follows:

[0117] The deep feature map X3 is first upsampled by a lightweight adaptive up-sampler to update the size of the high-level feature information and then positioned to a higher resolution. The upsampled feature map is then concatenated with the middle feature map X2, and the concatenated feature map is then passed to the C2f block of the YOLOV8 model to further extract key information to generate a feature F1. The F1 is then concatenated with the shallow feature map X1 to generate a feature F2 after being processed by the lightweight adaptive up-sampler.

[0118] The X3 containing rich abstract local information is input into the detection head 1, and the X3 is adjusted by a 3x3 convolution structure to adjust the feature space layout and then input into the detection head 2 after being concatenated with the F1. The concatenated result is input into the detection head 3 after being processed by the C2f block and a 3x3 convolution and being concatenated with the F2 containing more global information.

[0119] Finally, the three detection heads output tearing prediction results including bounding box prediction, class prediction, and segmentation mask prediction.

[0120] Optionally, as shown in Figure 10 The lightweight adaptive up-sampler generates an offset to achieve dynamic up-sampling and reduce the computational complexity of the up-sampling process, as follows:

[0121] First, the input feature Figure X Two different linear transformations are applied, and the result of one of the linear transformations is calculated using a 0.5 times sigmoid function to obtain a reasonable pixel value range to ensure the stability and controllability of the pixel value.

[0122] Then, the two results are multiplied and then rearranged by a pixel shuffle operation to convert the low-resolution feature map into a high-resolution feature map with a very low computational complexity, which increases the diversity and complexity of the sampling set while improving the up-sampling efficiency.

[0123] The pixel shuffle operation obtains an up-sampling offset O, which can adaptively adjust the position of the sampling points, further enhancing the flexibility and accuracy of the sampling set.

[0124] After that, the up-sampling offset O and the original up-sampling grid are combined to form a new sampling set S. Based on the built-in grid_sample function of Pytorch, the input feature map is accurately matched with the sampling set S. The powerful computing capability of the grid_sample function is used to complete the spatial transformation of the feature map. The grid_sample function takes the feature map in the original up-sampling grid and the sampling offset O as input, uses the bilinear interpolation method to calculate the pixel value of the new position after the sampling offset, and then adjusts the pixel position accurately to obtain the correct mapping of the feature map by controlling the alignment mode of the boundary pixels. Finally, the dynamic correspondence between the feature map and the sampling set is realized, and the up-sampled feature is generated Figure X .

[0125] S4, input the tearing prediction result into a post-processing module for post-processing, and output the final tearing detection result of the to-be-detected belt surface image.

[0126] After the boundary box, class probability and segmentation mask prediction of the detection head in the embodiment of the application, the model first performs non-maximum suppression (NMS) processing on the prediction result in the post-processing stage to eliminate boundary boxes with high overlap. Then, the model applies threshold filtering to the remaining boundary boxes and corresponding class probabilities, and only retains those targets with a class probability higher than a certain threshold. At the same time, for the segmentation mask, the model uses binary processing to convert the continuous value mask into a binary mask to optimize the quality of the mask. After the above processing, the optimized boundary box, class label and segmentation mask belt tearing final detection result are finally output.

[0127] The training process of the overall model composed of various modules in the embodiment of the application is as follows:

[0128] 1) Belt tearing image data collection.

[0129] In a complex industrial scene, 2D belt surface images during belt running are captured by an industrial camera at regular intervals. All image pixel sizes are 2048x2000. Due to the difficulty of data acquisition and the unevenness of the proportion of images containing tearing areas, subsequent manual expansion of the experimental data set is required to achieve a balanced state of the number ratio of defective and non-defective images.

[0130] 2) Data labeling.

[0131] The labelme labeling tool is used for data labeling of instance segmentation tasks. The polygon or brush tool is used to finely depict the boundaries of each pixel-level tearing area, and the target box is divided into edge damage, belt stacking, patch scratch, large area damage, small target tearing and other damage categories. The json segmentation annotation file is exported to complete the data preprocessing process.

[0132] 3) Data set division.

[0133] All image-label pairs are divided into a dataset in a ratio of 8:1:1, that is, 80% of the data is used as a training set for model training, 10% of the data is used as a validation set for adjusting model parameters and preventing overfitting, and the remaining 10% of the data is used as a test set for evaluating the final performance of the model, and all data division processes are completed by random sampling.

[0134] 4) Data preprocessing.

[0135] Data augmentation techniques such as rotation, scaling, and color transformation are used to increase the generalization ability of the model. Subsequently, the bounding box and segmentation mask are converted into the format required by the model, and the data loader is used to batch load and process the data to improve training efficiency. The processed original image is output to complete the data preprocessing process.

[0136] 5) Model training.

[0137] The model is trained using the training set.

[0138] 6) Model validation and testing.

[0139] The validation set is used to evaluate the performance of the model and adjust the hyperparameters. Then the model is tested on the test set to verify the model effect. The optimal one is taken as the final model.

[0140] As shown in Figure 11 , the embodiment of the present application also provides a lightweight belt tearing detection system based on feature information cooperative simplification, which comprises:

[0141] An acquisition module 1110 is configured to acquire a to-be-detected belt surface image in a belt running process.

[0142] A feature extraction module 1120 is configured to input the to-be-detected belt surface image into a feature extraction module based on a redundant feature compression convolution and a parameter simplified Mamba block to perform feature extraction and generate three different scale feature maps.

[0143] A feature fusion module 1130 is configured to input the three different scale feature maps into a feature fusion module based on a lightweight adaptive up-sampler to perform feature fusion and output tearing prediction results including bounding box prediction, class prediction, and segmentation mask prediction.

[0144] A post-processing module 1140 is configured to input the tearing prediction results into a post-processing module to perform post-processing and output a final tearing detection result of the to-be-detected belt surface image.

[0145] The light-weight belt tearing detection system based on feature information cooperative simplification provided by the embodiment of the present application has a function structure corresponding to the light-weight belt tearing detection method based on feature information cooperative simplification provided by the embodiment of the present application, and will not be described here.

[0146] Figure 12 is a structural schematic diagram of an electronic device 1200 provided by the embodiment of the present application. The electronic device 1200 can have a relatively large difference due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 1201 and one or more memories 1202, wherein the memory 1202 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 1201 to realize the steps of the light-weight belt tearing detection method based on feature information cooperative simplification.

[0147] In the exemplary embodiment, a computer readable storage medium, for example, a memory including instructions, is also provided, and the instructions can be executed by a processor in a terminal to complete the light-weight belt tearing detection method based on feature information cooperative simplification. For example, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0148] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware to complete, and the program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0149] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A lightweight belt tear detection method based on feature information collaborative simplification, characterized in that, The method comprises: S1, acquiring a to-be-detected belt surface image in a belt running process; S2, inputting the to-be-detected belt surface image into a feature extraction module based on a redundant feature compression convolution and a parameter simplified Mamba block to perform feature extraction, and generating three different scale feature maps; S3, inputting the three different scale feature maps into a feature fusion module based on a lightweight adaptive up-sampler to perform feature fusion, and outputting a tearing prediction result including a bounding box prediction, a category prediction and a segmentation mask prediction; S4, inputting the tearing prediction result into a post-processing module to perform post-processing, and outputting a final tearing detection result of the to-be-detected belt surface image.

2. The method of claim 1, wherein, The feature extraction module compresses the input pre-processed original image with a size of HxWx3, H and W being the height and width of the image respectively, through two redundant feature compression convolution operations, reducing the redundant feature information while halving the pixels, and changes the image size to C is the number of channels, and then the feature representation is further refined by the parameter simplified Mamba block. After repeating the process of redundant feature compression convolution and parameter simplified Mamba block twice, a shallow feature map X1 with a size of is generated. The Mamba block process of redundant feature compression convolution and parameter simplification is repeated once to generate a middle layer feature map X2 with a size of ​ After another redundant feature compression convolution, the spatial pyramid pooling layer SPPF block is further used to extract detailed feature information and generate a deep feature map X3 with a size of ​ The three different size feature maps provide rich feature representation for subsequent instance segmentation tasks.

3. The method of claim 1, wherein, The redundant feature compression convolution performs two convolution operations on the input image, the first convolution uses a large convolution kernel to reduce the height and width of the image by half, and the second convolution uses a small convolution kernel to extract preliminary feature information; Then the feature information is subjected to a spatial feature simplification operation by a redundancy space compression module, and a spatially simplified feature map X is output W Then the feature information is subjected to a channel number simplification operation by a redundancy channel compression module, and a channel-simplified feature map Y' is output Finally, the feature map Y' is matched in dimension by using a 1x1 convolution, and a feature map Y after redundant feature compression is output.

4. The method of claim 3, wherein, The redundancy space compression module uses a trainable variable θ in group normalization to evaluate the content of key information in different feature maps, and generates a weight W from the normalized feature maps based on θ i The formula expressions of group normalization and weight calculation are as follows: where X denotes the input feature map, X GN denotes the group-normalized feature map output, W i denotes the corresponding feature map weight, μ and σ denote the mean and standard deviation, θ and ε are trainable variables, and β is a small constant for stability, and a larger θ represents a larger variation between pixels and richer spatial information. The feature map weight W is then obtained i After being nonlinearly transformed by a sigmoid activation function, the range is limited between 0 and 1, and threshold discrimination is performed by using a gate function Gate, so that W1 above the threshold and W2 below the threshold are distinguished, and then multiplied by the corresponding feature maps to obtain feature maps with more feature information and feature maps with less feature information The feature map weight W is then obtained and the feature map weight W is then obtained Four spatial feature subsets are randomly divided, respectively and The subsets with large information amount and the subsets with small information amount are combined with each other through multiplication operation. Finally, the feature splicing avoids excessive small information amount feature map calculation, and outputs the spatially simplified feature map X W , which reduces the redundant features in the spatial dimension.

5. The method of claim 3, wherein, The redundant channel compression module, the spatially reduced feature map X W The channels are divided into two groups according to the number of channels: one group has a channel number of aC, and the other group has a channel number of (1-a)C, where a is a hyperparameter and 0≤a≤1. Then, the channel numbers of the two groups of feature maps are compressed by using 1x1 convolution to obtain X a and X b This design groups the compressed features by channel number, effectively reduces the channel number in the calculation process under the premise of ensuring the richness of the features, and realizes channel reduction. The input X is inputted into the channel key feature extraction module a As the input of the channel key feature extraction, global convolution and point convolution operations are respectively performed, and the output Y is obtained by adding the two 1， Capture the long-distance dependency between features Input X b As a supplement to the channel key feature extraction, a point convolution operation is performed, and the obtained result is spliced with the original input feature to obtain output Y2; Then, Y1 and Y2 are respectively globally pooled, and global spatial information and channel information are combined together; Then, the two pooled results are calculated by a Softmax function to obtain feature weight vectors γ1 and γ2; Finally, the output Y' = γ1Y1 + γ2Y2 is calculated based on the feature weight vectors and Y1, Y2, and Y' is the final output feature map after channel compression.

6. The method of claim 3, wherein, The parameter-reduced Mamba block compresses the output feature map Y of the redundant feature compression convolution, first reduces dimensionality by a 1x1 convolution to reduce computational complexity without significant loss of information, and obtains a feature tensor Y with a size of H'xW'xC' in The feature map is then divided into four groups according to the number of channels, and a feature directional scanning module is applied to each group of feature maps with a size of ​ The feature directional scanning module performs directional scanning along four directions of left to right, right to left, top to bottom and bottom to top along the marker sequence, concatenates the features extracted from the scanned positions in the scanning order to form a one-dimensional feature sequence, reduces the data amount when all features are processed synchronously, and only needs to consider the front and back relationship between sequences when calculating the one-dimensional feature sequence, instead of the relationship on the entire two-dimensional plane, so that the calculation overhead is reduced while maintaining high accuracy; After directional scanning, layer normalization processing is performed to stabilize the feature distribution, then the layer normalized feature sequence is taken as the input of the Mamba sub-block, the feature map dimension is adjusted through linear transformation operation, then the deep separable convolution is used to reduce the calculation amount of the Mamba sub-block while maintaining the feature extraction effect, then the one-dimensional self-attention mechanism 1D-SS in the Mamba sub-block is used to enhance the attention of the key feature area, and the feature sequence is subjected to layer normalization and linear transformation again to be taken as the output of the Mamba sub-block, and this process relies on the lightweight advantage of the one-dimensional attention mechanism calculation process to efficiently and accurately identify deep detail features; Then, the feature sequence output by the Mamba sub-block and the layer normalized feature sequence are combined through residual connection to ensure effective information transmission and prevent the common gradient vanishing or explosion problem in deep networks. The merged feature sequence is normalized again to ensure the stability of the data distribution and make the model converge faster. A feedforward network (FFN) is used to extract and convert the features of the normalized layer to a deeper level. The output of the FFN is added to the previous residual connection result to generate a flattened sequence that combines the input features and high-level semantic information, with a dimension of where B is the batch size, N is the number of tokens in the sequence, N = H' x W'; The flattened sequence is calculated by channel interaction to increase cross-channel information exchange.

7. The method of claim 6, wherein, The flattened sequence is calculated by channel interaction to increase cross-channel information exchange, specifically including: First, the feature tensor graph Y in Average pooling AvgPool is performed to calculate channel statistics and extract global feature information; Next, the channel interaction coefficient Y channel and the target feature map Y out The calculation formula is as follows: Y channel = δ(P2σ(P1 AvgPool(Y in ))) where δ and σ represent two different nonlinear functions, which are used to assign an importance weight for each channel to calculate the feature information correlation of the channel, P1 and P2 are two learnable weights, Y contact is the result of splicing the flattened sequence of the four feature directional scanning modules, and the symbol represents that the elements in the sequence are multiplied by the channel interaction coefficient respectively, and the results are merged and spliced into the target feature map Y out to recalibrate the feature sequence output of the Mamba subblock, and the information interaction between channels and the assignment of importance weights are realized through nonlinear functions and weights, thereby enhancing the sharing of general features between channels. The target feature map Y out The layer-wise normalization and the feed-forward network FFN operation are performed to further enhance the correlation between the features, and the feature tensor Y in The output of the feature extraction of the Mamba block after concatenation as a parameter reduction.

8. The method of claim 1, wherein, The feature fusion module integrates the deep feature map X3, the middle feature map X2 and the shallow feature map X1 to integrate the feature information of different levels, and the process is as follows: The deep feature map X3 is first processed by the lightweight adaptive up-sampler to update the size of the high-level feature information with a small amount of calculation, and then positioned to a higher resolution to splice the middle feature map X2, and then the spliced feature map is transmitted to the C2f block of the YOLOV8 model, and the key information is further extracted from the fused feature map to generate the feature F1, which is then spliced with the shallow feature map X1 to generate the feature F2 after being processed by the lightweight adaptive up-sampler. X3 containing rich abstract local information is input into detection head 1, and X3 is adjusted in feature space layout after 3*3 convolution structure and then input into detection head 2 after being spliced with F1, and the spliced result is input into detection head 3 after being spliced with F2 containing more global information through C2f block and 3*3 convolution. Finally, the three detection heads output tearing prediction results including bounding box prediction, class prediction and segmentation mask prediction.

9. The method of claim 1, wherein, The lightweight adaptive up-sampler realizes dynamic up-sampling by generating an offset, reduces the computational complexity of the up-sampling process, and the process is as follows: First, apply two different linear transformations to the input feature map X, and use a 0.5 times sigmoid function to calculate the result of one of the linear transformations to obtain a reasonable pixel value range to ensure the stability and controllability of the pixel value. Then, multiply the two results and rearrange the pixel values after the pixel shuffle operation to convert the low-resolution feature map into a high-resolution feature map with a very low amount of calculation, which increases the diversity and complexity of the sampling set while improving the up-sampling efficiency. The pixel shuffle obtains the up-sampling offset O, which can adaptively adjust the position of the sampling point, further enhancing the flexibility and accuracy of the sampling set. Then, the up-sampling offset O and the original up-sampling grid are combined to form a new sampling set S, and the input feature map is accurately matched with the sampling set S based on the built-in grid_sample function of Pytorch, which uses the powerful computing power of the grid_sample function to complete the spatial transformation of the feature map. The grid_sample function takes the feature map in the original up-sampling grid and the sampling offset O as input, uses bilinear interpolation to calculate the pixel value of the new position after sampling offset, and then controls the alignment method of the boundary pixels to accurately adjust the pixel positions to obtain the correct mapping of the feature map, finally realizes the dynamic correspondence between the feature map and the sampling set, and generates the up-sampled feature map X'.

10. A lightweight belt tear detection system based on feature information collaborative simplification, characterized in that, The system comprises: An acquisition module configured to acquire a to-be-detected belt surface image in a belt running process; A feature extraction module configured to input the to-be-detected belt surface image into a feature extraction module based on a redundant feature compression convolution and a parameter simplified Mamba block to generate three feature maps of different scales; The feature fusion module is configured to input the three feature maps of different scales into a feature fusion module based on a lightweight adaptive up-sampler to perform feature fusion, and output a tearing prediction result including a bounding box prediction, a category prediction, and a segmentation mask prediction. The post-processing module is configured to input the tearing prediction result into a post-processing module to perform post-processing, and output a final tearing detection result of the to-be-detected belt surface image.

Images