Deep network structure based abrasive plate bonding detection method and system

Abstract

The application discloses a deep network structure abrasive plate joint detection method and system, and a built-in multi-dimensional adaptive feature enhancement module of the detection method, which comprises a multi-scale feature extraction unit, a channel adaptive attention unit, a spatial adaptive attention unit and a residual connection unit. The multi-scale feature extraction unit is responsible for capturing feature information under different granularities. The channel adaptive attention unit strengthens the expression of key feature channels through weight distribution. The spatial adaptive attention unit is used for highlighting the spatial region related to the abrasive plate joint in the feature map. The residual connection unit fuses the enhanced features and the original input, which not only guarantees the effective propagation of the gradient, but also simplifies the learning process of the model. Through the synergistic effect of multi-scale perception and channel spatial double attention mechanism, the module can effectively enhance the weak and irregular feature performance of the abrasive plate joint in the complex background, thereby significantly improving the recognition ability and robustness of the detection system.

Description

Technical Field

[0001] This invention relates to the field of intelligent online inspection, and in particular to a method and system for detecting abrasive slabs with deep network structures. Background Technology

[0002] In high-pressure abrasive waterjet systems, the abrasive box is a key component for abrasive storage and transportation. Its internal environment, such as humidity, temperature fluctuations, the characteristics of the abrasive itself, and the accumulation of fine powder, can easily cause the metal abrasive to oxidize and rust and stick together, forming "caking". Caking leads to problems such as poor abrasive transportation and accelerated equipment wear, which in turn affects the quality of production and processing and shortens the life of equipment.

[0003] Traditional detection methods often rely on periodic manual inspections or indirect assessments by monitoring parameters such as abrasive flow rate and system pressure. These methods are inefficient, have significant time lag, and cannot provide specific feature information about the abrasive caking. Traditional image processing methods struggle to effectively extract and distinguish the complex shapes, irregular textures, and contour features of abrasive caking. Thresholding, edge detection, and morphological operations are highly sensitive to environmental factors such as lighting changes, abrasive dust, and reflections. While general-purpose deep learning networks such as SSD and YOLO can provide a foundation for feature extraction, the features of abrasive caking are often subtle texture variations, local density differences, and slight color unevenness. These are abnormal or defective features that general-purpose networks pay little attention to during training. For non-standard, irregular, and variable targets like abrasive caking, the detection accuracy and robustness are significantly insufficient, making it difficult to meet the requirements of real-time detection. Summary of the Invention

[0004] This invention provides a method and system for detecting abrasive slab knots using a deep network structure, which addresses the technical problem of low accuracy in detecting abrasive slab knots using existing image detection algorithms.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A method for detecting abrasive bridging with a deep network structure includes the following steps:

[0007] The collected abrasive images are input into a pre-trained abrasive caking detection model to obtain the detection results of whether the abrasive is caking.

[0008] The slab compaction detection model includes a multi-scale feature extraction network, a multi-scale feature fusion network, and a prediction network.

[0009] The multi-scale feature extraction network includes multiple feature extraction sub-networks connected in series for extracting features at different scales. The output of each feature extraction sub-network is connected to the input of the multi-scale feature fusion network. Each feature extraction sub-network includes a deep convolution module and a multi-dimensional adaptive feature enhancement module. The output of the deep convolution module is connected to the input of the multi-dimensional adaptive feature enhancement module.

[0010] The multi-dimensional adaptive feature enhancement module includes a multi-scale feature extraction unit, a channel adaptive attention unit, a spatial adaptive attention unit, and a residual connection unit. The multi-scale feature extraction unit obtains input features from the input of the multi-dimensional adaptive feature enhancement module. The output of the multi-scale feature extraction unit is connected to the input of the channel adaptive attention unit, the output of the channel adaptive attention unit is connected to the input of the spatial adaptive attention unit, and the output of the spatial adaptive attention unit is connected to the output of the residual connection unit. The input of the residual connection unit also obtains input features from the input of the multi-dimensional adaptive feature enhancement module and fuses the input features with the features output by the spatial adaptive attention unit.

[0011] Preferably, the multi-scale feature extraction unit includes: multiple parallel convolutional branches of different scales, a channel splicing layer, and a feature fusion convolutional layer;

[0012] The multiple convolutional branches all obtain input features from the input of the multidimensional adaptive feature enhancement module, and their outputs are all connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the feature fusion convolutional layer.

[0013] and / or

[0014] The channel adaptive attention unit includes: a global average pooling layer, a dimensionality-reduced fully connected layer, an activation function layer, an up-dimensional fully connected layer, a channel weight calculation layer, and a channel weighting layer; the global average pooling layer, the dimensionality-reduced fully connected layer, the activation function layer, the up-dimensional fully connected layer, and the channel weight calculation layer are connected sequentially; the input of the global average pooling layer is connected to the output of the multi-scale feature extraction unit; and the input of the channel weighting layer is connected to the output of the channel weight calculation layer and the output of the multi-scale feature extraction unit, respectively.

[0015] and / or

[0016] The spatial adaptive attention unit includes: a channel average pooling layer, a channel max pooling layer, a channel splicing layer, a convolutional layer, a spatial weight calculation layer, and a spatial weighting layer; the channel average pooling layer and the channel max pooling layer are connected in parallel, and their inputs are both connected to the output of the channel adaptive attention unit, and their outputs are both connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the spatial weight calculation layer through the convolutional layer. The outputs of the spatial weight calculation layer and the channel adaptive attention unit are both connected to the input of the spatial weighting layer.

[0017] and / or

[0018] The residual connection unit includes an element-level addition layer, the input of which is connected to the output of the spatial adaptive attention unit and the input of the multidimensional adaptive feature enhancement module.

[0019] Preferably, the multi-scale feature extraction unit includes: a first convolutional branch, a second convolutional branch, a third convolutional branch, a channel splicing layer, and a feature fusion convolutional layer;

[0020] The first convolutional branch, the second convolutional branch, and the third convolutional branch all obtain input features from the input end of the multidimensional adaptive feature enhancement module, and their output ends are all connected to the input end of the channel splicing layer. The output end of the channel splicing layer is connected to the input end of the feature fusion convolutional layer.

[0021] The scale of the first convolutional branch is larger than the scale of the second convolutional branch, the scale of the third convolutional branch is not smaller than the scale of the second convolutional branch, and the third convolutional branch is a dilated convolution.

[0022] Preferably, the multi-scale feature fusion network includes: a top-down fusion path, a bottom-up fusion path, and a context aggregation unit;

[0023] The feature extraction subnetworks are sorted from smallest to largest scale of the output features. The top-down fusion path sequentially upsamples and fuses the features of the feature extraction subnetworks according to the sorting, and sends the upsampled fused features after each upsampling to the bottom-up fusion path. The bottom-up fusion path is used to downsample and fuse the upsampled fused features sequentially from largest to smallest scale, and sends the downsampled fused features after each downsampling, as well as the upsampled fused feature with the largest scale, to the context aggregation unit. The context aggregation unit is used to perform feature enhancement on the sent downsampled fused features and the upsampled fused feature with the largest scale.

[0024] Preferably, the multi-scale feature extraction network includes N+1 feature extraction sub-networks, where N is greater than or equal to 3; the feature extraction sub-networks are sorted from smallest to largest scale of output features, and the top two feature extraction sub-networks are both first feature extraction sub-networks, and the bottom N-1 feature extraction sub-networks are both second feature extraction sub-networks.

[0025] The top-down fusion path includes N upsampling fusion sub-networks, which are divided into one first upsampling fusion sub-network and N-1 second upsampling fusion sub-networks. The output of the first feature extraction sub-network is connected to the input of the first upsampling fusion sub-network. The first upsampling fusion sub-network is used to fuse the output features of the first feature extraction sub-network.

[0026] The N-1 second feature extraction subnetworks correspond one-to-one with the N-1 second upsampling fusion subnetworks, and the output terminals of the second feature extraction subnetworks are all connected to the input terminals of their corresponding second upsampling fusion subnetworks. The second upsampling fusion subnetworks are sorted from smallest to largest scale according to the output feature scale of their corresponding second feature extraction subnetworks. The input terminal of the first-ranked second upsampling fusion subnetwork is also connected to the output terminal of the first upsampling fusion subnetwork, and the input terminals of the other second upsampling fusion subnetworks ranked lower are all connected to the output terminals of their adjacent second upsampling fusion subnetworks ranked higher. The second upsampling fusion subnetwork is used to fuse the output features of its corresponding second feature extraction subnetwork with the output features of its adjacent upsampling fusion subnetwork.

[0027] Preferably, the first upsampling fusion subnetwork includes: two first channel alignment units, a first upsampling unit, a first feature fusion unit, and a first denoising unit; the two first channel alignment units correspond one-to-one with the two first feature extraction subnetworks, and the input end of each first channel alignment unit is connected to the output end of its corresponding first feature extraction subnetwork. Among the two first channel alignment units, the output end of the first channel alignment unit with a larger input feature scale is connected to the input end of the first feature fusion unit, and the output end of the first channel alignment unit with a smaller input feature scale is connected to the input end of the first feature fusion unit through the first upsampling unit. The output end of the first feature fusion unit is connected to the input end of the first denoising unit, and the output end of the first denoising unit is connected to the input end of the second upsampling fusion subnetwork (ranked first) and the input end of the bottom-up fusion path, respectively.

[0028] and / or

[0029] The second upsampling fusion subnetwork includes: a second channel alignment unit, a second upsampling unit, a second feature fusion unit, and a second denoising unit; the input of the second channel alignment unit is connected to the output of the second feature extraction subnetwork corresponding to the second upsampling fusion subnetwork, the output of the second channel alignment unit is connected to the input of the second feature fusion unit, the input of the second upsampling unit is connected to the output of the second upsampling fusion subnetwork that is ranked first and adjacent to it, the output of the second upsampling unit is connected to the input of the second feature fusion unit, the output of the second feature fusion unit is connected to the input of the second denoising unit, and the output of the second denoising unit is connected to the input of the second upsampling unit of the second upsampling fusion subnetwork that is ranked last and adjacent to it, as well as the input of the bottom-up fusion path.

[0030] Preferably, the bottom-up fusion path includes N-1 downsampling fusion sub-networks, and the N-1 downsampling fusion sub-networks include 1 first downsampling fusion sub-network and N-2 second downsampling fusion sub-networks;

[0031] The input terminals of the first downsampling fusion subnetwork are connected to the output terminals of the last two second upsampling fusion subnetworks in the second upsampling fusion subnetwork. The N-2 second downsampling fusion subnetworks correspond one-to-one with the other N-2 upsampling fusion subnetworks in the top-down fusion path, and the input terminal of each second downsampling fusion subnetwork is connected to the output terminal of its corresponding upsampling fusion subnetwork. The second downsampling fusion subnetworks are sorted from largest to smallest according to the input feature scale. The input terminal of the first second downsampling fusion subnetwork is also connected to the output terminal of the first downsampling fusion subnetwork, and the input terminals of other non-first second downsampling fusion subnetworks are also connected to the output terminals of the adjacent and earlier second downsampling fusion subnetworks in the sort.

[0032] Preferably, the first downsampling fusion subnetwork includes: a first downsampling unit, a third feature fusion unit, and a third denoising unit; the input of the first downsampling unit is connected to the output of the second upsampling fusion subnetwork that is last in the second upsampling fusion subnetwork, the output of the first downsampling unit is connected to the input of the third feature fusion unit, the input of the third feature fusion unit is also connected to the output of the second upsampling fusion subnetwork that is second to last in the second upsampling fusion subnetwork, the output of the third feature fusion unit is connected to the input of the third denoising unit, and the output of the third denoising unit is connected to the input of the second downsampling fusion subnetwork that is first in the second upsampling fusion subnetwork;

[0033] and / or

[0034] The second downsampling fusion subnetwork includes a second downsampling unit, a fourth feature fusion unit, and a fourth denoising unit;

[0035] The input of the second downsampling unit is connected to the output of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks first; the output of the second downsampling unit is connected to the input of the fourth feature fusion unit, and the input of the fourth feature fusion unit is also connected to the output of its corresponding upsampling fusion subnetwork; the output of the fourth feature fusion unit is connected to the input of the fourth denoising unit, and the output of the fourth denoising unit is connected to the input of the second downsampling unit of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks last.

[0036] Preferably, the context aggregation unit includes N context aggregation modules, including one first context aggregation module and N-1 second context aggregation modules. The input of the first context aggregation module is connected to the output of the second upsampling fusion subnetwork that is ranked last in the second upsampling fusion subnetwork. The N-1 second context aggregation modules correspond one-to-one with the N-1 downsampling fusion subnetworks, and the input of each second context aggregation module is connected to the output of its corresponding downsampling fusion subnetwork.

[0037] The context aggregation module includes a local feature extraction layer, multiple dilated convolutional layers with different dilation rates, a global pooling layer, a first channel adjustment layer, an upsampling layer, a channel splicing layer, and a second channel adjustment layer.

[0038] A local feature extraction layer, multiple dilated convolutional layers with different dilation rates, and a global pooling layer are connected in parallel. The outputs of the local feature extraction layer and the multiple dilated convolutional layers with different dilation rates are all connected to the input of the channel concatenation layer. The output of the global pooling layer is connected to the input of the channel concatenation layer through a first channel adjustment layer and an upsampling layer in sequence. The output of the channel concatenation layer is connected to the input of the second channel adjustment layer, and the second channel adjustment layer is connected to the input of the prediction network.

[0039] Preferably, the prediction network includes multiple prediction heads, each prediction head comprising: multiple feature reconstruction modules and an output module connected in sequence; the feature reconstruction module comprising: a first convolutional layer, a batch normalization layer, a ReLU activation layer, and a double-layer upsampling unit connected in sequence; the output module comprising: a second convolutional layer, a batch normalization layer, a ReLU activation layer, a third channel adjustment layer, and a sigmoid activation layer connected in sequence.

[0040] and / or

[0041] The plurality of prediction heads includes a detection prediction head and a segmentation prediction head, and the loss function of the slab detection model is:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] in, The total loss is represented by k, which indicates the total number of prediction heads. This represents the loss weight coefficient of the z-th detection prediction head. This represents the detection loss of the z-th detection prediction head. This represents the loss weighting coefficient of the segmentation prediction head. It is the segmentation loss of the segmentation prediction head. Indicates detection loss, For confidence loss, a binary cross-entropy loss is used to determine whether each prediction box contains the target: where It is the total number of predicted locations on the current feature map. This is the actual label; it is 1 if position i contains a slab, and 0 otherwise. The location predicted by the model i Confidence level including compaction, The category classification loss measures the model's ability to correctly identify the abrasive caking category at locations where caking is suspected. It is the total number of predicted locations that are assigned as positive samples. It's the true category label; for the caking category, it's always 1. The location predicted by the model j The probability of belonging to the "hardening" category, This is the bounding box regression loss, used to optimize the position and size between the predicted box and the ground truth box. It uses a loss based on IoU (Intersection over Union). It is the total number of predicted locations assigned as positive samples. It is the k-th bounding box predicted by the model. It is the ground truth bounding box corresponding to the k-th predicted box. It is a standard IoU function. It is the segmentation loss, used to measure the accuracy of the model in predicting whether a pixel belongs to a clump. It is the total number of pixels in the mask image. It is the label of pixel p in the actual mask. It is the probability predicted by the model that pixel p belongs to the caking zone.

[0049] A computer system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0050] The present invention has the following beneficial effects:

[0051] This invention proposes an innovative network structure integrating a multi-dimensional adaptive feature enhancement module, aiming to improve the accuracy of abrasive caking detection in complex backgrounds. This enhancement module consists of four core modules: a multi-scale feature extraction unit (MSFE), a channel-adaptive attention unit (CAA), a spatial adaptive attention unit (SAA), and a residual connection unit. The MSFE captures feature information at different granularities; the CAA strengthens the expression of key feature channels through weight allocation; the SAA highlights spatial regions related to abrasive caking in the feature map; and the residual connection unit fuses the enhanced features with the original input, ensuring effective gradient propagation and simplifying the model's learning process. Through the synergistic effect of multi-scale perception and the channel-spatial dual attention mechanism, this module effectively enhances the weak and irregular feature representation of abrasive caking in complex backgrounds, thereby significantly improving the detection system's recognition capability and robustness.

[0052] In a preferred embodiment, the Multi-Level Context Feature Fusion Module (MCFFM) of this invention constructs an efficient bidirectional feature fusion path and combines it with context aggregation, effectively bridging the semantic gap between features at different levels and providing high-quality fused features for multi-task prediction.

[0053] In a preferred embodiment, the present invention designs a lightweight abrasive slab segmentation head (LSSH) that can accurately depict the pixel-level contour of the abrasive slab while ensuring real-time performance.

[0054] In the preferred embodiment, the present invention systematically solves the problems of insufficient ability to extract abrasive caking features, poor robustness, and inability to provide fine segmentation information in specific industrial scenarios by combining these modules.

[0055] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0056] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0057] Figure 1 This is a simplified structural diagram of the platen filtration detection model (APFDM) in a preferred embodiment of the present invention;

[0058] Figure 2 This is a simplified structural diagram of the Multidimensional Adaptive Feature Enhancement Module (MAFEM) in a preferred embodiment of the present invention;

[0059] Figure 3 This is a simplified structural diagram of the ASPP in a preferred embodiment of the present invention;

[0060] Figure 4 This is a simplified structural diagram of the multi-level context feature fusion module MCFFM in a preferred embodiment of the present invention;

[0061] Figure 5 This is a simplified diagram of the lightweight plate-joint segment head LSSH structure in a preferred embodiment of the present invention. Detailed Implementation

[0062] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0063] Example 1:

[0064] In this embodiment, to address the shortcomings of the prior art, the present invention provides a method for detecting abrasive agglomerates with a deep network structure, comprising the following steps:

[0065] The collected abrasive images are input into a pre-trained abrasive caking detection model to obtain the detection results of whether the abrasive is caking.

[0066] The slab compaction detection model includes a multi-scale feature extraction network, a multi-scale feature fusion network, and a prediction network.

[0067] The multi-scale feature extraction network includes multiple feature extraction sub-networks connected in series for extracting features at different scales. The output of each feature extraction sub-network is connected to the input of the multi-scale feature fusion network. Each feature extraction sub-network includes a deep convolution module and a multi-dimensional adaptive feature enhancement module. The output of the deep convolution module is connected to the input of the multi-dimensional adaptive feature enhancement module.

[0068] The multi-dimensional adaptive feature enhancement module includes a multi-scale feature extraction unit, a channel adaptive attention unit, a spatial adaptive attention unit, and a residual connection unit. The multi-scale feature extraction unit obtains input features from the input of the multi-dimensional adaptive feature enhancement module. The output of the multi-scale feature extraction unit is connected to the input of the channel adaptive attention unit, the output of the channel adaptive attention unit is connected to the input of the spatial adaptive attention unit, and the output of the spatial adaptive attention unit is connected to the output of the residual connection unit. The input of the residual connection unit also obtains input features from the input of the multi-dimensional adaptive feature enhancement module and fuses the input features with the features output by the spatial adaptive attention unit.

[0069] This invention utilizes the synergistic effect of multi-scale perception and channel-space dual attention mechanism to effectively enhance the weak and irregular feature representation of abrasive slabs in complex backgrounds, thereby significantly improving the recognition capability and robustness of the detection system.

[0070] In a preferred embodiment, the multi-scale feature extraction unit includes: multiple parallel convolutional branches of different scales, a channel splicing layer, and a feature fusion convolutional layer;

[0071] The multiple convolutional branches all obtain input features from the input of the multidimensional adaptive feature enhancement module, and their outputs are all connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the feature fusion convolutional layer.

[0072] In a preferred embodiment, the multi-scale feature extraction unit includes: a first convolutional branch, a second convolutional branch, a third convolutional branch, a channel splicing layer, and a feature fusion convolutional layer;

[0073] The first convolutional branch, the second convolutional branch, and the third convolutional branch all obtain input features from the input end of the multidimensional adaptive feature enhancement module, and their output ends are all connected to the input end of the channel splicing layer. The output end of the channel splicing layer is connected to the input end of the feature fusion convolutional layer.

[0074] The scale of the first convolutional branch is larger than the scale of the second convolutional branch, the scale of the third convolutional branch is not smaller than the scale of the second convolutional branch, and the third convolutional branch is a dilated convolution.

[0075] Specifically, the Multi-Scale Feature Extraction Unit (MSFE) includes: a first convolutional branch (1x1 convolution), a second convolutional branch (3x3 convolution), a third convolutional branch (3x3 dilated convolution with a dilation rate of 2), a channel concatenation layer, and a feature fusion convolutional layer. The input features of the multi-dimensional adaptive feature enhancement module are connected to the inputs of the first, second, and third convolutional branches, respectively. The outputs of the first, second, and third convolutional branches are all connected to the input of the channel concatenation layer. The output of the channel concatenation layer is connected to the input of the feature fusion convolutional layer, and the output of the feature fusion convolutional layer is connected to the input of the channel adaptive attention unit.

[0076] The channel adaptive attention unit includes: a global average pooling layer, a dimensionality-reduced fully connected layer (i.e., the first fully connected layer FC1), an activation function layer (ReLU), an increased dimensionality fully connected layer (the second fully connected layer FC2), a channel weight calculation layer (Sigmoid activation layer), and a channel weighting layer (element-wise multiplication layer). The global average pooling layer, the dimensionality-reduced fully connected layer, the activation function layer, the increased dimensionality fully connected layer, and the channel weight calculation layer are connected sequentially. The input of the global average pooling layer is connected to the output of the multi-scale feature extraction unit, and the input of the channel weighting layer is connected to the output of the channel weight calculation layer and the output of the multi-scale feature extraction unit, respectively.

[0077] In this embodiment, the specific channel adaptive attention unit (CAA) includes: a global average pooling layer, a first fully connected layer FC1, an activation function (ReLU), a second fully connected layer FC2, a sigmoid activation layer, and an element-wise multiplication layer. The output of the multi-scale feature extraction unit is connected to the input of the global average pooling layer, the output of the global average pooling layer is connected to the input of the first fully connected layer, the output of the first fully connected layer is connected to the input of the ReLU activation function, the output of the ReLU activation function is connected to the input of the second fully connected layer, the output of the second fully connected layer is connected to the input of the sigmoid activation layer, the output of the sigmoid activation layer is connected to the input of the element-wise multiplication layer, and the input features of the multi-scale feature extraction module (i.e., the output of the multi-scale feature extraction unit) are also connected to the input of the element-wise multiplication layer. The output of the element-wise multiplication layer is connected to the input of the spatial adaptive attention unit.

[0078] The spatial adaptive attention unit includes: a channel average pooling layer, a channel max pooling layer, a channel splicing layer, a convolutional layer (7x7 convolution), a spatial weight calculation layer (Sigmoid activation layer), and a spatial weighting layer (element-wise multiplication layer); the channel average pooling layer and the channel max pooling layer are connected in parallel, and their inputs are both connected to the output of the channel adaptive attention unit. Their outputs are both connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the spatial weight calculation layer through the convolutional layer. The output of the spatial weight calculation layer and the output of the channel adaptive attention unit are both connected to the input of the spatial weighting layer.

[0079] In this embodiment, the Spatial Adaptive Attention Unit (SAA) includes: a channel average pooling layer, a channel max pooling layer, a channel splicing layer, a convolutional layer (7x7 convolution), a sigmoid activation layer, and an element-wise multiplication layer. The output of the SAA is connected to the input of the channel average pooling layer and the channel max pooling layer, respectively. The outputs of the channel average pooling layer and the channel max pooling layer are both connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the 7x7 convolutional layer. The output of the convolutional layer is connected to the input of the sigmoid activation layer. The output of the sigmoid activation layer is connected to the input of the element-wise multiplication layer. Simultaneously, the output of the SAA is also connected to the input of the element-wise multiplication layer, and the output of the element-wise multiplication layer is connected to the input of the residual connection unit.

[0080] The residual connection unit includes an element-level addition layer, the input of which is connected to the output of the spatial adaptive attention unit and the input of the multidimensional adaptive feature enhancement module.

[0081] In this embodiment, the output of the spatial adaptive attention unit is connected to the input of the element-level addition layer, and the original input features of the multidimensional adaptive feature enhancement module are also connected to the input of the element-level addition layer. The output of the element-level addition layer is the enhanced feature output of the multidimensional adaptive feature enhancement module.

[0082] In a preferred embodiment, the multi-scale feature fusion network includes: a top-down fusion path, a bottom-up fusion path, and a context aggregation unit;

[0083] The feature extraction subnetworks are sorted from smallest to largest scale of the output features. The top-down fusion path sequentially upsamples and fuses the features of the feature extraction subnetworks according to the sorting, and sends the upsampled fused features after each upsampling to the bottom-up fusion path. The bottom-up fusion path is used to downsample and fuse the upsampled fused features sequentially from largest to smallest scale, and sends the downsampled fused features after each downsampling, as well as the upsampled fused feature with the largest scale, to the context aggregation unit. The context aggregation unit is used to perform feature enhancement on the sent downsampled fused features and the upsampled fused feature with the largest scale.

[0084] In the preferred embodiment, the multi-scale feature extraction network is assumed to contain N+1 feature extraction sub-networks, where N is greater than or equal to 3; the feature extraction sub-networks are sorted according to the scale of the output features from smallest to largest, and the top two feature extraction sub-networks are both first feature extraction sub-networks, and the bottom N-1 feature extraction sub-networks are both second feature extraction sub-networks.

[0085] The top-down fusion path includes N upsampling fusion sub-networks, which are divided into one first upsampling fusion sub-network and N-1 second upsampling fusion sub-networks. The output of the first feature extraction sub-network is connected to the input of the first upsampling fusion sub-network. The first upsampling fusion sub-network is used to fuse the output features of the first feature extraction sub-network.

[0086] The N-1 second feature extraction subnetworks correspond one-to-one with the N-1 second upsampling fusion subnetworks, and the output terminals of the second feature extraction subnetworks are all connected to the input terminals of their corresponding second upsampling fusion subnetworks. The second upsampling fusion subnetworks are sorted from smallest to largest scale according to the output feature scale of their corresponding second feature extraction subnetworks. The input terminal of the first-ranked second upsampling fusion subnetwork is also connected to the output terminal of the first upsampling fusion subnetwork, and the input terminals of the other second upsampling fusion subnetworks ranked lower are all connected to the output terminals of their adjacent second upsampling fusion subnetworks ranked higher. The second upsampling fusion subnetwork is used to fuse the output features of its corresponding second feature extraction subnetwork with the output features of its adjacent upsampling fusion subnetwork.

[0087] Preferably, the first upsampling fusion subnetwork includes: two first channel alignment units, a first upsampling unit, a first feature fusion unit, and a first denoising unit; the two first channel alignment units correspond one-to-one with the two first feature extraction subnetworks, and the input end of each first channel alignment unit is connected to the output end of its corresponding first feature extraction subnetwork. Among the two first channel alignment units, the output end of the first channel alignment unit with a larger input feature scale is connected to the input end of the first feature fusion unit, and the output end of the first channel alignment unit with a smaller input feature scale is connected to the input end of the first feature fusion unit through the first upsampling unit. The output end of the first feature fusion unit is connected to the input end of the first denoising unit, and the output end of the first denoising unit is connected to the input end of the second upsampling fusion subnetwork (ranked first) and the input end of the bottom-up fusion path, respectively.

[0088] The second upsampling fusion subnetwork includes: a second channel alignment unit, a second upsampling unit, a second feature fusion unit, and a second denoising unit; the input of the second channel alignment unit is connected to the output of the second feature extraction subnetwork corresponding to the second upsampling fusion subnetwork, the output of the second channel alignment unit is connected to the input of the second feature fusion unit, the input of the second upsampling unit is connected to the output of the second upsampling fusion subnetwork that is ranked first and adjacent to it, the output of the second upsampling unit is connected to the input of the second feature fusion unit, the output of the second feature fusion unit is connected to the input of the second denoising unit, and the output of the second denoising unit is connected to the input of the second upsampling unit of the second upsampling fusion subnetwork that is ranked last and adjacent to it, as well as the input of the bottom-up fusion path.

[0089] In a preferred embodiment, the bottom-up fusion path includes N-1 downsampled fusion sub-networks, and the N-1 downsampled fusion sub-networks include one first downsampled fusion sub-network and N-2 second downsampled fusion sub-networks;

[0090] The input terminals of the first downsampling fusion subnetwork are connected to the output terminals of the last two second upsampling fusion subnetworks in the second upsampling fusion subnetwork. The N-2 second downsampling fusion subnetworks correspond one-to-one with the other N-2 upsampling fusion subnetworks in the top-down fusion path, and the input terminal of each second downsampling fusion subnetwork is connected to the output terminal of its corresponding upsampling fusion subnetwork. The second downsampling fusion subnetworks are sorted from largest to smallest according to the input feature scale. The input terminal of the first second downsampling fusion subnetwork is also connected to the output terminal of the first downsampling fusion subnetwork, and the input terminals of other non-first second downsampling fusion subnetworks are also connected to the output terminals of the adjacent and earlier second downsampling fusion subnetworks in the sort.

[0091] The first downsampling fusion subnetwork includes: a first downsampling unit, a third feature fusion unit, and a third denoising unit; the input of the first downsampling unit is connected to the output of the second upsampling fusion subnetwork that is last in the second upsampling fusion subnetwork, the output of the first downsampling unit is connected to the input of the third feature fusion unit, the input of the third feature fusion unit is also connected to the output of the second upsampling fusion subnetwork that is second to last in the second upsampling fusion subnetwork, the output of the third feature fusion unit is connected to the input of the third denoising unit, and the output of the third denoising unit is connected to the input of the second downsampling fusion subnetwork that is first in the second upsampling fusion subnetwork.

[0092] The second downsampling fusion subnetwork includes a second downsampling unit, a fourth feature fusion unit, and a fourth denoising unit;

[0093] The input of the second downsampling unit is connected to the output of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks first; the output of the second downsampling unit is connected to the input of the fourth feature fusion unit, and the input of the fourth feature fusion unit is also connected to the output of its corresponding sampling fusion subnetwork; the output of the fourth feature fusion unit is connected to the input of the fourth denoising unit, and the output of the fourth denoising unit is connected to the input of the second downsampling unit of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks last.

[0094] In this embodiment, the first to fourth denoising units are all 3*3 convolutions, and the first to fourth feature fusion units are all element-level addition modules.

[0095] Specifically, in the preferred solution, such as Figure 4As shown, the Multi-Level Context Feature Fusion (MCFFM) module aims to efficiently integrate features provided by the Adaptive Feature Enhancement (MAFEM) module. Through multi-level context aggregation, it provides comprehensive and semantically rich feature representations for subsequent prediction heads. The MCFFM module is designed as a channel alignment unit, a top-down fusion path, a bottom-up fusion path, and a context aggregation unit. The intermediate outputs of the top-down and bottom-up fusion paths are interconnected and further fused. The final output of the bottom-up fusion path is connected to the input of the context aggregation unit. The output of the context aggregation unit is the final fused feature output of the multi-level context feature fusion module, where:

[0096] Specifically, in this embodiment, the channel alignment unit includes: a first-stage channel alignment convolutional layer, a second-stage channel alignment convolutional layer, a third-stage channel alignment convolutional layer, and a fourth-stage channel alignment convolutional layer; the first-order feature input of the multi-level context feature fusion module is connected to the input of the first-stage channel alignment convolutional layer, the second-order feature input is connected to the input of the second-stage channel alignment convolutional layer, the third-order feature input is connected to the input of the third-stage channel alignment convolutional layer, and the fourth-order feature input is connected to the input of the fourth-stage channel alignment convolutional layer; the outputs of the first, second, third, and fourth-stage channel alignment convolutional layers serve as the corresponding inputs for subsequent fusion paths.

[0097] The top-down fusion path includes: multiple upsampling layers, multiple element-wise summing layers, and multiple 3x3 convolutional layers (including Batch Normalization and ReLU activation); the output of the fourth-stage channel-aligned convolutional layer is element-wise summed with the output of the third-stage channel-aligned convolutional layer through the upsampling layer, and the element-wise summed output is passed through a 3x3 convolutional layer to obtain the third-layer top-down fusion feature; the third-layer top-down fusion feature is element-wise summed with the output of the second-stage channel-aligned convolutional layer through the upsampling layer, and the element-wise summed output is passed through a 3x3 convolutional layer to obtain the second-layer top-down fusion feature; the second-layer top-down fusion feature is element-wise summed with the output of the first-stage channel-aligned convolutional layer through the upsampling layer, and the element-wise summed output is passed through a 3x3 convolutional layer to obtain the first-layer top-down fusion feature.

[0098] The bottom-up fusion path includes: multiple downsampling layers, multiple element-wise addition layers, and multiple 3x3 convolutional layers (containing Batch Normalization and ReLU activation); the first bottom-up fusion feature is element-wise added to the second bottom-up fusion feature through the downsampling layer, and the output of the element-wise addition is passed through a 3x3 convolutional layer to obtain the second bottom-up fusion feature; the second bottom-up fusion feature is element-wise added to the third bottom-up fusion feature through the downsampling layer, and the output of the element-wise addition is passed through a 3x3 convolutional layer to obtain the third bottom-up fusion feature; the first, second, and third bottom-up fusion features are respectively connected to the corresponding inputs of the context aggregation unit.

[0099] Preferably, the context aggregation unit includes N context aggregation modules, including one first context aggregation module and N-1 second context aggregation modules. The input of the first context aggregation module is connected to the output of the second upsampling fusion subnetwork that is ranked last in the second upsampling fusion subnetwork. The N-1 second context aggregation modules correspond one-to-one with the N-1 downsampling fusion subnetworks, and the input of each second context aggregation module is connected to the output of its corresponding downsampling fusion subnetwork.

[0100] The context aggregation module includes a local feature extraction layer, multiple dilated convolutional layers with different dilation rates, a global pooling layer, a first channel adjustment layer, an upsampling layer, a channel splicing layer, and a second channel adjustment layer.

[0101] A local feature extraction layer, multiple dilated convolutional layers with different dilation rates, and a global pooling layer are connected in parallel. The outputs of the local feature extraction layer and the multiple dilated convolutional layers with different dilation rates are all connected to the input of the channel concatenation layer. The output of the global pooling layer is connected to the input of the channel concatenation layer through a first channel adjustment layer and an upsampling layer in sequence. The output of the channel concatenation layer is connected to the input of the second channel adjustment layer, and the second channel adjustment layer is connected to the input of the prediction network.

[0102] In this embodiment, both the first channel adjustment layer and the second channel adjustment layer are 1*1conv.

[0103] In a preferred embodiment, the context aggregation unit comprises: multiple parallel ASPP (Atrous Spatial Pyramid Pooling, see...) Figure 3The ASPP module consists of a 1x1 convolutional branch, three 3x3 dilated convolutional branches with different dilation rates (6, 12, and 18), a global average pooling branch (followed by a 1x1 convolution and upsampling), a feature concatenation layer, and a final 1x1 convolutional layer. The first bottom-up fused feature layer is connected to the input of the first ASPP module, the second ASPP module is connected to the second bottom-up fused feature layer, and the third ASPP module is connected to the third bottom-up fused feature layer. Inside each ASPP module, its input features are connected to the inputs of the 1x1 convolutional branch, multiple 3x3 dilated convolutional branches, and the global average pooling branch. The outputs of all branches are connected to the input of the feature concatenation layer, and the output of the feature concatenation layer is connected to the input of the final 1x1 convolutional layer. The output of the final 1x1 convolutional layer is the fused feature output of that layer. The output of the first ASPP module (highest resolution fused features) is connected to prediction head 3, the output of the second ASPP module (medium resolution fused features) is connected to prediction head 2, and the output of the third ASPP module (lower resolution fused features) is connected to prediction head 1.

[0104] Preferably, the prediction network includes multiple prediction heads, each prediction head comprising: multiple feature reconstruction modules and an output module connected in sequence; the feature reconstruction module comprising: a first convolutional layer, a batch normalization layer, a ReLU activation layer, and a double-layer upsampling unit connected in sequence; the output module comprising: a second convolutional layer, a batch normalization layer, a ReLU activation layer, a third channel adjustment layer, and a sigmoid activation layer connected in sequence.

[0105] In a preferred embodiment, the plurality of prediction heads includes a detection prediction head and a segmentation prediction head, and the loss function of the slab detection model is:

[0106]

[0107]

[0108]

[0109]

[0110]

[0111]

[0112] in, The total loss is represented by k, which indicates the total number of prediction heads. This represents the loss weight coefficient of the z-th detection prediction head. The detection loss of the z-th detection prediction head, This represents the loss weighting coefficient of the segmentation prediction head. It is the segmentation loss of the segmentation prediction head. Indicates detection loss, For confidence loss, a binary cross-entropy loss is used to determine whether each prediction box contains the target: where It is the total number of predicted locations on the current feature map. This is the actual label; it is 1 if position i contains a slab, and 0 otherwise. The location predicted by the model i Confidence level including compaction, The category classification loss measures the model's ability to correctly identify the abrasive caking category at locations where caking is suspected. It is the total number of predicted locations assigned as positive samples (including abrasive agglomerates). It's the true category label; for the caking category, it's always 1. The location predicted by the model j The probability of belonging to the "hardening" category, This is the bounding box regression loss, used to optimize the position and size between the predicted box and the ground truth box. It uses a loss based on IoU (Intersection over Union). It is the total number of predicted locations assigned as positive samples. It is the k-th bounding box predicted by the model. It is the ground truth bounding box corresponding to the k-th predicted box. It is a standard IoU function. It is the segmentation loss, used to measure the accuracy of the model in predicting whether a pixel belongs to a clump. It is the total number of pixels in the mask image. It is the label of pixel p in the actual mask. It is the probability predicted by the model that pixel p belongs to the caking zone.

[0113] In a preferred embodiment, the prediction network includes a lightweight knot segmentation head (LSSH) that generates a pixel-accurate shape mask for each detected knot, such as... Figure 5As shown, the input feature map P3 is connected to the input layer of the first 3x3 Conv convolutional layer. The output of the first 3x3 Conv convolutional layer is connected to the first upsampling layer. The output of the first upsampling layer is connected to the input layer of the second 3x3 Conv convolutional layer. The output of the second 3x3 Conv convolutional layer is connected to the input layer of the second upsampling layer. The output of the second upsampling layer is connected to the input layer of the third 3x3 Conv convolutional layer. The output of the third 3x3 convolutional layer is connected to the input layer of the 1x1 Conv convolutional layer. The output of the 1x1 Conv convolutional layer is connected to the input layer of the sigmoid activation layer. The output of the sigmoid activation layer is the output of LSSH.

[0114] Example 2:

[0115] To address the shortcomings of existing technologies, this invention provides an abrasive slab caking detection model (APFDM). This model consists of a multi-stage convolutional backbone network, a multi-dimensional adaptive feature enhancement module (MAFEM), a multi-level contextual feature fusion module (MCFFM), and multiple prediction heads. Through layered and customized design, the model improves its ability to capture and process features such as the unique aggregate texture of slabs, color and gloss changes caused by oxidation, and shadows caused by dust or abrasive accumulation, thereby enhancing the accuracy and stability of predictions.

[0116] like Figure 1 As shown, the Abrasive Junction Detection Model (APFDM) consists of a multi-stage convolutional backbone network, a multi-dimensional adaptive feature enhancement module (MAFEM), a multi-level contextual feature fusion module (MCFFM), and three prediction heads. The backbone network, prediction head 1, and prediction head 2 can be implemented using existing technologies. The multi-dimensional adaptive feature enhancement module (MAFEM), the multi-level contextual feature fusion module (MCFFM), and prediction head 3 (Lightweight Junction Segmentation Head LSSH) are specifically designed for feature extraction and fusion of abrasive junctions in this invention. Figure 1 As shown:

[0117] The network structure uses a four-stage stacked deep convolutional neural network as the backbone feature extractor. After the convolution operation of each stage, a multidimensional adaptive feature enhancement module (MAFEM) is connected. This module is designed to adaptively enhance the features of the current stage to capture subtle caking features and improve robustness in harsh industrial environments. The output of MAFEM is the enhanced feature map (C1-C4) of the corresponding stage.

[0118] Feature maps C1-C4 are fed into the Multi-Level Contextual Feature Fusion (MCFFM) module. MCFFM employs a bidirectional (top-down and bottom-up) fusion path and combines it with a context aggregation mechanism to fully integrate multi-scale features and generate semantically rich and precisely localized fused features.

[0119] Finally, the fused features output by MCFFM are distributed to three parallel prediction heads to achieve multi-task detection: Prediction Head 1 and Prediction Head 2 are decoupled detection heads that predict slabs at different scales and output bounding boxes (Rect) and confidence scores for metal abrasive slabs, while Prediction Head 3 (LSSH) provides finer bounding boxes (Rect) and pixel-level segmentation masks to achieve accurate identification of slab regions.

[0120] by Figure 2 Explaining the working principle and functions of the MAFEM module:

[0121] The MAFEM module is designed to consist of four parts: a multi-scale feature extraction unit (MSFE), a channel adaptive attention unit (CAA), a spatial adaptive attention unit (SAA), and a residual connection unit. The multi-scale feature extraction unit is responsible for capturing feature information from different scales, the channel adaptive attention unit is responsible for analyzing and adjusting the importance of different feature channels, the spatial adaptive attention unit is responsible for analyzing and adjusting the importance of different spatial locations on the feature map, and the residual connection unit is responsible for fusing the enhanced features with the original features to ensure information flow and simplify the learning process. The data flow within MAFEM is a series enhancement and final fusion process. The original feature maps are first fed into the Multi-Scale Feature Extraction Unit (MSFE), and after parallel processing, they are fused into a feature map Fms rich in multi-scale information. Fms is then fed into the Channel Adaptive Attention Unit (CAA), which calculates the weight of each channel and weights Fms to generate the channel-enhanced feature map Fca. Fca is then fed into the Spatial Adaptive Attention Unit (SAA), which calculates the weight of each spatial location and weights Fca to generate the final enhanced feature map Fsa. The Residual Connection Unit adds Fsa to the original input of the module element-wise, and the result Xenhanced is used as the final output of the MAFEM module and passed to the next layer of the network.

[0122] The function of MSFE is to simultaneously capture information at different scales from input features. Metal abrasive agglomerates exhibit diverse morphologies and sizes, ranging from fine particle adhesion to large-area blocky formations. Multi-scale extraction ensures the model has good perception capabilities for agglomerates of different sizes, achieved through three parallel convolutional branches: a 1x1 convolutional branch for rapid inter-channel information interaction, a 3x3 convolutional branch for extracting standard local features, and a 3x3 dilated convolutional branch to expand the receptive field and capture a wider range of contextual information without increasing computational cost, which is particularly effective for large, irregularly shaped agglomerates with blurred boundaries. Channel concatenation combines the feature maps extracted from the three branches at different scales along the channel dimension, forming a larger feature map containing rich multi-scale information. 1×1 Conv (fusion) performs a 1x1 convolution on the concatenated feature map, serving two main purposes: first, to effectively fuse multi-scale information; and second, to adjust the number of channels back to a standard value for subsequent module processing.

[0123] The function of the Channel Adaptive Attention Unit (CAA) is to learn and determine which feature channels are more important. In the industrial environment of abrasive slag detection, different feature channels may contain varying degrees of effective information and noise. CAA can automatically assign higher weights to more useful channels and suppress useless channels, achieving adaptive feature selection. Global Average Pooling (GAP) compresses the spatial dimension of the feature map output by MSFE on each channel, compressing an H×W two-dimensional plane into a single value to obtain a channel descriptor representing the global information of each channel; FC1 (dimensionality reduction) -> ReLU -> FC2 (dimensionality increase) is a bottleneck structure. Through two fully connected layers (FC) and a non-linear activation function (ReLU), the complex correlations between channels are learned, reducing the number of parameters and preventing overfitting. The Sigmoid (channel weight Wc) transforms the learned channel correlation information into a weight vector ranging from 0 to 1. Element-wise multiplication (channel weighting) multiplies the calculated channel weight vector with the original output of MSFE (i.e., the feature map entering the CAA module) channel by channel, amplifying the feature responses of important channels and shrinking the feature responses of unimportant channels.

[0124] The Spatial Adaptive Attention Unit (SAA) learns and determines which spatial locations in the feature map are more important, allowing the model to focus on areas with potential congestion while ignoring irrelevant background areas. Channel average pooling and channel max pooling compress the feature map output by CAA along the channel dimension. Average pooling captures smooth feature information for each spatial location, while max pooling captures the most salient feature information. Combining the two provides a more comprehensive spatial feature description. Channel concatenation combines the two spatial feature maps obtained from the above two pooling operations. The 7×7 Conv uses a large convolutional kernel (7x7) to convolve the concatenated spatial feature map to capture broader spatial dependencies. Element-wise multiplication (spatial weighting) multiplies this spatial weight map element-wise with the original output of CAA, amplifying the features of important regions in the image and suppressing the features of unimportant regions.

[0125] Residual connections can preserve the original information, and the original feature information is not completely lost even after a series of complex augmentation operations. In addition, they alleviate gradient vanishing and make the network easier to train.

[0126] by Figure 4 Explain the working principle and function of the MCFFM module:

[0127] The core objective of MCFFM is to receive four feature maps (C1, C2, C3, C4) from different layers of the backbone network and, through a bidirectional (top-down and bottom-up) fusion path, ultimately generate three fully fused and enhanced multi-scale feature maps (P1, P2, P3) for use by the subsequent prediction head. The entire process can be divided into four main stages:

[0128] Channel alignment: This unit unifies the number of channels across all input features. Data starts from the four inputs on the left: C1, C2, C3, and C4. These four inputs represent feature maps extracted from different depths of the backbone network. C1 comes from a shallower layer, has the highest resolution, and contains rich spatial details and edge information, but weaker semantic information. C4 comes from the deepest layer, has the lowest resolution, but contains the strongest global semantic information. Before feature fusion, the channel alignment unit must ensure that the feature maps participating in the computation have the same number of channels. This unit (which can use a 1x1 convolutional layer) is responsible for adjusting the number of channels in C1, C2, C3, and C4 to a preset value.

[0129] Top-down fusion path: This approach passes semantic information from higher layers to lower layers, enhancing the semantic expression of features. Starting with channel-aligned C4, F4_td is generated. The resolution of F4_td is upsampled (bilinear interpolation) to double, making it consistent with the spatial size of C3. The upsampled F4_td is then added pixel-by-pixel to the channel-aligned C3, achieving the first feature fusion and injecting the semantic information of C4 into the C3 layer. A 3x3 convolution is performed on the added feature map to smooth and refine the fused features, eliminating the aliasing effect that may be caused by upsampling. The output is F3_td. Similarly, F3_td is upsampled and added to aligned C2, then convolved with 3x3 to obtain F2_td. F2_td is then upsampled and added to aligned C1, then convolved with 3x3 to obtain the final F1_td.

[0130] Bottom-up fusion path: The localization information of the lower layer is passed to the higher layer to enhance the localization accuracy of the features. Starting from F1_td, it is directly used as F1_Bu. The resolution of F1_Bu is reduced by half through downsampling (convolution with stride of 2) to make it consistent with the spatial size of F2_td. The downsampled F1_Bu is added to F2_td generated in the previous stage pixel by pixel. The purpose is to supplement the accurate localization information of F1 layer to F2 layer. The feature map after addition is subjected to 3x3 convolution to refine the features, and the output result is F2_Bu. Repeat the process: F2_Bu is downsampled and added to F3_td, and then subjected to 3x3 convolution to obtain F3_Bu.

[0131] Context aggregation and output: ASPP is used to further enhance the fused features. F1_Bu, F2_Bu, and F3_Bu generated by the bottom-up path are fed into their respective ASPP modules and output to the prediction head. ASPP, through parallel convolutional kernels with different dilation rates, can detect features with multiple different receptive fields without changing the feature map resolution. This helps to identify metal abrasive slabs of different sizes and irregular shapes, and can simultaneously focus on the local details and overall outline of the slabs.

[0132] Training and use of the Abrasive Bonding Detection Model (APFDM):

[0133] A. Data preparation and annotation:

[0134] A large number of images from inside the abrasive box of the high-pressure abrasive waterjet system were collected, covering various working conditions, lighting conditions, abrasive states (normal flow, initial caking, severe caking, different caking shapes and sizes), and potential interferences (such as dust, reflection, vibration blurring). The collected images were finely annotated. For each instance of metal abrasive caking in the image, its location was accurately marked with a rectangular bounding box. For each caking instance, pixel-level segmentation and annotation were performed to delineate its precise shape and outline. The annotated dataset was divided into training set, validation set, and test set according to 70%:15%:15%.

[0135] B. Model Initialization:

[0136] Pre-trained backbone network: The backbone network is YOLO, which is pre-trained on the large-scale general image dataset (ImageNet), and MAFEM, MCFFM and three prediction heads are randomly initialized.

[0137] C. Define the loss function

[0138] Since the model includes two tasks: detection and segmentation, and the detection task itself includes classification, confidence, and regression, a multi-task joint loss function is defined, and the total loss function is...

[0139]

[0140] in , , These are weighting coefficients that balance the losses from different tasks. It is the detection loss of the first prediction. It is the detection loss of the first two predictions. It is the segmentation loss for predicting the first 3, where, and They have the same structure, that is

[0141]

[0142] For confidence loss, binary cross-entropy loss is used to determine whether each prediction box contains the target:

[0143]

[0144] in It is the total number of predicted locations on the current feature map. This is the actual label; it is 1 if position i contains a slab, and 0 otherwise. The location predicted by the model i Confidence level including compaction,

[0145] The category classification loss measures the model's ability to correctly identify the abrasive caking category at locations where caking is suspected.

[0146]

[0147] in It is the total number of predicted locations assigned as positive samples (including abrasive agglomerates). It's the true category label; for the caking category, it's always 1. It is the probability that position j, as predicted by the model, belongs to the "solidification" category;

[0148] This is the bounding box regression loss, used to optimize the position and size between the predicted box and the ground truth box, using a loss based on IoU (Intersection over Union).

[0149]

[0150] in It is the total number of predicted locations assigned as positive samples. It is the k-th bounding box predicted by the model. It is the ground truth bounding box corresponding to the k-th predicted box. It is a standard IoU function;

[0151] It is the segmentation loss, used to measure the accuracy of the model in predicting whether a pixel belongs to a clump:

[0152]

[0153] in, It is the total number of pixels in the mask image. It is the label of pixel p in the actual mask. It is the probability predicted by the model that pixel p belongs to the caking zone.

[0154] D model training

[0155] The training data is divided into several small batches, with each iteration processing one batch (epoch). Before each training batch is input into the model, the images are randomly transformed to increase data diversity and improve the model's generalization ability and robustness. General enhancements such as random cropping, random flipping (horizontal / vertical), random scaling, rotation, color jittering (brightness, contrast, saturation, hue), and Gaussian blur are used. Industrial scene-specific enhancements such as simulating dust, simulating uneven lighting (randomly adding highlights / shadows), simulating vibration blur, and adding Gaussian noise are used to improve the model's robustness in harsh industrial environments. The enhanced image batches are input into the model through forward propagation, with the data flowing sequentially through the backbone network, MAFEM, and MCFFM, finally reaching three prediction heads to generate their respective prediction results (bounding boxes, confidence scores, class scores, and segmentation masks). The model's prediction results are compared with the corresponding ground truth labels, and the total loss is calculated according to a predefined loss function. Based on the calculated total loss, the gradients of all trainable parameters in the model are calculated using the backpropagation algorithm. The optimizer updates the model's weight parameters based on the calculated gradient and learning rate, thereby reducing the loss function value. This process continues through multiple iterations until the model performance converges or reaches the preset maximum number of epochs.

[0156] E. Evaluation and Validation

[0157] During training, model performance is periodically evaluated on the validation set every N epochs to monitor training progress and prevent overfitting. Detection metrics include: mAP (Mean Average Precision), which measures the model's average precision across different IoU (Intersection over Union) thresholds; Precision and Recall, which measure the model's accuracy in prediction and its ability to detect all targets, respectively; and F1-Score: the harmonic mean of precision and recall. mIoU (Mean Intersection over Union) measures the degree of overlap between the predicted and ground truth masks.

[0158] Table 1 shows a comparison of the performance indicators of the abrasive blister detection model (APFDM) of this invention with SSD, YOLOv5, and Mask R-CNN on the abrasive blister detection task:

[0159] Table 1 Comparison of various performance indicators in abrasive caking detection tasks

[0160]

[0161] General-purpose models (SSD, YOLO, Mask R-CNN) excel in their respective domains (speed or accuracy), but they are designed for general scenarios. As can be seen, they all have their shortcomings when facing the specific and complex industrial problem of abrasive caking. This invention, APFDM, through customized modules such as MAFEM and MCFFM, accurately solves the core pain points in caking detection, such as weak features, irregular shapes, multi-scale processing, and environmental interference. It not only improves detection and segmentation accuracy compared to traditional models but also achieves a more economical balance in inference speed.

[0162] In summary, the detection model in this invention can keenly capture the initial and subtle features of metal abrasive agglomeration. Traditional methods or general networks may ignore early adhesion that is very similar in color and texture to the surrounding normal abrasive. The MAFEM module in this invention can effectively enhance and identify this early adhesion through its multi-scale feature extraction and adaptive attention mechanism.

[0163] The bidirectional fusion path of the MCFFM of this invention ensures that the final feature map used for prediction contains both high-level semantic information and retains the fine edge information of the low level, overcoming the limitations of traditional networks in processing multi-scale targets. Whether it is a large area of ​​severe compaction or a small, scattered compaction, the model can accurately identify and locate it.

[0164] The model outputs no longer a single signal, but a multi-dimensional diagnostic report. Predictive heads 1 and 2 determine the existence and location of the slabs, while predictive head 3 can further calculate the size and shape of the slabs, providing more detailed information for system monitoring and diagnosis.

[0165] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting abrasive agglomeration with a deep network structure, characterized in that, Includes the following steps: The collected abrasive images are input into a pre-trained abrasive caking detection model to obtain the detection results of whether the abrasive is caking. The slab formation detection model includes a multi-scale feature extraction network, a multi-scale feature fusion network, and a prediction network. The multi-scale feature extraction network includes multiple feature extraction sub-networks connected in series for extracting features at different scales. The output of each feature extraction sub-network is connected to the input of the multi-scale feature fusion network. Each feature extraction sub-network includes a deep convolution module and a multi-dimensional adaptive feature enhancement module. The output of the deep convolution module is connected to the input of the multi-dimensional adaptive feature enhancement module. The multi-dimensional adaptive feature enhancement module includes a multi-scale feature extraction unit, a channel adaptive attention unit, a spatial adaptive attention unit, and a residual connection unit. The multi-scale feature extraction unit obtains input features from the input of the multi-dimensional adaptive feature enhancement module. The output of the multi-scale feature extraction unit is connected to the input of the channel adaptive attention unit, the output of the channel adaptive attention unit is connected to the input of the spatial adaptive attention unit, and the output of the spatial adaptive attention unit is connected to the output of the residual connection unit. The input of the residual connection unit also obtains input features from the input of the multi-dimensional adaptive feature enhancement module and fuses the input features with the features output by the spatial adaptive attention unit. The multi-scale feature fusion network includes: a top-down fusion path, a bottom-up fusion path, and a context aggregation unit; The feature extraction subnetworks are sorted from smallest to largest scale of the output features. The top-down fusion path sequentially upsamples and fuses the features of the feature extraction subnetworks according to the sorting, and sends the upsampled and fused features after each upsampled fusion to the bottom-up fusion path. The bottom-up fusion path is used to downsample and fuse the upsampled and fused features sequentially from largest to smallest scale, and sends the downsampled and fused features after each downsampled fusion, as well as the upsampled and fused features with the largest scale, to the context aggregation unit. The context aggregation unit is used to perform feature enhancement on the sent downsampled and fused features and the upsampled and fused features with the largest scale. The prediction network includes multiple prediction heads, each prediction head comprising: multiple feature reconstruction modules and an output module connected in sequence; the feature reconstruction module comprising: a first convolutional layer, a batch normalization layer, a ReLU activation layer, and a double-layer upsampling unit connected in sequence; the output module comprising: a second convolutional layer, a batch normalization layer, a ReLU activation layer, a third channel adjustment layer, and a sigmoid activation layer connected in sequence. The plurality of prediction heads includes a detection prediction head and a segmentation prediction head, and the loss function of the slab detection model is: in, The total loss is represented by k, which indicates the total number of prediction heads. This represents the loss weight coefficient of the z-th detection prediction head. This represents the detection loss of the z-th detection prediction head. This represents the loss weighting coefficient of the segmentation prediction head. It is the segmentation loss of the segmentation prediction head. Indicates detection loss, For confidence loss, a binary cross-entropy loss is used to determine whether each prediction box contains the target: where It is the total number of predicted locations on the current feature map. This is the actual label; it is 1 if position i contains a slab, and 0 otherwise. The location predicted by the model i Confidence level including compaction, The category classification loss measures the model's ability to correctly identify the abrasive caking category at locations where caking is suspected. It is the total number of predicted locations assigned as positive samples. These are the actual category labels; for the caking category, it's always 1. The location predicted by the model j The probability of belonging to the "hardening" category, It's a bounding box regression loss, used to optimize the position and size between the predicted box and the ground truth box, using an IoU-based loss. It is the total number of predicted locations assigned as positive samples. It is the k-th bounding box predicted by the model. It is the ground truth bounding box corresponding to the k-th predicted box. It is a standard IoU function. It is the segmentation loss, used to measure the accuracy of the model in predicting whether a pixel belongs to a clump. It is the total number of pixels in the mask image. It is the label of pixel p in the actual mask. It is the probability that pixel p, predicted by the model, belongs to the caking zone.

2. The method for detecting abrasive agglomeration in a deep network structure according to claim 1, characterized in that, The multi-scale feature extraction unit includes: multiple parallel convolutional branches of different scales, channel splicing layers, and feature fusion convolutional layers; The multiple convolutional branches all obtain input features from the input of the multidimensional adaptive feature enhancement module, and their outputs are all connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the feature fusion convolutional layer. and / or The channel adaptive attention unit includes: a global average pooling layer, a dimensionality-reduced fully connected layer, an activation function layer, an up-dimensional fully connected layer, a channel weight calculation layer, and a channel weighting layer; the global average pooling layer, the dimensionality-reduced fully connected layer, the activation function layer, the up-dimensional fully connected layer, and the channel weight calculation layer are connected sequentially; the input of the global average pooling layer is connected to the output of the multi-scale feature extraction unit; and the input of the channel weighting layer is connected to the output of the channel weight calculation layer and the output of the multi-scale feature extraction unit, respectively. and / or The spatial adaptive attention unit includes: a channel average pooling layer, a channel max pooling layer, a channel splicing layer, a convolutional layer, a spatial weight calculation layer, and a spatial weighting layer; the channel average pooling layer and the channel max pooling layer are connected in parallel, and their inputs are both connected to the output of the channel adaptive attention unit, and their outputs are both connected to the input of the channel splicing layer. The output of the channel splicing layer is connected to the input of the spatial weight calculation layer through the convolutional layer. The outputs of the spatial weight calculation layer and the channel adaptive attention unit are both connected to the input of the spatial weighting layer. and / or The residual connection unit includes an element-level addition layer, the input of which is connected to the output of the spatial adaptive attention unit and the input of the multidimensional adaptive feature enhancement module.

3. The method for detecting abrasive agglomeration in a deep network structure according to claim 2, characterized in that, The multi-scale feature extraction unit includes: a first convolutional branch, a second convolutional branch, a third convolutional branch, a channel splicing layer, and a feature fusion convolutional layer; The first convolutional branch, the second convolutional branch, and the third convolutional branch all obtain input features from the input end of the multidimensional adaptive feature enhancement module, and their output ends are all connected to the input end of the channel splicing layer. The output end of the channel splicing layer is connected to the input end of the feature fusion convolutional layer. The scale of the first convolutional branch is larger than the scale of the second convolutional branch, the scale of the third convolutional branch is not smaller than the scale of the second convolutional branch, and the third convolutional branch is a dilated convolution.

4. The method for detecting abrasive agglomeration in a deep network structure according to claim 1, characterized in that, Suppose that the multi-scale feature extraction network contains N+1 feature extraction sub-networks, where N is greater than or equal to 3; sort the feature extraction sub-networks according to the scale of the output features from smallest to largest, and assume that the top two feature extraction sub-networks are the first feature extraction sub-networks, and the bottom N-1 feature extraction sub-networks are the second feature extraction sub-networks. The top-down fusion path includes N upsampling fusion sub-networks, which are divided into one first upsampling fusion sub-network and N-1 second upsampling fusion sub-networks. The output of the first feature extraction sub-network is connected to the input of the first upsampling fusion sub-network. The first upsampling fusion sub-network is used to fuse the output features of the first feature extraction sub-network. The N-1 second feature extraction subnetworks correspond one-to-one with the N-1 second upsampling fusion subnetworks, and the output terminals of the second feature extraction subnetworks are all connected to the input terminals of their corresponding second upsampling fusion subnetworks. The second upsampling fusion subnetworks are sorted from smallest to largest scale according to the output feature scale of their corresponding second feature extraction subnetworks. The input terminal of the first-ranked second upsampling fusion subnetwork is also connected to the output terminal of the first upsampling fusion subnetwork, and the input terminals of the other second upsampling fusion subnetworks ranked lower are all connected to the output terminals of their adjacent second upsampling fusion subnetworks ranked higher. The second upsampling fusion subnetwork is used to fuse the output features of its corresponding second feature extraction subnetwork with the output features of its adjacent upsampling fusion subnetwork.

5. The method for detecting abrasive agglomeration in a deep network structure according to claim 4, characterized in that, The first upsampling fusion subnetwork includes: two first channel alignment units, a first upsampling unit, a first feature fusion unit, and a first denoising unit; the two first channel alignment units correspond one-to-one with the two first feature extraction subnetworks, and the input of each first channel alignment unit is connected to the output of its corresponding first feature extraction subnetwork. Among the two first channel alignment units, the output of the first channel alignment unit with the larger input feature scale is connected to the input of the first feature fusion unit, and the output of the first channel alignment unit with the smaller input feature scale is connected to the input of the first feature fusion unit through the first upsampling unit. The output of the first feature fusion unit is connected to the input of the first denoising unit, and the output of the first denoising unit is connected to the input of the second upsampling fusion subnetwork (ranked first) and the input of the bottom-up fusion path, respectively. and / or The second upsampling fusion subnetwork includes: a second channel alignment unit, a second upsampling unit, a second feature fusion unit, and a second denoising unit; the input of the second channel alignment unit is connected to the output of the second feature extraction subnetwork corresponding to the second upsampling fusion subnetwork, the output of the second channel alignment unit is connected to the input of the second feature fusion unit, the input of the second upsampling unit is connected to the output of the second upsampling fusion subnetwork that is ranked first and adjacent to it, the output of the second upsampling unit is connected to the input of the second feature fusion unit, the output of the second feature fusion unit is connected to the input of the second denoising unit, and the output of the second denoising unit is connected to the input of the second upsampling unit of the second upsampling fusion subnetwork that is ranked last and adjacent to it, as well as the input of the bottom-up fusion path.

6. The method for detecting abrasive crusts with a deep network structure according to claim 4, characterized in that, The bottom-up fusion path includes N-1 downsampling fusion sub-networks, and the N-1 downsampling fusion sub-networks include 1 first downsampling fusion sub-network and N-2 second downsampling fusion sub-networks; The input terminals of the first downsampling fusion subnetwork are connected to the output terminals of the last two second upsampling fusion subnetworks in the second upsampling fusion subnetwork. The N-2 second downsampling fusion subnetworks correspond one-to-one with the other N-2 upsampling fusion subnetworks in the top-down fusion path, and the input terminal of each second downsampling fusion subnetwork is connected to the output terminal of its corresponding upsampling fusion subnetwork. The second downsampling fusion subnetworks are sorted from largest to smallest according to the input feature scale. The input terminal of the first second downsampling fusion subnetwork is also connected to the output terminal of the first downsampling fusion subnetwork, and the input terminals of other non-first second downsampling fusion subnetworks are also connected to the output terminals of the adjacent and earlier second downsampling fusion subnetworks in the sort.

7. The method for detecting abrasive agglomeration in a deep network structure according to claim 6, characterized in that, The first downsampling fusion subnetwork includes: a first downsampling unit, a third feature fusion unit, and a third denoising unit; the input of the first downsampling unit is connected to the output of the second upsampling fusion subnetwork that is last in the second upsampling fusion subnetwork, the output of the first downsampling unit is connected to the input of the third feature fusion unit, the input of the third feature fusion unit is also connected to the output of the second upsampling fusion subnetwork that is second to last in the second upsampling fusion subnetwork, the output of the third feature fusion unit is connected to the input of the third denoising unit, and the output of the third denoising unit is connected to the input of the second downsampling fusion subnetwork that is first in the second upsampling fusion subnetwork. and / or The second downsampling fusion subnetwork includes a second downsampling unit, a fourth feature fusion unit, and a fourth denoising unit; The input of the second downsampling unit is connected to the output of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks first; the output of the second downsampling unit is connected to the input of the fourth feature fusion unit, and the input of the fourth feature fusion unit is also connected to the output of its corresponding upsampling fusion subnetwork; the output of the fourth feature fusion unit is connected to the input of the fourth denoising unit, and the output of the fourth denoising unit is connected to the input of the second downsampling unit of the second downsampling fusion subnetwork that is adjacent to it in the order and ranks last.

8. The method for detecting abrasive crusts with a deep network structure according to claim 6, characterized in that, The context aggregation unit includes N context aggregation modules, including one first context aggregation module and N-1 second context aggregation modules. The input of the first context aggregation module is connected to the output of the second upsampling fusion subnetwork that is the last one in the second upsampling fusion subnetwork. The N-1 second context aggregation modules correspond one-to-one with the N-1 downsampling fusion subnetworks, and the input of each second context aggregation module is connected to the output of its corresponding downsampling fusion subnetwork. The context aggregation module includes a local feature extraction layer, multiple dilated convolutional layers with different dilation rates, a global pooling layer, a first channel adjustment layer, an upsampling layer, a channel splicing layer, and a second channel adjustment layer. A local feature extraction layer, multiple dilated convolutional layers with different dilation rates, and a global pooling layer are connected in parallel. The outputs of the local feature extraction layer and the multiple dilated convolutional layers with different dilation rates are all connected to the input of the channel concatenation layer. The output of the global pooling layer is connected to the input of the channel concatenation layer through a first channel adjustment layer and an upsampling layer in sequence. The output of the channel concatenation layer is connected to the input of the second channel adjustment layer, and the second channel adjustment layer is connected to the input of the prediction network.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images