A dust screen detection method and system based on multi-scale state space pyramid and space-semantics bidirectional fusion

Abstract

The application discloses a dust screen detection method and system based on multi-scale state space pyramid and space-semantics bidirectional fusion. The method realizes low-contrast target detection based on a trained deep learning network model. The model responds to an input image, obtains a basic feature tensor through convolution, and then obtains first fusion features by adaptively fusing local grid texture features and global contour features through a stacked VMC3k2 local-global pyramid module. Spatial features and semantic features in the first fusion features are bidirectionally interactively enhanced by a space-semantics bidirectional fusion module to obtain second fusion features. The second fusion features are predicted by a detection head structure to obtain a bounding box and a class confidence of a dust screen target. The application can realize efficient positioning and identification of low-contrast dust screens.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and remote sensing detection technology, and in particular to a dust net detection method and system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion. Background Technology

[0002] In construction, environmental protection, and industrial warehousing, the proper coverage and damage monitoring of dust control netting are crucial for assessing environmental compliance and site safety. Due to the unique materials used in dust control netting, it generally exhibits low contrast, semi-transparency, and susceptibility to glare, making it difficult to locate quickly with the naked eye in the complex backgrounds of construction sites. Traditional manual inspection methods are not only time-consuming and labor-intensive but also prone to omissions and misjudgments in outdoor environments with drastically changing lighting. Therefore, introducing automated inspection technology based on computer vision to achieve efficient, all-weather intelligent monitoring of dust control netting has significant engineering application value.

[0003] Relying on end-to-end feature extraction and high inference speed, deep learning models, led by the YOLO series, have become mainstream in industrial automation inspection. However, when faced with a specific target like dust nets, existing deep learning detection models reveal significant limitations: On the one hand, traditional convolutional neural networks (CNNs), limited by their inherent local receptive fields, struggle to establish long-distance pixel-level dependencies. This results in a lack of global context aids when the contrast between the dust net and the background is extremely weak, easily confusing the target with the complex background. On the other hand, existing network feature fusion strategies are relatively simple. When dealing with the drastic changes in the dust net from the largest near-end scale to the smallest far-end scale, they struggle to effectively balance the fine spatial location at the lower level with the abstract category semantics at the higher level, inevitably introducing a large amount of background noise. It is worth noting that although recently emerging state-space models (SSMs, such as Mamba) have the potential to overcome the bottleneck of long-term dependency modeling, hastily transplanting them to two-dimensional vision tasks will not only disrupt the already weak spatial topology of the dust net but also cause a surge in computational overhead due to excessively long sequences. Therefore, the industry urgently needs to develop a new generation of targeted object detection methods that can not only break through the limitations of the local field of view of traditional operators to efficiently capture multi-scale global features, but also achieve deep adaptive interaction between low-level details and high-level semantics, thereby completely solving the detection problem of low-contrast dustproof nets in complex environments. Summary of the Invention

[0004] The purpose of this invention is to provide a dust net detection method and system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, in order to solve the problems of existing visual detection technology when dealing with dust net targets, such as the easy loss of local details due to complex environmental backgrounds and extremely weak mesh textures, and the difficulty in balancing multi-scale features due to the large target span. By introducing a multi-scale Mamba pyramid to reconstruct global feature representation, and using adaptive fusion and bidirectional attention mechanisms, the complex background can be accurately stripped away, achieving efficient localization and recognition of low-contrast dust nets.

[0005] To achieve the above objectives, the following technical solution is adopted: In a first aspect, the present invention provides a dustproof net detection method based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, the method comprising: Obtain an image dataset containing dust nets, and annotate the ground truth bounding box and category label of the dust net for each frame in the image dataset to obtain the training dataset; A deep learning network model is constructed, comprising a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure; wherein, multiple VMC3k2 local-global pyramid modules are configured, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone feature extraction network and the neck feature extraction network, and the spatial-semantic bidirectional fusion module is configured in the neck feature extraction network; The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then processed by several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target. The deep learning network model is trained using the training dataset to obtain a trained dust net detection model; the image to be detected is input into the trained dust net detection model to obtain the detection result of the dust net target.

[0006] Furthermore, the data processing flow of the VMC3k2 local-global pyramid module includes: Input fundamental feature tensor Parallel features are obtained through unit convolution and equal division along the channel dimension. and : in, For the real number space, For batch size, For the number of channels, For height, Width; It is a 1×1 convolution; This is an operation to divide the channel along its dimension. Based on parallel characteristics Local mesh features are extracted through a cascaded Bottleneck local feature extraction module. : in, , and The first n The, the n -1 and the first Bottleneck local feature extraction module, n The number of cascaded elements; Based on parallel characteristics Wide-area contour features were extracted using a multi-scale Mamba pyramid module. : in, For multi-scale Mamba pyramid modules; Local mesh features and wide-area contour features Dynamically allocate fusion weights and calculate fusion features. : in, and The local fusion weights and global fusion weights are dynamically allocated. Parallel features , and fusion characteristics After concatenation along the channel dimension, the output features of the VMC3k2 local-global pyramid module are obtained through unit convolution. .

[0007] Furthermore, based on parallel features Wide-area contour features were extracted using a multi-scale Mamba pyramid module. The process includes: With the aforementioned parallel features As input, set Several perceptual primitives at different scales, each perceptual primitive processing the input features through VisionMambaBlock. The process is performed to obtain feature maps at the corresponding scale. : in, For the first k A sensory primitive. Its dimension parameters, For the first k Feature maps extracted by scale branching To perceive the number of primitives; After concatenating the feature maps at different scales, the wide-area contour features are obtained through unit convolution. : in, This is the channel stitching result of feature maps at various scales.

[0008] Furthermore, the dynamically allocated local fusion weights and global fusion weights are determined as follows: For the local mesh features Perform global average pooling to obtain the channel-level global fingerprint metric. : in, For global average pooling; The fingerprint measurement The parameter matrix of the over-hidden layer is obtained by sequentially performing dimensionality reduction unit convolution, batch normalization, and ReLU activation. : in, For batch normalization, It is the ReLU activation function. To reduce the proportion of dimensions; The excessively hidden layer parameter matrix After being mapped to 2D via unit convolution and normalized using the Softmax function, the local fusion weights and global fusion weights sum to 1 are obtained: in, These are the initial local and global weight values ​​estimated by the neural network. This is the normalization function.

[0009] Furthermore, the VisionMambaBlock performs 2D unfolding of features and fusion with alternating horizontal and vertical scans, and uses the resulting effective features as a feature map. Specifically, it includes: With the aforementioned parallel features The input features are subjected to layer normalization and linear projection, and then separated into main feature branches. and gated branches : in, For layer normalization, The input matrix is ​​rearranged in multiple dimensions. The features are normalized. For linear projection; Local receptive field features are extracted from the main feature branch using depthwise separable 1D convolution: in, , , , These represent the coordinate indices of the element in the batch, height, width, and depth dimensions, respectively. The width of the convolution kernel. For depth-separable 1D convolution in the depth channel With offset The weight matrix at the location, For smoothing nonlinear activation functions, This represents a local receptive field characteristic; The local receptive field features are reshaped into one-dimensional sequence sets arranged by rows and columns, respectively. Unidirectional state space scanning is then performed on each set, and the scanning results are reshaped back to their original dimensions and averaged to obtain bidirectional scan fusion features. in, For tensor reshaping operations, For the hidden channel dimension of internal expansion, It is a unidirectional state space scan. and These are sets of one-dimensional sequences expanded by rows and columns, respectively. and To reconstruct the horizontal and vertical scan outputs back to their original dimensions, T is the matrix transpose in the spatial dimension. This represents the bidirectional scanning fusion feature, which is the mean state of the two scanning directions; The bidirectional scanning fusion features are multiplied element-wise by the SiLU-activated gated branches, and then linearly fused and projected to obtain the effective features. : in, For element-wise multiplication, For the end linear fusion projection layer, The effective features are the feature maps output by the VisionMambaBlock. .

[0010] Furthermore, the unidirectional state space scan The output is obtained through discretization of the state space, including: The continuous state-space model is discretized, and zero-order piecewise execution is used to compute the discrete transition tensor and discrete input tensor: in, and This refers to the discrete transition and discrete input tensors that dynamically change with time step after zero-order preserved discretization. For the long walk, The state transition parameter matrix, For the input projection matrix, This is the current input in the discrete recursion. It is a natural exponential function; Recursively calculate local hidden state variables and parse the output: in, For local hidden state variables divided within a length block, This is the hidden state from the previous moment. To output the projection matrix, For parsing output, This allows for direct jumps between layers in the connection variable.

[0011] Furthermore, the state transition parameter matrix Constrained to be a negative definite matrix: in, These are the parameters of the learnable background matrix stored in logarithmic space; The distance from the walk Set within the preset range: in, For forced amplitude limiting function, An activation function that outputs a value greater than zero. For linear prediction functions with respect to the distance step size, According to the low-rank ratio of the characteristic channel The tensor slice truncation operation performed. and These are the minimum and maximum limits for the step size, respectively. and These are the input projection and output projection matrices, respectively.

[0012] Furthermore, the spatial-semantic bidirectional fusion module performs bidirectional interactive enhancement on the spatial and semantic features in the first fused feature to obtain the second fused feature, including: Obtain the underlying spatial feature map from the first fused feature. and high-level semantic feature maps and will Bilinear interpolation upsampling to For the same space dimensions, : in, and The height and width of the spatial features, It is a bilinear interpolation fitting function. This is the semantic feature map after upsampling and alignment; Based on the underlying spatial feature map Transition context vectors are extracted through convolution and global average pooling. : in, The number of channels representing spatial characteristics. To bridge the constant embedding dimension of multi-layered semantics, For global average pooling; Based on the transition context vector Semantic attention masks are generated through convolution and the sigmoid function. : in, For the Sigmoid function, The number of channels for semantic features; The semantic attention mask Compared with the upsampled high-level semantic feature map Element-wise multiplication yields the enhanced semantic feature map. : Based on the high-level semantic feature map Semantic clustering guidance information is extracted through convolution and the Sigmoid function. : Will Upsampling to The same spatial dimensions are used, and spatial attention masks are generated through convolution and the sigmoid function. : in, This provides guidance information for semantic clustering after upsampling; The spatial attention mask With underlying spatial feature map Element-wise multiplication yields the enhanced spatial feature map. : Enhanced spatial feature map With the enhanced semantic feature map Channel concatenation is performed, followed by convolution to compress the number of channels to [a specific value]. The second fusion feature is obtained: in, For channel cascading operation, This is the second fusion feature.

[0013] Furthermore, after obtaining the detection results of the dust control net target, the method further includes: The bounding box coordinates are obtained by decoding the regression branch obtained from the detection head structure, and the coordinates are restored to the scale of the input image by combining different step sizes; Applying the Sigmoid activation function to the classification branch yields the confidence scores for each category. Candidate boxes are filtered according to confidence threshold, sorted from high to low confidence, and the intersection-over-union ratio (IoU) between candidate boxes is calculated one by one. Redundant predicted boxes with IoU greater than the set threshold are removed. After performing non-maximum suppression, the final set of detection boxes is obtained. If the final detection box set is not empty, it is determined that the dust net coverage area or the damaged target exists; otherwise, it is determined that no dust net target was detected.

[0014] Secondly, the present invention provides a dustproof net detection system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, the system comprising: The dataset construction module is configured to acquire an image dataset containing dust nets, and to annotate the ground truth bounding boxes and category labels of the dust nets for each frame in the image dataset to obtain the training dataset; The model building module is configured to build a deep learning network model, which includes a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure. Multiple VMC3k2 local-global pyramid modules are configured, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone and neck feature extraction networks. The spatial-semantic bidirectional fusion module is located within the neck feature extraction network. The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then processed by several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target. The target detection module is configured to train the deep learning network model using the training dataset to obtain a trained dust net detection model; and to input the image to be detected into the trained dust net detection model to obtain the detection result of the dust net target.

[0015] The beneficial effects of this invention are: This invention embeds a VMC3k2 local-global pyramid module into the backbone feature extraction network. Utilizing a parallel architecture of 2D bidirectional state space scanning and the Bottleneck local extraction module, it dynamically allocates local and global fusion weights based on the input content, adaptively adjusting the distribution ratio between local mesh texture details and global mesh contours. This allows for the extraction of multi-scale features with a large span while controlling the computational complexity of sequences. Simultaneously, a spatial-semantic bidirectional fusion module is employed in the feature fusion stage. This module uses the low-level spatial feature map to guide the high-level semantic feature map and leverages high-level semantic category information to suppress background interference in the low-level spatial feature map. This avoids boundary blurring and background noise interference during multi-scale feature fusion, improving robustness to environmental interference such as backlighting and dust occlusion. This invention is beneficial for the practical application of deep learning object detection technology in construction site supervision and environmental protection inspections. Attached Figure Description

[0016] Figure 1 This is an application scenario diagram of the low-contrast target detection method provided in the embodiments of the present invention; Figure 2 The flowchart illustrates a dust net detection method based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, provided for embodiments of the present invention.

[0017] Figure 3 This is a schematic diagram of the structure of a deep learning network model provided in an embodiment of the present invention.

[0018] Figure 4 This is a schematic diagram of the VMC3k2 local-global pyramid module architecture provided in an embodiment of the present invention.

[0019] Figure 5 This is a schematic diagram of the spatial-semantic bidirectional fusion module architecture provided in an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram of the Vision Mamba 2D bidirectional scanning architecture provided in an embodiment of the present invention.

[0021] Figure 7 This is a structural diagram of a dustproof net detection system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, provided for an embodiment of the present invention. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0023] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0024] This invention provides a method for detecting dust nets based on a multi-scale state space pyramid and spatial-semantic bidirectional fusion. Please refer to [link / reference]. Figure 1 This diagram illustrates an application scenario of the low-contrast target detection method provided in this embodiment of the invention. The application scenario includes a front-end image acquisition terminal 103 and a back-end computing and analysis platform 101. The front-end image acquisition terminal 103 is mounted on the inspection equipment 102 and is responsible for continuously capturing images of the work area 104. The computing and analysis platform 101 is equipped with a deep learning network model trained according to this invention. Based on this deep learning network model, it can process the image data transmitted by the front-end image acquisition terminal 103 in real time and output the precise coordinate boundaries of the dustproof net coverage area and its classification confidence level.

[0025] Please see Figure 2 The dust net detection method based on multi-scale state space pyramid and spatial-semantic bidirectional fusion specifically includes the following steps S10-S30.

[0026] S10: Obtain an image dataset containing dust nets, and annotate the true bounding box and category label of the dust net for each frame in the image dataset to obtain the training dataset.

[0027] In this embodiment, combined with Figure 1 As shown, multi-view images of the work site can be acquired using an image acquisition terminal to establish a dust control net dataset containing complex working conditions (such as backlighting, heavy dust obstruction, etc.). The total number of samples in the dataset is [number missing]. The dimension of a single frame image is defined as follows: The actual boundaries of the dustproof netting were defined and category labels were assigned using professional annotation software X-AnyLabeling. After data processing, the data was randomly divided into training and validation subsets.

[0028] S20: Construct a deep learning network model, which includes a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure; among them, multiple VMC3k2 local-global pyramid modules are set, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone feature extraction network and the neck feature extraction network; the spatial-semantic bidirectional fusion module is set in the neck feature extraction network. The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then passed through several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is fed into the neck feature extraction network, where the spatial and semantic features in the first fused feature are bidirectionally enhanced by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target.

[0029] In one specific embodiment, the constructed deep learning network model is as follows: Figure 3 As shown, the deep learning network model includes a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a YOLOHead detection head structure.

[0030] In this embodiment, the image input to the deep learning network model is convolved to obtain a basic feature tensor. This basic feature tensor is processed by four stacked VMC3k2 local-global pyramid modules in the backbone feature extraction network. Each VMC3k2 local-global pyramid module has a Conv module at its input. The output features of the fourth VMC3k2 local-global pyramid module are processed by the SPPF module and the CSPSA module to obtain the third-scale features. The output features of the third VMC3k2 local-global pyramid module (second-scale features) are processed by a Conv module and then enter the VMC3k2 local-global pyramid module located below its data stream. On the other hand, they are directly fed into the neck feature extraction network. Similarly, the output features of the second VMC3k2 local-global pyramid module (first-scale features) are processed by a Conv module and then enter the VMC3k2 local-global pyramid module located below its data stream. On the other hand, they are directly fed into the neck feature extraction network.

[0031] The Neck feature extraction network is configured with an upsampling module, a Concat module, a VMC3k2 local-global pyramid module, and a spatial-semantic bidirectional fusion module (SSBF). Multiple upsampling, Concat, and VMC3k2 local-global pyramid modules are used. The third-scale feature is upsampled by an upsampling module, then concatenated with the second-scale feature by a Concat module before being processed by a VMC3k2 local-global pyramid module to obtain the first feature. This first feature is then upsampled and processed by Concat before being fed into another VMC3k2 local-global pyramid module to obtain the first fused feature. This first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain the second fused feature. The second fused feature is then fed into a YOLOHead detection head structure (Detect) for prediction, obtaining the bounding box and class confidence of the dust net target. This yields the target detection result at one scale.

[0032] Furthermore, the second fused feature can be concatenated with the first feature and then fed into a VMC3k2 local-global pyramid module for processing to obtain a third fused feature, which is then fed into a YOLOHead detection head structure Detect for prediction. This allows for the acquisition of target detection results at another scale.

[0033] Furthermore, the third fusion feature can be further convolved and then concatenated with the third scale feature before being fed into a VMC3k2 local-global pyramid module for processing to obtain the fourth fusion feature. This fourth fusion feature is also fed into a YOLOHead detection head structure Detect to perform prediction, where another scale of target detection result can be obtained.

[0034] Therefore, the deep learning network model mentioned in this embodiment can obtain predicted feature maps at three scales.

[0035] In the backbone region (i.e., the backbone feature extraction network) of the deep learning network model, a custom VMC3k2 local-global pyramid module is used to replace the traditional convolutional blocks. For example... Figure 4 As shown, the VMC3k2 local-global pyramid module transmits feature tensors to shallow networks. This feature tensor is processed. The feature map is obtained by performing preliminary feature extraction on the input image through shallow convolutional layers in the backbone feature extraction network. Specifically, the input image first passes through several regular convolutional layers in the initial stage of the backbone feature extraction network. Through progressive downsampling operations, the original image is mapped from a high-resolution space to a low-resolution feature space, thereby obtaining a shallow feature map containing basic edge, texture, and simple shape information.

[0036] The VMC3k2 local-global pyramid module responds to the feature tensors passed by the shallow network. First, we use mapping convolution to split it into independent branches: in, For the real number space, For batch size, For the number of channels, For height, Width; It is a 1×1 convolution; This is a division operation along the channel dimension. and This is a parallel feature.

[0037] right Perform concurrent computation by offloading: Based on parallel characteristics Local mesh features are extracted through a cascaded Bottleneck local feature extraction module. : in, , and The first n The, the n -1 and the first Bottleneck local feature extraction module, n The number of cascaded elements; Based on parallel characteristics Wide-area contour features were extracted using a multi-scale Mamba pyramid module. : in, For multi-scale Mamba pyramid modules; Local mesh features and wide-area contour features Dynamically allocate fusion weights and calculate fusion features. : in, and The local fusion weights and global fusion weights are dynamically allocated. Parallel features , and fusion characteristics After concatenation along the channel dimension, the output features are obtained through unit convolution. : in, This involves splicing features along the channel dimension.

[0038] In one specific embodiment, a method is provided for extracting wide-area contour features using a multi-scale Mamba pyramid module. The process. To accommodate the significant size difference between the distant dust control net and the damaged mesh, parallel features are used. As input, set Several perceptual primitives at different scales, each perceptual primitive processing the input features through VisionMambaBlock. The process is performed to obtain feature maps at the corresponding scale. : in, For the first k A sensory primitive. Its dimension parameters, For the first k Feature maps extracted by scale branching To perceive the number of primitives; After concatenating the feature maps at different scales, wide-area contour features are obtained through unit convolution. : in, This is the channel stitching result of feature maps at various scales.

[0039] Please see Figure 6 This is a schematic diagram of the VisionMamba2D bidirectional scanning architecture provided in an embodiment of the present invention. The VisionMamba2D bidirectional scanning architecture, based on the input two-dimensional feature map, performs sequence unfolding and state space scanning processing on the same two-dimensional feature map in two dimensions. The two-dimensional feature map can be a local receptive field feature tensor, which is obtained by the main feature branch after depthwise separable 1D convolution and SiLU activation. Specifically, one path performs a horizontal SSM scan: the two-dimensional feature map is unfolded row-wise into a one-dimensional sequence set and then input into the unidirectional state space scanning function (i.e., the SSM module) to complete the long-distance dependency modeling in the horizontal dimension, and the output result is reshaped into... The other approach performs a vertical SSM scan: the 2D feature map is transposed, expanded column-wise into a set of one-dimensional sequences, and then input into the SSM module to complete long-distance dependency modeling in the vertical dimension. The output is then reshaped into... Output results of two independent SSM modules and The elements are collectively fed into a mean fusion unit, where element-wise mean calculation is performed to obtain bidirectional scanning fusion features. This results in a feature output that simultaneously integrates global contextual information from both horizontal and vertical dimensions. This bidirectional scanning mechanism can completely preserve the two-dimensional mesh topology of the dustproof net, avoiding the destruction of spatial topological relationships caused by traditional one-dimensional serialization methods. At the same time, through global state space modeling in both horizontal and vertical directions, it effectively captures long-distance pixel associations of the wide-area contour of the dustproof net.

[0040] Specifically, to prevent the 2D mesh topology of the dustproof net from being destroyed during serialization, VisionMambaBlock is used to perform 2D feature unrolling and alternating horizontal and vertical scan fusion. First, projection and convolution are performed: In the formula, The input matrix is ​​rearranged in multiple dimensions. For layer normalization, These are the features after normalization; Separate features into main feature branches and gated branches ; For width Depth separable Convolution in depth channel With offset The weight matrix at the location, These represent the coordinate indices of the element in the batch, height, width, and depth dimensions, respectively. To obtain local receptive field features for a smooth nonlinear activation function with Sigmoid properties Then, bidirectional development and integrated output will be carried out: In the formula, For tensor reshaping, The hidden channel dimension for internal expansion; It is a unidirectional state space scan. and These represent sets of one-dimensional sequences arranged by rows (horizontal) and columns (vertical), respectively, when the feature map is unfolded. and To reconstruct the horizontal and vertical scan outputs back to their original dimensions, superscript... Indicates the span of spatial dimensions and The matrix transpose operation performed on the above; This represents the mean state across both scanning directions. This represents element-wise multiplication; For the end of the linear fusion projection layer, For As the final effective output characteristic of the control valve, this effective characteristic is the feature map output by VisionMambaBlock. .

[0041] One-way state space scan The output is obtained through discretized state-space derivation. Specifically, the continuous derivation is transformed into discrete parameters adapted to graphical representation. To prevent the detection system from diverging, the state transition matrix is ​​constrained to the negative logarithmic space. The derivation core is executed piecewise using Zero-Order Hold (ZOH): In the formula, and Let be the derivatives of the hidden state variables of the system in continuous time and the current state value, respectively. and The input and output signal waveforms are in the continuous domain. The state transition parameter matrix is ​​in diagonal form; and These are the input projection and output projection matrices, respectively. Connect variables to directly jump across layers; It is a natural exponential function; The parameters of the learnable background matrix stored in logarithmic space are used to enforce the matrix... It is a negative definite matrix to maintain system stability; For the linear prediction function applied to the distance from the walk length; Tensor slicing operations performed in a specific dimension; The activation function for the translational sliding property is always greater than zero. To force the amplitude limiting function, the dynamic step size is... Strictly controlled to the minimum With the maximum limit To prevent overflow; and The extracted input-output dependent parameter matrix; and The discrete transition and discrete input tensors are dynamically changing with time step after zero-order hold (ZOH) discretization. For memory calculation blocks, segment number notation. These are local hidden state variables divided within a length block; , This represents the current input source and parsed output in a discrete recursive chain. Specifically, it can represent the input sequence element at the current time step, that is, the first element in the one-dimensional sequence after being expanded by row or column. t Feature vectors at each position, for example, a sequence expanded row by row. The first in t One element; This represents the output sequence element at the current time step after processing by the state-space model, i.e., the scan result at the corresponding position. In the discrete recursion process, it can be based on the current input... Update the current hidden state with the hidden state from the previous time step, and calculate the current output. ,for The function responds to the current input The response output.

[0042] In one specific embodiment, a method for determining dynamically allocated local fusion weights and global fusion weights is provided. This applies to the local mesh features. Perform global average pooling to obtain the channel-level global fingerprint metric. : in, For global average pooling; fingerprint measurement The parameter matrix of the over-hidden layer is obtained by sequentially performing dimensionality reduction unit convolution, batch normalization, and ReLU activation. : in, For batch normalization, It is the ReLU activation function. To reduce the proportion of dimensions; The excessively hidden layer parameter matrix After being mapped to 2D via unit convolution and normalized using the Softmax function, the local fusion weights and global fusion weights sum to 1 are obtained: in, These are the initial local and global weight values ​​estimated by the neural network. For the normalization function; by using the normalization function Ensure that the rough values ​​are converted into optimal allocation coefficients that always sum to 1. and .

[0043] In one specific embodiment, please refer to Figure 5 As shown, the spatial-semantic bidirectional fusion module performs bidirectional interactive enhancement on the spatial and semantic features in the first fusion feature to obtain the second fusion feature, including: Obtain the underlying spatial feature map from the first fusion feature. and high-level semantic feature maps and will Bilinear interpolation upsampling to For the same space dimensions, : in, and The height and width of the spatial features, It is a bilinear interpolation fitting function. This is the semantic feature map after upsampling and alignment; Based on the underlying spatial feature map Transition context vectors are extracted through convolution and global average pooling. : in, The number of channels representing spatial characteristics. To bridge the constant embedding dimension of multi-layered semantics, For global average pooling; Based on transition context vector Semantic attention masks are generated through convolution and the sigmoid function. : in, For the Sigmoid function, The number of channels for semantic features; semantic attention mask Compared with the upsampled high-level semantic feature map Element-wise multiplication yields the enhanced semantic feature map. : Based on high-level semantic feature maps Semantic clustering guidance information is extracted through convolution and the Sigmoid function. : Will Upsampling to The same spatial dimensions are used, and spatial attention masks are generated through convolution and the sigmoid function. : in, This provides guidance information for semantic clustering after upsampling; Spatial attention mask With underlying spatial feature map Element-wise multiplication yields the enhanced spatial feature map. : Enhanced spatial feature map With the enhanced semantic feature map Channel concatenation is performed, followed by convolution to compress the number of channels to [a specific value]. The second fusion feature is obtained: in, For channel cascading operation, This is the second fusion feature.

[0044] S30: Train the deep learning network model using the training dataset to obtain a trained dust net detection model; input the image to be detected into the trained dust net detection model to obtain the detection result of the dust net target.

[0045] In this embodiment, the training process includes 600 iterations, and the loss function configuration used is consistent with that of the basic YOLO object detection network (covering bounding box regression loss, classification loss, and distributed focus loss). The set training hyperparameters include an initial learning rate lr=0.0005, batch size=4, a preset training set to validation set split ratio of 8:2, and the selected optimizer type is AdamW.

[0046] The trained network is used for prediction. A test image is input, and the network outputs the predicted bounding box and class confidence score of the dust net target. First, the image of the dust net to be tested is input into the network, with the image size uniformly adjusted to 640×640 pixels. The image undergoes multi-scale feature extraction via the backbone network, and a local-global adaptive fusion module (VMC3k2) based on a multi-scale state space pyramid is used to enhance the feature representation ability of low-contrast textures. The extracted multi-scale features are fused and enhanced by the Spatial-Semantic Bidirectional Fusion (SSBF) module in the Neck part. SSBF includes: extracting global location information based on spatial features and enhancing semantic features in the channel dimension, and extracting class information based on semantic features and enhancing spatial features in the spatial dimension. After fusion, the network outputs feature maps at three scales: 80×80, 40×40, and 20×20, used to detect dust net targets of different sizes. The feature maps at the three scales are then input into the detection head for prediction. The detection head includes a regression branch and a classification branch: the regression branch decodes the relevant output to obtain bounding box regression information (reg_max=16), and the classification branch outputs the category confidence of the dust net target. The classification and regression results at different scales are concatenated, and the prediction results are uniformly organized into a representation with 8400 prediction points. During the inference phase, the bounding box coordinates are obtained by decoding the regression branch output, and the prediction results are restored to the input image scale by combining different scale strides (stride=8 / 16 / 32); a sigmoid activation function is applied to the classification branch output to obtain the confidence of each category. Subsequently, initial screening is performed according to the confidence threshold, and the boxes are sorted from high to low confidence; the intersection-over-union ratio (IoU=0.45) between candidate boxes is calculated one by one, redundant prediction boxes with an IoU greater than the set threshold are removed, and non-maximum suppression (NMS=0.25) is performed to obtain the final set of detection boxes. Finally, based on the scaling and padding relationship in the preprocessing stage, the coordinates of the retained detection boxes are restored from the network input scale of 640×640 to the original image scale. The output includes the detection results including the bounding box coordinates, confidence score, and category index. The final output detection box count does not exceed the preset maximum number of detections max_per_img=300.

[0047] Obtain the normalized position coordinates (X,Y,W,H) corresponding to the self-coordinate position (x,y,w,h) of the described detection box, and generate and overlay the corresponding bounding box on the test image. If at least one valid detection box is obtained in the test image, it is determined that the dust net coverage area or the damaged target exists; if no valid detection box is obtained, it is determined that no dust net target was detected.

[0048] This invention also provides a dust net detection system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion. Please refer to [link to relevant documentation]. Figure 7 The system includes: The dataset construction module 701 is configured to acquire an image dataset containing dust nets, and to annotate the true bounding box and category label of the dust net for each frame of the image dataset to obtain the training dataset; The model building module 702 is configured to build a deep learning network model, which includes a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure; wherein, multiple VMC3k2 local-global pyramid modules are configured, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone feature extraction network and the neck feature extraction network, and the spatial-semantic bidirectional fusion module is set in the neck feature extraction network; The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then processed by several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target. The target detection module 703 is configured to train the deep learning network model using the training dataset to obtain a trained dust net detection model; and to input the image to be detected into the trained dust net detection model to obtain the detection result of the dust net target.

[0049] It should be noted that the dust net detection system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion belongs to the same technical concept as the previously described method, and has the same technical principle and beneficial effect, so it will not be repeated here.

[0050] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims

1. A dustproof net detection method based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, characterized in that, The method includes: Obtain an image dataset containing dust nets, and annotate the ground truth bounding box and category label of the dust net for each frame in the image dataset to obtain the training dataset; A deep learning network model is constructed, comprising a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure; wherein, multiple VMC3k2 local-global pyramid modules are configured, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone feature extraction network and the neck feature extraction network, and the spatial-semantic bidirectional fusion module is configured in the neck feature extraction network; The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then processed by several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target. The deep learning network model is trained using the training dataset to obtain a trained dust net detection model; the image to be detected is input into the trained dust net detection model to obtain the detection result of the dust net target.

2. The method according to claim 1, characterized in that, The data processing flow of the VMC3k2 local-global pyramid module includes: Input fundamental feature tensor Parallel features are obtained through unit convolution and equal division along the channel dimension. and : in, For the real number space, For batch size, For the number of channels, For height, Width; It is a 1×1 convolution; This is an operation to divide the channel along its dimension. Based on parallel characteristics Local mesh features are extracted through a cascaded Bottleneck local feature extraction module. : in, , and The first n The, the n -1 and the first Bottleneck local feature extraction module, n The number of cascaded elements; Based on parallel characteristics Wide-area contour features were extracted using a multi-scale Mamba pyramid module. : in, For multi-scale Mamba pyramid modules; Local mesh features and wide-area contour features Dynamically allocate fusion weights and calculate fusion features. : in, and The local fusion weights and global fusion weights are dynamically allocated. Parallel features , and fusion characteristics After concatenation along the channel dimension, the output features of the VMC3k2 local-global pyramid module are obtained through unit convolution. .

3. The method according to claim 2, characterized in that, Based on parallel characteristics Wide-area contour features were extracted using a multi-scale Mamba pyramid module. The process includes: With the aforementioned parallel features As input, set Several perceptual primitives at different scales, each perceptual primitive processing the input features through VisionMambaBlock. The process is performed to obtain feature maps at the corresponding scale. : in, For the first k A sensory primitive. Its dimension parameters, For the first k Feature maps extracted by scale branching To perceive the number of primitives; After concatenating the feature maps at different scales, the wide-area contour features are obtained through unit convolution. : in, This is the channel stitching result of feature maps at various scales.

4. The method according to claim 2, characterized in that, The dynamically allocated local fusion weights and global fusion weights are determined as follows: For the local mesh features Perform global average pooling to obtain the channel-level global fingerprint metric. : in, For global average pooling; The fingerprint measurement The parameter matrix of the over-hidden layer is obtained by sequentially performing dimensionality reduction unit convolution, batch normalization, and ReLU activation. : in, For batch normalization, It is the ReLU activation function. To reduce the proportion of dimensions; The excessively hidden layer parameter matrix After being mapped to 2D via unit convolution and normalized using the Softmax function, the local fusion weights and global fusion weights sum to 1 are obtained: in, These are the initial local and global weight values ​​estimated by the neural network. This is the normalization function.

5. The method according to claim 3, characterized in that, The VisionMambaBlock performs 2D unfolding and alternating horizontal and vertical scanning fusion of the features, and uses the resulting effective features as a feature map. Specifically, it includes: With the aforementioned parallel features The input features are subjected to layer normalization and linear projection, and then separated into main feature branches. and gated branches : in, For layer normalization, The input matrix is ​​rearranged in multiple dimensions. The features are normalized. For linear projection; Local receptive field features are extracted from the main feature branch using depthwise separable 1D convolution: in, , , , These represent the coordinate indices of the element in the batch, height, width, and depth dimensions, respectively. The width of the convolution kernel. For depth-separable 1D convolution in the depth channel With offset The weight matrix at the location, For smoothing nonlinear activation functions, This represents a local receptive field characteristic; The local receptive field features are reconstructed into one-dimensional sequence sets arranged by rows and columns, respectively. Unidirectional state space scanning is then performed on each set, and the scanning results are reconstructed back to their original dimensions and averaged to obtain bidirectional scan fusion features. in, For tensor reshaping operations, For the hidden channel dimension of internal expansion, It is a unidirectional state space scan. and These are sets of one-dimensional sequences expanded by rows and columns, respectively. and To reconstruct the horizontal and vertical scan outputs back to their original dimensions, T is the matrix transpose in the spatial dimension. This represents the bidirectional scanning fusion feature, which is the mean state of the two scanning directions; The bidirectional scanning fusion features are multiplied element-wise by the SiLU-activated gated branches, and then linearly fused and projected to obtain the effective features. : in, For element-wise multiplication, For the end linear fusion projection layer, The effective features are the feature maps output by the VisionMambaBlock. .

6. The method according to claim 5, characterized in that, The unidirectional state space scan The output is obtained through discretization of the state space, including: The continuous state-space model is discretized, and zero-order piecewise execution is used to compute the discrete transition tensor and discrete input tensor: in, and This refers to the discrete transition and discrete input tensors that dynamically change with time step after zero-order preserved discretization. For the long walk, The state transition parameter matrix, For the input projection matrix, This is the current input in the discrete recursion. It is a natural exponential function; Recursively calculate local hidden state variables and parse the output: in, For local hidden state variables divided within a length block, This represents the hidden state from the previous moment. To output the projection matrix, For parsing output, This allows for direct jumps between layers in the connection variable.

7. The method according to claim 6, characterized in that, The state transition parameter matrix Constrained to be a negative definite matrix: in, These are the parameters of the learnable background matrix stored in logarithmic space; The distance from the walk Set within the preset range: in, For forced amplitude limiting function, An activation function that outputs a value greater than zero. For linear prediction functions with respect to the distance step size, According to the low-rank ratio of the characteristic channel The tensor slice truncation operation performed. and These are the minimum and maximum limits for the step size, respectively. and These are the input projection and output projection matrices, respectively.

8. The method according to claim 1, characterized in that, The spatial-semantic bidirectional fusion module performs bidirectional interactive enhancement on the spatial and semantic features in the first fused feature to obtain the second fused feature. The process includes: Obtain the underlying spatial feature map from the first fused feature. and high-level semantic feature maps and will Bilinear interpolation upsampling to For the same space dimensions, : in, and The height and width of the spatial features, It is a bilinear interpolation fitting function. This is the semantic feature map after upsampling and alignment; Based on the underlying spatial feature map Transition context vectors are extracted through convolution and global average pooling. : in, The number of channels representing spatial characteristics. To bridge the constant embedding dimension of multi-layered semantics, For global average pooling; Based on the transition context vector Semantic attention masks are generated through convolution and the sigmoid function. : in, For the Sigmoid function, The number of channels for semantic features; The semantic attention mask Compared with the upsampled high-level semantic feature map Element-wise multiplication yields the enhanced semantic feature map. : Based on the high-level semantic feature map Semantic clustering guidance information is extracted through convolution and the Sigmoid function. : Will Upsampling to The same spatial dimensions are used, and spatial attention masks are generated through convolution and the sigmoid function. : in, This provides guidance information for semantic clustering after upsampling; The spatial attention mask With underlying spatial feature map Element-wise multiplication yields the enhanced spatial feature map. : Enhanced spatial feature map With the enhanced semantic feature map Channel concatenation is performed, followed by convolution to compress the number of channels to [a specific value]. The second fusion feature is obtained: in, For channel cascading operation, This is the second fusion feature.

9. The method according to claim 1, characterized in that, After obtaining the detection results of the dust control net target, the method further includes: The bounding box coordinates are obtained by decoding the regression branch obtained from the detection head structure, and the coordinates are restored to the scale of the input image by combining different step sizes; Applying the Sigmoid activation function to the classification branch yields the confidence scores for each category. Candidate boxes are filtered according to confidence threshold, sorted from high to low confidence, and the intersection-over-union ratio (IoU) between candidate boxes is calculated one by one. Redundant predicted boxes with IoU greater than the set threshold are removed. After performing non-maximum suppression, the final set of detection boxes is obtained. If the final detection box set is not empty, it is determined that the dust net coverage area or the damaged target exists; otherwise, it is determined that no dust net target was detected.

10. A dustproof net detection system based on multi-scale state space pyramid and spatial-semantic bidirectional fusion, characterized in that, The system includes: The dataset construction module is configured to acquire an image dataset containing dust nets, and to annotate the ground truth bounding boxes and category labels of the dust nets for each frame in the image dataset to obtain the training dataset; The model building module is configured to build a deep learning network model, which includes a backbone feature extraction network, a neck feature extraction network, a VMC3k2 local-global pyramid module, a spatial-semantic bidirectional fusion module, and a detection head structure. Multiple VMC3k2 local-global pyramid modules are configured, and at least one VMC3k2 local-global pyramid module is embedded in both the backbone and neck feature extraction networks. The spatial-semantic bidirectional fusion module is located within the neck feature extraction network. The deep learning network model responds to the input image by convolution to obtain a basic feature tensor. The basic feature tensor is then processed by several stacked VMC3k2 local-global pyramid modules, and after adaptive fusion of local mesh texture features and global contour features, a first fused feature is obtained. The first fused feature is then processed by the spatial-semantic bidirectional fusion module to obtain a second fused feature. The second fused feature is then fed into the detection head structure for prediction to obtain the bounding box and class confidence of the dust net target. The target detection module is configured to train the deep learning network model using the training dataset to obtain a trained dust net detection model; and to input the image to be detected into the trained dust net detection model to obtain the detection result of the dust net target.

Images