Fine-grained target real-time image segmentation method and system based on dynamic state modeling network

Abstract

The invention relates to a fine-grained target real-time image segmentation method and system based on a dynamic state modeling network, and belongs to the technical field of intelligent image processing. The method comprises the following steps: extracting multi-scale detail features by using a lightweight backbone network; through a dual-scale two-dimensional selective scanning module, the features are divided into a thin branch and a thick branch, and local scanning and global scanning are executed respectively; a dynamic cross-scale feature selection and aggregation module is adopted, redundancy is suppressed through reweighting and statistical filtering, and key target responses are highlighted; at a decoding end, local details and global semantics are fused through jump connection and an edge extractor; and finally, introducing a form-guided pseudo label hierarchical supervision strategy, and improving the structure learning ability of the model by using a coarse-to-fine morphological prior. According to the method, the segmentation precision, the boundary integrity and the tiny target recall rate of the fine-grained target under the scenes of ore separation, industrial defect detection, pavement crack recognition and the like are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent image processing and segmentation technology, and in particular to a method and system for real-time fine-grained target image segmentation based on a dynamic state modeling network. The method, by introducing cross-scale feature interaction and a dynamic state modeling mechanism, achieves real-time segmentation of fine-grained target regions in complex scenes, and is applicable to various visual tasks such as ore image segmentation, industrial defect detection, and road damage recognition. Background Technology

[0002] Image segmentation aims to accurately distinguish between targets and background in images at the pixel level. With the widespread adoption of high-resolution sensing and edge computing, intelligent segmentation technology plays a core role in real-time quality inspection, online monitoring, and automated control in key applications such as intelligent ore sorting, industrial surface defect detection, and road defect identification. These applications emphasize the discernibility of fine-grained structures, robustness to complex scenarios, and low-latency deployment, placing higher demands on the modeling capabilities and computational efficiency of existing methods: First, small / slender targets account for a very low proportion and have fragmented shapes, making them prone to structural breaks and missed detections during downsampling and cross-layer fusion, thus affecting sorting purity, defect classification, and defect assessment; Second, complex backgrounds and interference are prevalent in various scenarios—such as dust and foreign objects in ore imaging, noise and reflections on workpiece surfaces, and marking lines and shadows in road images—leading to decreased target-background separability and blurred inter-class boundaries, thereby reducing detection reliability and subsequent decision-making accuracy; Third, online applications for various scenarios place more stringent comprehensive requirements on model throughput and memory usage.

[0003] Existing solutions based on convolutional neural networks, due to their fixed receptive field and local modeling characteristics, struggle to fully capture long-range dependencies and global context. This makes them prone to structural breaks, adhesion, or missegmentation on slender or small targets, and these issues become more pronounced under complex background interference. To enhance global perception, self-attention has been introduced, leading to improved accuracy. However, attention computation increases rapidly with sequence length. Even with methods like windowing and sparsification to reduce complexity, a trade-off between global context and efficiency remains, making it difficult to meet the engineering constraints of real-time performance and lightweight design.

[0004] In recent years, dynamic state modeling networks based on selective state-space models have efficiently propagated information in the spatial / sequence dimensions through input-dependent state transfer and gating, exhibiting near-linear complexity and good parallelism. However, practical applications targeting fine-grained objectives still face the following technical bottlenecks: 1. Long-range information gradually attenuates during spatial transmission, and weak edges and slender structures are easily submerged during deep fusion; 2. Cross-scale and multi-path fusion lacks selective reweighting and redundancy suppression, and multi-scale / multi-scan features are prone to information congestion and invalid correlations, leading to dilution of key structures and impaired cross-scale semantic alignment; 3. Under conditions of significant class imbalance and strong interference, there is a lack of hierarchical supervision guided by morphological priors, making it difficult for the model to simultaneously guarantee structural consistency and recall of small targets, easily resulting in boundary instability, under-classification, and misclassification; 4. Limited computing power and high-precision requirements coexist, and there is a lack of an integrated solution that simultaneously considers high-resolution input, global dependency modeling, and low-latency inference. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time image segmentation method and system for fine-grained targets based on a dynamic state modeling network, to solve the problems of easy breakage of slender or small target structures in complex backgrounds, difficulty in long-range dependency modeling, redundancy in cross-scale fusion, and insufficient real-time performance. This invention can complete high-precision, low-latency segmentation tasks of fine-grained targets in scenarios such as ore sorting, industrial defect detection, and road crack identification.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides a real-time image segmentation method for fine-grained targets based on a dynamic state modeling network, which completes fine-grained target segmentation based on input image data and a cross-scale dynamic state modeling network.

[0008] The steps for constructing the segmentation model include:

[0009] Step 1, Construction of fine-grained target dataset: Using online platforms for various scenarios including ore sorting, road inspection, and industrial testing, real-time image data of scenarios containing fine-grained targets, such as images of ore transportation, road cracks, and workpieces, are collected and constructed through data annotation.

[0010] Step 2, Lightweight detail feature extraction: Using a lightweight feature extraction backbone composed of depthwise separable convolutions, multi-level feature extraction is performed on the input image to obtain multi-scale feature representations including edge and texture priors, and the features are sequentially processed to adapt to subsequent state space modeling.

[0011] Step 3, Multi-scale semantic feature extraction: Multi-scale selective state blocks are used to realize semantic modeling and long-range dependency capture in the mid-to-deep feature space. Information flow across space and scale is realized through selective state updates, while obtaining effective global context while maintaining linear computational complexity.

[0012] Step 4, Dual-scale two-dimensional selective scanning: Semantic features are divided into fine-grained branches and coarse-grained branches according to channels. The fine-grained branches perform local region and cross-direction scanning, while the coarse-grained branches perform multi-directional global scanning. The two branches update the state in the selective state space units that depend on the input, thereby realizing the propagation of cross-scale context.

[0013] Step 5, Dynamic cross-scale feature selection and aggregation: Features from different scales and scanning paths are first reweighted at multiple scales, and then path information is filtered based on statistics to suppress redundant and consistent regions, highlight target responses with significant structural changes, and obtain aggregated features with stronger discriminative power.

[0014] Step 6, Local-Global Fusion Decoding: At the decoding end, cross-scale state blocks and lightweight convolutional blocks are alternately stacked, upsampled step by step, and fused with the corresponding scale features from the feature extraction side through skip connections; before the final output, an edge extractor composed of differential Gaussian, Laplacian Gaussian and convolution is introduced to enhance boundary details under the condition of no edge annotation.

[0015] Step 7, Morphology-guided pseudo-label hierarchical supervision: Generating multi-level morphological dilation pseudo-labels from coarse to fine through morphological dilation, and mapping them as spatial weights into the network using a pseudo-label attention mechanism, forming a coarse-to-fine hierarchical supervision, improving structural consistency and recall of small targets.

[0016] Step 8, Construction and Inference of Training Loss: Construct a training loss system consisting of main loss and multi-level auxiliary loss. The main loss is used to constrain the final segmentation map to achieve a balance between foreground recall and boundary consistency. The auxiliary loss is used to constrain the output of each layer of side path to gradually approximate the morphological prior. In the inference stage, export the model format adapted for deployment, support half-precision or integer quantization inference, and achieve real-time segmentation with low memory and low latency.

[0017] A further improvement of the present invention is that step 1 specifically includes:

[0018] Step 1.1 Using the imaging platform, imaging sensor components, and lighting and environmental control module, stable imaging acquisition is performed on fine-grained target scenes, including ore conveying, road surface cracks, and industrial workpieces. The time, position, and attitude information are recorded in conjunction with the synchronization and positioning module.

[0019] Step 1.2 The data acquisition and annotation management module receives images and metadata, performs data cleaning, grayscale normalization, and manual or semi-automatic annotation to build a high-quality training and inference dataset.

[0020] A further improvement of the present invention is that step 2 specifically includes:

[0021] Step 2.1 Construct a training dataset based on the target application, and perform data augmentation by random cropping, horizontal flipping, brightness and contrast perturbation, mild noise, and color jitter to enhance the model's robustness to changes in lighting, material, and scale.

[0022] Step 2.2 The lightweight detail feature extractor consists of two or more levels of depth-separable convolutional blocks. After each level of convolution, batch normalization and modified linear unit (ReLU) activation functions are applied. Downsampling with a stride of two and one-to-one channel projection are set to control the computational load and improve the channel expressiveness.

[0023] Step 2.3 Sequentialize the shallow features and, if necessary, introduce learnable positional encoding to improve the ability of subsequent state space modeling to perceive spatial order.

[0024] A further improvement of the present invention is that step 3 specifically includes:

[0025] Step 3.1 Input adaptation: Perform layer normalization and linear transformation on the two-dimensional features from Step 1, rearrange them according to the preset sequence length and channel dimension, and generate a serialized representation that meets the state space input requirements;

[0026] Step 3.2 Selective state modeling: Input the sequence into the state space model, divide the feature subflows of different scales according to the receptive domain during the state update process, and perform selective state scanning and aggregation at each scale. Small scale focuses on local details, while large scale focuses on the long-range context.

[0027] Step 3.3 Gated fusion: By inputting relevant weights to generate a network, the output of different scale states is dynamically weighted to enhance the response of significant regions such as crack skeletons and thin boundaries, and to suppress texture noise and background interference;

[0028] Step 3.4 Output Reconstruction: The fused state sequence is reconstructed and inversely transformed back to a two-dimensional feature map, and residuals are superimposed with the input of this layer. If necessary, one-to-one convolution is used for channel alignment, and the enhanced multi-scale semantic representation is output.

[0029] A further improvement of the present invention is that step 4 specifically includes:

[0030] Step 4.1 Divide the channel into fine branches and coarse branches proportionally. Use depthwise convolution with a unit stride to maintain resolution in the fine branches, and use depthwise convolution with a stride of 2 for downsampling in the coarse branches.

[0031] Step 4.2 The fine branch first performs local partitioning and Z-shaped scanning within the region, and then performs cross scanning in the horizontal and vertical directions to achieve the gradual fusion of local to cross-regional information;

[0032] Step 4.3 The coarse branch performs a multi-directional cross-scan to obtain the global context and establish stable dependencies on the large-scale structure;

[0033] Step 4.4 The two branches enter the selective state space unit that depends on the input for state update and are restored to two-dimensional features by inverse transformation;

[0034] Step 4.5 aligns the coarse branches to the spatial resolution of the fine branches using interpolation and outputs the result for subsequent cross-scale aggregation.

[0035] A further improvement of the present invention is that step 5 specifically includes:

[0036] Step 5.1 Multi-scale reweighting: Global average pooling and multilayer perceptron mapping are performed on the fusion features of fine branches and coarse branches to obtain adaptive branch weights and channel-level recalibration. The initial fusion features are then back-fused using the residual method.

[0037] Step 5.2 Multi-path statistical filtering: Calculate cross-path statistics on the multi-scan path output, construct a relative dispersion index to measure path response differences, set an empirical threshold to generate a filtering attention map, and weight and aggregate the features of each path point by point to retain areas with obvious structural changes and suppress redundant and consistent areas.

[0038] A further improvement of the present invention is that step 6 specifically includes:

[0039] Step 6.1 Each decoding layer adopts a cascaded structure of upsampling, cross-scale state blocks and lightweight convolutional blocks, and performs skip connections and fusion with the corresponding scale features of the feature extractor to achieve the synergy of semantic recovery and detail compensation.

[0040] Step 6.2 Before the final output, set up an edge extraction unit, which generates edge cue features by combining differential Gaussian, Laplacian Gaussian and convolution, and fuses them with the backbone features to improve the separability and stability of fine boundaries.

[0041] A further improvement of the present invention is that step 7 specifically includes:

[0042] Step 7.1 Generate multi-level morphological dilation pseudo-labels using real labels to provide the model with coarse-to-fine structural priors;

[0043] Step 7.2 Generate spatial attention weights through convolution and activation, inject pseudo-labels into the network, and suppress responses in non-target regions;

[0044] Step 7.3 Select a single-scale intra-block refinement or multi-scale inter-layer refinement strategy according to application requirements, generate side outputs at the corresponding positions and calculate bypass supervision signals to guide the gradual refinement of representations at different levels.

[0045] A further improvement of the present invention is that step 8 specifically includes:

[0046] Step 8.1 Training Loss Construction: The main loss adopts a composite loss for binary segmentation tasks to simultaneously consider foreground recall and boundary consistency; multi-level auxiliary losses are applied to the outputs of each side path to constrain the coarse-to-fine structure learning; the main loss and auxiliary losses are summed according to weights, which can be set according to the importance of the hierarchy and the degree of sample imbalance or adaptively learned.

[0047] Step 8.2 Training stability configuration: Adaptive optimizer and learning rate scheduling are adopted, combined with gradient clipping, mixed precision and regularization strategies to improve convergence stability against strong interference backgrounds and extremely small targets;

[0048] Step 8.3 Inference and Deployment: Export deployable model files, perform computation graph optimization and normalization fusion, and support half-precision or integer quantization inference; on resource-constrained platforms, prioritize the preservation of higher numerical precision of decoding ends and edge extraction units to ensure boundary quality and detail restoration.

[0049] On the other hand, this invention also provides a fine-grained real-time target image segmentation system based on a dynamic state modeling network. This system features a modular design encompassing the entire process of dataset construction, feature extraction, cross-scale modeling, decoding and reconstruction, supervised training, and inference deployment to meet the real-time requirements of online or in-vehicle scenarios. The system includes:

[0050] A dataset construction unit is used to collect scene image data containing fine-grained targets online, such as images of ore transportation processes, road crack inspection images, and workpiece images of industrial inspection objects, and to construct the dataset through an annotation management module; the dataset construction unit includes:

[0051] An imaging platform is used to carry the object under test and operate online according to a predetermined motion or positioning method;

[0052] Imaging sensor components are used to switch between X-ray detectors, industrial cameras, or linear array cameras depending on the scenario to acquire the desired images;

[0053] The lighting and environmental control module is used to provide stable and controllable lighting in different detection environments, eliminate reflections or noise, and ensure clear visibility of fine-grained targets;

[0054] The synchronization and positioning module is used to record metadata such as image acquisition time, platform / object position and attitude, and trigger signals, and to control imaging triggering to ensure synchronization between frames and between devices;

[0055] The data acquisition and annotation management module is used to receive acquired images and associated metadata, perform preprocessing, organize manual or semi-automatic annotation generation, and build high-quality datasets required for training and inference.

[0056] An image segmentation unit is used to perform fine-grained target segmentation on the image data and output structured results in real time. The image segmentation unit includes:

[0057] The lightweight detail feature extraction module is used to extract multi-level features from the input image using lightweight depthwise separable convolution and complete sequential processing, outputting a multi-scale feature tensor.

[0058] The multi-scale semantic feature extraction module is used to obtain mid-to-deep multi-scale semantic representations through input adaptation, selective state modeling, gated fusion and output reconstruction, and write the enhanced semantics back to the backbone in the form of residuals.

[0059] The dual-scale two-dimensional selective scanning module is used to perform local and global scans of fine and coarse branches, and to update the state in the state space unit that depends on the input, thereby mitigating the long-range information decay in high-resolution scenes.

[0060] The dynamic cross-scale feature selection and aggregation module is used to complete multi-scale reweighting and multi-path statistical filtering, and output aggregated features with strong discriminative power and low redundancy.

[0061] The local-global fusion decoding module is used for upsampling and semantic recovery during the decoding stage, and performs detail compensation and boundary stabilization through skip connections and edge extraction subunits;

[0062] A morphology-guided pseudo-label hierarchical supervision module is used to generate multi-level morphological dilation pseudo-labels and implement hierarchical supervision under two strategies: intra-scale block refinement or inter-scale layer refinement, through a pseudo-label attention mechanism.

[0063] The training and inference module is used for main and auxiliary loss construction, end-to-end training, model export and quantization deployment, and integrates optimization strategies such as mixed precision and memory reuse to meet the real-time requirements of edge and vehicle platforms.

[0064] A further improvement of the present invention lies in the specific steps performed by the lightweight detail feature extraction module:

[0065] First, data augmentation is performed using random cropping, horizontal flipping, brightness and contrast perturbation, mild noise, and color jitter to enhance the model's robustness to changes in lighting, material, and scale. Then, a lightweight extractor composed of depthwise separable convolutions is used as the backbone to extract features from the input image at multiple levels, obtaining multi-scale feature representations that include edge and texture priors. The features are then processed sequentially to adapt to subsequent state space modeling.

[0066] A further improvement of the present invention is that the multi-scale semantic feature extractor module includes:

[0067] The input adaptation submodule is used to perform layer normalization and linear mapping on the features from the feature extraction end, and rearrange them into a sequence according to a preset length and channels;

[0068] The selective state modeling submodule is used to divide multi-scale pathways according to receptive domains and perform selective state scanning and aggregation within a state-space framework.

[0069] The gated fusion submodule is used to dynamically weight the state output at each scale according to the relevant input weights, highlighting significant areas and suppressing background interference.

[0070] The output reconstruction submodule is used to map the fused sequence back to two-dimensional features through linear reconstruction and inverse transformation, and then superimpose it with the input residual to complete channel alignment.

[0071] A further improvement of the present invention is that the dual-scale two-dimensional selective scanning module includes:

[0072] Fine-branch paths are used to maintain resolution. They first perform local partitioning and intra-block scanning, and then perform cross-scanning in the horizontal and vertical directions to propagate cross-regional context.

[0073] Coarse-branched pathways are used to expand the global receptive field through downsampling and to perform multi-directional cross-scanning to obtain the global context;

[0074] Both branches perform state updates through selective state space units and align the spatial resolution after interpolation restoration.

[0075] A further improvement of the present invention is that the dynamic cross-scale feature selection and aggregation module includes:

[0076] The global reweighting submodule is used to generate adaptive branch weights through global average pooling and multilayer perceptron, and to perform channel-level recalibration and residual back-fusion.

[0077] The multi-path statistical filtering submodule is used to calculate the statistical differences of multiple scan paths, generate a filtering attention map, and perform point-by-point weighted aggregation to retain areas with significant structural changes while suppressing redundant and consistent areas.

[0078] A further improvement of the present invention is that the local-global fusion decoding module includes:

[0079] The cascaded decoding submodule is used to reconstruct spatial resolution by progressive upsampling, cross-scale state recovery and lightweight convolution, and to perform skip connections and fusion with the corresponding scale features of the feature extraction side to achieve the synergy of semantic reconstruction and detail compensation.

[0080] The edge extraction submodule is used to generate edge cue features before the final output. It uses a combination of differential Gaussian, Laplacian Gaussian and convolution, and fuses them with the backbone features to improve the separability and stability of weak boundaries.

[0081] A further improvement of the present invention is that the morphology-guided pseudo-label hierarchical supervision module includes:

[0082] The pseudo-tag generation submodule generates pseudo-tags from coarse to fine by setting decreasing morphological dilation rates according to different levels.

[0083] The single-scale block refinement submodule is used to inject spatial weights of morphological dilation pseudo-labels into the final decoding layer, generate side-path outputs and apply auxiliary supervision to enhance macroscopic structural connectivity.

[0084] The multi-scale interlayer refinement submodule is used to inject pseudo-label weights that match the resolution of each decoding layer, generate side-path outputs in layers, and supervise the gradual convergence of details to improve the recall of small targets and boundary accuracy.

[0085] The single-scale intra-block refinement submodule and the multi-scale inter-layer refinement submodule are two optional refinement strategies. One can be enabled according to application requirements, but not both.

[0086] A further improvement of the present invention is that the training and inference module includes:

[0087] The loss construction submodule is used to form the training objective with the main loss and multi-level auxiliary losses, taking into account for foreground recall, region consistency and boundary accuracy.

[0088] The training stability submodule is used for managing the adaptive optimizer, learning rate scheduling, gradient pruning, mixed precision, and regularization strategies.

[0089] The quantization and deployment submodule is used for half-precision or integer quantization inference and uses representative data for quantization calibration, prioritizing the numerical accuracy of end-decoding and edge extraction.

[0090] Compared with existing technologies, this invention has the following advantages: First, by using dual-scale scanning and input-dependent state updates, it significantly alleviates long-range information decay in high-resolution scenarios, improving the connectivity and integrity of slender and sparse structures. Second, through dynamic cross-scale feature selection and statistically based multi-path filtering, it effectively suppresses cross-scale and multi-path redundancy, balancing segmentation accuracy and inference latency. Third, through edge extraction hierarchical supervision and morphology-guided pseudo-label hierarchical supervision, it can still obtain stable and clear boundaries and a high recall rate for small targets even in complex textures and strong interference backgrounds. Fourth, through lightweight backbone and engineering-based inference optimization, it meets the memory and latency constraints of edge and vehicle deployments, enabling rapid deployment. Fifth, through modular and parametric design, it can be quickly adapted to different scenarios such as ore sorting, industrial defects, and road damage, exhibiting good versatility and scalability. Attached Figure Description

[0091] Figure 1 This is a flowchart of a fine-grained real-time target image segmentation method based on a dynamic state modeling network according to the present invention;

[0092] Figure 2 This is a general framework diagram of a fine-grained real-time target image segmentation system based on a dynamic state modeling network according to the present invention.

[0093] Figure 3 This is a framework diagram of the dual-scale two-dimensional selective scanning module of the present invention;

[0094] Figure 4 This is a framework diagram of the dynamic cross-scale feature selection and aggregation module of the present invention;

[0095] Figure 5 This is a framework diagram of the morphology-guided pseudo-label hierarchical supervision module of the present invention. Detailed Implementation

[0096] To make the objectives, technical effects, and technical solutions of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described are some embodiments of the present invention. Based on the disclosure of the present invention, other equivalent modifications or substitutions made by those skilled in the art without creative effort all fall within the protection scope of the present invention.

[0097] Please see Figure 1 The present invention provides a fine-grained real-time target image segmentation method based on a dynamic state modeling network, the process of which is as follows:

[0098] Step S1: Construction of fine-grained target dataset

[0099] This step utilizes online platforms for various scenarios such as ore sorting, road inspection, and industrial testing to collect scene image data containing fine-grained targets in real time, and constructs a dataset through annotation management.

[0100] In this embodiment of the invention, step S1 specifically includes:

[0101] Step S1.1 Through the imaging platform, imaging sensor components and lighting and environmental control module, stable imaging acquisition is performed on fine-grained target scenes such as ore conveying, road cracks and industrial workpieces, and time, position and attitude information are recorded in combination with the synchronization and positioning module.

[0102] Step S1.2 The data acquisition and annotation management module receives images and metadata, performs data cleaning, grayscale normalization, and manual or semi-automatic annotation to build a high-quality training and inference dataset.

[0103] Step S2: Lightweight Detail Feature Extraction

[0104] This step employs a lightweight feature extractor composed of depthwise separable convolutions to extract detailed features from the input image at a shallow level, focusing on preserving edges and detailed textures, and performing sequential processing for subsequent state space modeling.

[0105] In this embodiment of the invention, step S2 specifically involves the following process:

[0106] First, the data-enhanced image is input into a depthwise separable convolutional module to efficiently capture fine-grained edge features and preserve local details for the final prediction. Specifically, the depthwise separable convolutional module consists of channel-wise convolution and pointwise convolution: channel-wise convolution extracts spatial features independently for each channel, while pointwise convolution models cross-channel correlations and aggregates information at pixel locations through 1×1 convolution. This structure achieves efficient shallow feature representation while significantly reducing computational cost. To stabilize training, batch normalization and modified linear unit activation functions are applied after each depthwise separable convolution; and downsampling layers are set at the ends of each block to reduce spatial resolution, while 1×1 convolution projects spatial features to a higher channel dimension. Finally, the features are flattened, converting the two-dimensional features into sequence labels.

[0107] Step S3: Multi-scale semantic feature extraction

[0108] This step utilizes multi-scale selective state blocks to perform multi-scale context modeling and long-range dependency capture on mid-to-deep features. Input features achieve cross-space and cross-scale information flow in the state modeling unit; the obtained multi-scale states are weighted according to the input adaptive weights in the gated fusion unit to suppress background interference and enhance salient regions.

[0109] In this embodiment of the invention, step S3 specifically includes:

[0110] Step S3.1 Input adaptation: Perform layer normalization and linear mapping on the two-dimensional features from S2, and rearrange them into a sequence according to the preset length and channels.

[0111] Step S3.2 Dual-path decomposition: The sequence is divided into two paths: the first path passes through a linear layer and a Sigmoid weighted linear unit activation function (SiLU activation function) to obtain the semantic channel; the second path passes through a linear layer, a depthwise separable convolution and a Sigmoid weighted linear unit activation function (SiLU activation function) and is then fed into the two-dimensional selective scanning described later, performing multi-scale state scanning in the horizontal and vertical directions with different time lengths and attenuations.

[0112] Step S3.3 Gated fusion: The multi-scale state output is dynamically weighted using input-related gating weights to suppress background redundancy and highlight the thin boundaries of fine-grained targets.

[0113] Step S3.4 Output Reconstruction: The weighted state sequence is restored to two-dimensional features through linear reconstruction and inverse transformation, and then superimposed with the residual of the input of this layer. The features are then passed through layer normalization, linear layer, Gaussian error linear unit activation function (GeLU activation), and linear layer in sequence to obtain an enhanced mid-to-deep semantic representation.

[0114] Step S4: Dual-scale two-dimensional selective scanning

[0115] This step divides semantic features into fine-grained and coarse-grained branches according to channels to alleviate the "forgetting" of long-range information in high-resolution scenes, and performs state updates in selective state space units that depend on the input, and finally outputs the results after aligning with the resolution.

[0116] In this embodiment of the invention, step S4 specifically includes:

[0117] Step S4.1 Channel partitioning and resolution adjustment: Let the feature of the linear layer output of the second path in step S3.2 be x∈R. C×H×W By using different convolution kernels and strides, convolution processing can be separated to form feature maps with different resolutions. The extraction formats for fine-grained and coarse-grained features are as follows:

[0118] x fine =DWConv s＝1 (x[:D / 2,:,:]),

[0119] x coarse =DWConv s＝2 (x[D / 2:,:,:]),

[0120] Among them, DWConv x＝1 and DWConv x＝2Let represent depthwise convolutions with strides of 1 and 2, respectively, where D is the number of channels. Each branch occupies half of the channels to strike a balance between computational efficiency and information preservation.

[0121] Step S4.2 Fine-grained Branch Local-Global Scan: A bidirectional cross-scan strategy based on local regions is introduced into the fine-grained branch. This strategy consists of two phases: an intra-regional scan phase and an inter-regional scan phase. Specifically, in the intra-regional scan phase, the fine-grained features are flattened into H / 2 × W / 2 small blocks to spatially align local and global features; for each small block, the features are flattened into short sequences and a left-to-right Z-shaped scan is performed independently to reduce redundancy. In the inter-regional scan phase, cross-scans are performed along the horizontal and vertical axes to propagate contextual information causally.

[0122] Step S4.3 Coarse-grained branch global scan: coarse-grained feature x coarse The sequence is directly flattened and four common cross-scanning patterns are used to capture the global receptive field. The overall process can be written as follows:

[0123]

[0124] Where BiCrossScan represents a bidirectional cross-scan along the horizontal and vertical axes, Scan r For local area scanning, For tiling operations, N s This represents the number of scan paths.

[0125] Step S4.4 Selective State-Space Update: Input any scan sequence into the selective state-space model block S6. Block S6 originates from the state-space model and takes the input sequence... Map to hidden state h(t) and predict the output sequence

[0126]

[0127] in Let h′(t) be a learnable parameter, and h′(t) be the first derivative of h(t). For compatibility with deep learning, a time-scale parameter is used. Discretize the continuous-time SSM with zero-order preservation. Discrete input sequence With output sequence satisfy:

[0128]

[0129] in Equivalently, the above mapping can be written in convolution form:

[0130]

[0131] Where * denotes convolution operation. S6 further sets the corresponding convolution kernel. C and Δ adapt to the input, thus selectively “focusing on / forgetting” different parts of the input sequence and dynamically adjusting the convolution kernel to prioritize modeling key spatial dependencies.

[0132] Step S4.5 Two-dimensional reconstruction and resolution alignment: The fine / coarse branch outputs are transformed by inverse γ. R By restoring the γ-ray to a 2D layout, the inverse operation from 2D to 1D scanning is completed:

[0133]

[0134] Interpolate(·) is the nearest neighbor interpolation function, used to... Upsampling with Same spatial resolution; k represents the k-th N scan A path.

[0135] Step S5: Dynamic cross-scale feature selection and aggregation

[0136] This step first performs adaptive reweighting at the "multi-scale" level, and then performs statistical filtering at the "multi-path" level to reduce redundancy.

[0137] In this embodiment of the invention, step S5 specifically includes:

[0138] Step S5.1 Multi-scale aggregation: For each scan path k, multi-scale features are aggregated. The channel dimensions are spliced ​​together to form a fused representation. The fused features are then subjected to global average pooling (GAP), and adaptive attention weights for each branch are generated using a multilayer perceptron (MLP) containing rectified linear units (ReLU) and sigmoid functions.

[0139]

[0140] in This is the learnable weight matrix of the MLP. Then w... k Divide into w k,1 With w k,2 , respectively and Perform adaptive recalibration:

[0141]

[0142] W3 is a linear projection matrix used to restore the calibrated features to the channel dimension of the initial fused representation; and the information flow of the original fused features is preserved through residual connections.

[0143] Step S5.2 Multipath Statistical Filtering: The multipath filtering mechanism reduces redundancy between different scan paths by statistically analyzing the feature changes between paths. Fine-grained targets often exhibit significant response differences across different scan paths; while background regions with broader coverage typically have directional consistency and smaller cross-path variations. Therefore, the standard deviation σ across paths is calculated:

[0144]

[0145] Where σ represents the absolute dispersion. A further relative dispersion index RV is constructed to reduce the impact of differences in numerical magnitude:

[0146]

[0147] Where μ adj =μ-min(z) ensures non-negativity, and ∈ avoids zero as the denominator. Then, attention map A is formed based on an empirical threshold θ. filter =ReLU(RV-θ) and applied to each path feature; finally, the filtered representation is obtained by summing the path features after the attention mask:

[0148]

[0149] Where ⊙ represents Hadamard multiplication.

[0150] Step S6: Local-Global Fusion Decoding

[0151] This step alternately stacks cross-scale state blocks and lightweight convolutional blocks at the decoding end, upsamples them step by step, and fuses them with the corresponding scale features from the feature extraction side via skip connections to achieve semantic recovery and detail compensation. An edge extraction subunit is introduced before the final output, which can generate edge prompts without edge annotation and fuse them with the backbone features to improve the separability and stability of weak boundaries and fine structures.

[0152] In this embodiment of the invention, step S6 specifically includes:

[0153] Step S6.1 Cascaded Decoding: Each stage of feature decoding first upsamples the input features to restore spatial resolution, and then performs skip connections and concatenations with the same-scale features of the corresponding feature extraction layer; subsequently, the image features restore cross-scale semantic consistency through lightweight state space blocks, and then refine local textures and suppress structural noise through depthwise convolution, pointwise convolution, normalization, and activation in sequence.

[0154] Step S6.2 Edge Extraction: Before final prediction, differential Gaussian and Laplacian Gaussian filters are applied to the highest resolution backbone features, respectively, and then convolved to form an edge cue map. This cue map is then fused with the backbone features to explicitly enhance the target contour, end details, and weak connections. Finally, it is input into the segmentation head to generate a segmentation probability map. This extractor can stably improve boundary clarity and connectivity without edge annotation. The overall decoding process can be trained end-to-end with the feature extraction end, maintaining real-time performance while ensuring accuracy and robustness.

[0155] Step S7: Morphology-guided pseudo-label hierarchical supervision

[0156] This step gradually locates coarse / fine crack regions within the hierarchical structure of deep supervision, thereby learning robust and discriminative representations to alleviate class imbalance, improve structural connectivity, and enhance the recall of small targets.

[0157] In this embodiment of the invention, step S7 specifically includes:

[0158] Step S7.1: Pseudo-tag generation. In the deep layer stage, pseudo-tags can be set to a relatively large expansion rate o. (l) This provides structural priors for macroscopic crack detection; subsequently, the expansion rate is gradually reduced layer by layer to refine the prediction. Specifically, the expansion rate of the l-th layer is defined as o (l) =o (0) ·β (l) , where β (l) <1. Deep pseudo-labels with a larger inflation rate encourage the model to first locate the coarse region of the target, while shallow pseudo-labels with a smaller inflation rate provide supervision for identifying fine targets. The final prediction is still based on the true labels. This invention provides two selectable refinement strategies, namely S62 and S63.

[0159] Step S7.2: Intra-layer refinement at a single scale. In the final decoding stage, coarse-to-fine crack segmentation is achieved using an "intra-layer—stepwise refinement" approach. To propagate pseudo-label guidance, a morphology-guided pseudo-label hierarchical supervision module is iteratively inserted before final prediction: the morphology-guided pseudo-label hierarchical supervision generates spatial attention weights through a convolution and a sigmoid function, multiplying them point-by-point with the original features, selectively preserving target region information and suppressing background noise; subsequently, a target segmentation map is predicted, and auxiliary loss is calculated with pseudo-labels at the corresponding scale.

[0160] Step S7.3: Multi-scale Interlayer Refinement. Multi-scale interlayer refinement is a cross-layer hierarchical supervision strategy that achieves coarse-to-fine predictions at the outputs of each layer of the decoder. Generally, shallow features are better suited for capturing high-resolution textures and edge details, while deep features are better suited for capturing semantic context and overall target morphology. Through hierarchical supervision, target-related information in key regions is continuously optimized, thus providing stronger support for the final segmentation. During training, the spatial attention map generated by multi-scale interlayer refinement is upsampled to a resolution matching the decoding outputs of each layer to align with the pseudo-labels at the corresponding scale.

[0161] Step S8: Training Loss Construction and Inference

[0162] This step is used to establish an end-to-end training loss system and provide an inference deployment process to ensure real-time performance while taking into account for foreground recall, region consistency, and boundary accuracy.

[0163] In this embodiment of the invention, step S8 specifically includes:

[0164] Step S8.1: Constructing the training loss. A composite loss is applied simultaneously to the final prediction and the hierarchical supervision sidepath. The main loss consists of binary cross-entropy and Dice loss:

[0165]

[0166] in The principal loss is the final predicted crack probability map ψ and the actual segmentation mask φ; For the prediction of the l-th layer Corresponding scale-inflated pseudo-labels Auxiliary losses. The total loss is defined as...

[0167]

[0168] Where η and λ (l) To balance the weights of the primary and secondary losses, in practice, η = 1. λ (l) ∈[0,1] are used as learnable parameters to balance multi-scale loss and satisfy ∑ l＝1 λ (l) =1.

[0169] Step S8.2: To improve the effectiveness and stability of training, an adaptive optimizer, learning rate scheduling, gradient clipping, mixed precision, and regularization strategy are employed. The adaptive optimizer and learning rate scheduling, combined with gradient clipping, mixed precision, and regularization strategies, enhance convergence stability against strongly interfering backgrounds and extremely small targets.

[0170] Step S8.3: Inference and Deployment. Export the trained network to a deployment format, perform graph-level optimization and operator fusion, and enable half-precision or integer quantization inference on resource-constrained platforms. Quantization calibration uses representative road surface samples to reduce distribution drift. To ensure boundary quality, prioritize maintaining high numerical accuracy at the decoding end and edge extraction sub-units, while low-bit inference is used for the remaining layers to reduce latency and memory usage, thereby achieving real-time segmentation output in vehicle / device scenarios.

[0171] Please see Figure 2 This invention discloses a fine-grained real-time target image segmentation system based on a dynamic state modeling network. The system employs a modular design across the entire process, including dataset construction, feature extraction, cross-scale modeling, decoding and reconstruction, supervised training, and inference deployment, to meet the real-time requirements of online or in-vehicle scenarios. The system includes:

[0172] The dataset construction unit is used to collect scene image data containing fine-grained targets, such as images of ore transportation processes, road crack inspection images, and workpiece images of industrial inspection objects, and constructs the dataset through an annotation management module; the data acquisition unit includes:

[0173] An imaging platform is used to carry the object being measured (such as ore, industrial workpiece, road surface) and operate online according to a predetermined motion or positioning method;

[0174] Imaging sensor components are used to switch between X-ray detectors (ore sorting scenarios), industrial cameras (defect detection scenarios), and linear array cameras (road scenarios) to acquire the required images, depending on the different scenarios.

[0175] The lighting and environmental control module is used to provide stable and controllable lighting in different testing environments, eliminate reflections or noise, and ensure that fine-grained targets (such as small minerals, workpiece defects, and road surface cracks) are clearly visible.

[0176] The synchronization and positioning module is used to record metadata such as image acquisition time, platform / object position and attitude, and trigger signals, and to control imaging triggering to ensure synchronization between frames and between devices;

[0177] The data acquisition and annotation management module is used to receive acquired images and associated metadata, perform preprocessing (such as data cleaning and grayscale normalization), organize manual or semi-automatic annotation generation, and build high-quality datasets required for training and inference.

[0178] An image segmentation unit is used to perform fine-grained target segmentation on the image data and output structured results in real time. The image segmentation unit includes:

[0179] The dual-scale two-dimensional selective scanning module is used to perform local and global scans of fine and coarse branches, and to update the state in the state space unit that depends on the input, thereby mitigating the long-range information decay in high-resolution scenes.

[0180] The dynamic cross-scale feature selection and aggregation module is used to complete multi-scale reweighting and multi-path statistical filtering, and output aggregated features with strong discriminative power and low redundancy.

[0181] The local-global fusion decoding module is used for upsampling and semantic recovery during the decoding stage, and performs detail compensation and boundary stabilization through skip connections and edge extraction subunits;

[0182] A morphology-guided pseudo-label hierarchical supervision module is used to generate multi-level morphological dilation pseudo-labels and implement hierarchical supervision under two strategies: intra-scale block refinement or inter-scale layer refinement, through a pseudo-label attention mechanism.

[0183] The training and inference module is used for main and auxiliary loss construction, end-to-end training, model export and quantization deployment, and integrates optimization strategies such as mixed precision and memory reuse to meet the real-time requirements of edge and vehicle platforms.

[0184] The lightweight detail feature extraction module includes:

[0185] The data processing submodule enhances the model's robustness to changes in lighting, material, and scale by performing data augmentation through random cropping, horizontal flipping, brightness and contrast perturbation, mild noise, and color jitter.

[0186] The local feature extraction submodule utilizes a lightweight extractor composed of depthwise separable convolutions as its backbone to perform multi-level feature extraction on the input image, obtaining multi-scale feature representations that include edge and texture priors. The features are then sequentially processed to adapt for subsequent state space modeling. Specifically, the processed image is input into the depthwise separable convolution module to efficiently capture fine-grained edge features and preserve local details for the final prediction. The depthwise separable convolution module consists of channel-wise convolutions and pointwise convolutions: channel-wise convolutions independently extract spatial features for each channel, while pointwise convolutions model cross-channel correlations and aggregate information at pixel locations using 1×1 convolutions. This structure significantly reduces computational cost while achieving efficient shallow feature representation. To stabilize training, batch normalization and modified linear unit activation functions are applied after each depthwise separable convolution; downsampling layers are set at the ends of each block to reduce spatial resolution, while 1×1 convolutions project spatial features to higher channel dimensions. Finally, the features are flattened, sequentially converting the two-dimensional features into sequence labels.

[0187] The multi-scale semantic feature extractor module includes:

[0188] Input adaptation submodule: Performs layer normalization and linear mapping on the two-dimensional features from S1, and rearranges them into a sequence according to the preset length and channels;

[0189] Selective state modeling submodule: The sequence is divided into two paths: the first path passes through a linear layer and a Sigmoid weighted linear unit activation function (SiLU activation function) to obtain the semantic channel; the second path passes through a linear layer, a depthwise separable convolution and a Sigmoid weighted linear unit activation function (SiLU activation function) and is then fed into the two-dimensional selective scanning described later, performing multi-scale state scanning in the horizontal and vertical directions with different step lengths and attenuations;

[0190] Gated fusion submodule: Dynamically weights the multi-scale state output with input-related gating weights to suppress background redundancy and highlight the thin boundaries of fine-grained targets;

[0191] Output reconstruction submodule: The weighted state sequence is first restored to two-dimensional features through linear reconstruction and inverse transformation, and then superimposed with the residual of the input of this layer. The features are then passed through layer normalization, linear layer, Gaussian error linear unit activation function (GeLU activation), and linear layer in sequence to obtain an enhanced mid-to-deep semantic representation.

[0192] The process of the dual-scale two-dimensional selective scanning module is as follows: Figure 3 As shown, the details are as follows:

[0193] Channel partitioning and resolution adjustment: Let the features after depthwise convolution in the multi-scale semantic feature extractor module be x∈R. C×H×W By using different convolution kernels and strides, convolution processing can be separated to form feature maps with different resolutions. The extraction formats for fine-grained and coarse-grained features are as follows:

[0194] x fine =DWConv s＝1 (x[:D / 2,:,:]),

[0195] x coarse =DWConv s＝2 (x[D / 2:,:,:]),

[0196] Among them, DWConv s＝1 and DWConv s＝2 Let represent depthwise convolutions with strides of 1 and 2, respectively, where D is the number of channels. Each branch occupies half of the channels to strike a balance between computational efficiency and information preservation.

[0197] Fine-grained branching pathways: A bidirectional cross-scanning strategy based on local regions is introduced into fine-grained branching. This strategy comprises two phases: an intra-regional scanning phase and an inter-regional scanning phase. Specifically, in the intra-regional scanning phase, fine-grained features are flattened into H / 2 × W / 2 small blocks to spatially align local and global features; for each small block, the features are flattened into short sequences and independently performed with a left-to-right Z-shaped scan to reduce redundancy. In the inter-regional scanning phase, cross-scanning is performed along the horizontal and vertical axes to propagate contextual information causally.

[0198] Coarse-branched pathways: coarse-grained features x coarse The sequence is directly flattened and four common cross-scanning patterns are used to capture the global receptive field. The overall process can be written as follows:

[0199]

[0200] Where BiCrossScan represents a bidirectional cross-scan along the horizontal and vertical axes, Scan r For local area scanning, For tiling operations, N s This represents the number of scan paths.

[0201] Selective State-Space Update: Any scan sequence is fed into the selective state-space model block S6. Block S6 originates from the state-space model, which maps the input sequence x(t) to the hidden state h(t) and predicts the output sequence y(t).

[0202]

[0203] in Let h′(t) be a learnable parameter, and h′(t) be the first derivative of h(t). For compatibility with deep learning, a time-scale parameter is used. Discretize the continuous-time SSM with zero-order preservation. Discrete input sequence With output sequence satisfy:

[0204]

[0205] in Equivalently, the above mapping can be written in convolution form:

[0206]

[0207] Where * denotes convolution operation. S6 further sets the corresponding convolution kernel. C and Δ adapt to the input, thus selectively “focusing on / forgetting” different parts of the input sequence and dynamically adjusting the convolution kernel to prioritize modeling key spatial dependencies.

[0208] 2D Reconstruction and Resolution Alignment: Fine / Coarse Branch Outputs Through Inverse Transform γ R By restoring the γ-ray to a 2D layout, the inverse operation from 2D to 1D scanning is completed:

[0209]

[0210] Interpolate(·) is the nearest neighbor interpolation function, used to... Upsampling with Same spatial resolution; k represents the k-th N scan A path.

[0211] The dynamic cross-scale feature selection and aggregation module (process as follows) Figure 4 (As shown) includes:

[0212] Global reweighting submodule: For each scan path k, multi-scale features are... The channel dimensions are spliced ​​together to form a fused representation. The fused features are then subjected to global average pooling (GAP), and adaptive attention weights for each branch are generated using a multilayer perceptron (MLP) containing rectified linear units (ReLU) and sigmoid functions.

[0213]

[0214] in This is the learnable weight matrix of the MLP. Then w... k Divide into w k,1 With w k,2 , respectively and Perform adaptive recalibration:

[0215]

[0216] W3 is a linear projection matrix used to restore the calibrated features to the channel dimension of the initial fused representation; and the information flow of the original fused features is preserved through residual connections.

[0217] Multi-path statistical filtering submodule: The multi-path filtering mechanism reduces redundancy between different scan paths by statistically analyzing the feature changes between paths. Since fine-grained targets often exhibit significant response differences across different scan paths, while background regions with broader coverage typically show directional consistency and smaller cross-path variations, the standard deviation σ across paths is calculated.

[0218]

[0219] Where σ represents the absolute dispersion. A further relative dispersion index RV is constructed to reduce the impact of differences in numerical magnitude:

[0220]

[0221] Where μ adj =μ-min(z) ensures non-negativity, and ∈ avoids zero as the denominator. Then, attention map A is formed based on an empirical threshold θ. filter =ReLU(RV-θ) and applied to each path feature; finally, the filtered representation is obtained by summing the path features after the attention mask:

[0222]

[0223] Where ⊙ represents Hadamard multiplication.

[0224] The local-global fusion decoding module includes:

[0225] Cascaded Decoding Submodule: Each stage of feature decoding first upsamples the input features to restore spatial resolution, then performs skip connections and concatenates them with the same-scale features from the corresponding feature extraction layer; subsequently, image features restore cross-scale semantic consistency through lightweight state space blocks, and then refine local textures and suppress structural noise through depthwise convolution, pointwise convolution, normalization, and activation in sequence.

[0226] The edge extraction submodule applies differential Gaussian and Laplacian Gaussian filters to the highest-resolution backbone features before final prediction, and then integrates them through convolution to form an edge cue map. This cue map is then fused with the backbone features to explicitly enhance the target contour, terminal details, and weak connections. Finally, it is input into the segmentation head to generate a segmentation probability map. This extractor can stably improve boundary sharpness and connectivity without edge annotation. The overall decoding process can be trained end-to-end with the feature extraction end, maintaining real-time performance while ensuring accuracy and robustness.

[0227] The morphology-guided pseudo-label hierarchical supervision module includes:

[0228] The pseudo-tag generation submodule. In the deeper stages, pseudo-tags can be set to a relatively large expansion rate. (l) This provides structural priors for macroscopic crack detection; subsequently, the expansion rate is gradually reduced layer by layer to refine the prediction. Specifically, the expansion rate of the l-th layer is defined as o (l) =o (0) ·β (l) , where β (l) <1. Deep pseudo-labels with a larger inflation rate encourage the model to first locate the coarse region of the target, while shallow pseudo-labels with a smaller inflation rate provide supervision for identifying fine targets. The final prediction is still based on the true labels.

[0229] Sub-modules within a single-scale layer. For example... Figure 5 In (a), the final decoding stage achieves coarse-to-fine crack segmentation using an "intra-layer—stepwise refinement" approach. To propagate pseudo-label guidance, a morphology-guided pseudo-label hierarchical supervision module is iteratively inserted before the final prediction: the morphology-guided pseudo-label hierarchical supervision generates spatial attention weights through a convolution and a sigmoid function, and multiplies them point-by-point with the original features, selectively preserving target region information and suppressing background noise; subsequently, a target segmentation map is predicted, and auxiliary loss is calculated with pseudo-labels of the corresponding scale.

[0230] Multi-scale inter-layer refinement submodule. Multi-scale inter-layer refinement is a cross-layer hierarchical supervision strategy that achieves coarse-to-fine predictions at the output of each layer throughout the decoder (see...). Figure 5 (b) Generally, shallow features are better at capturing high-resolution textures and edge details, while deep features are better at capturing semantic context and overall target morphology. Through hierarchical supervision, target-related information in key regions is continuously optimized, thus providing stronger support for the final segmentation. During training, the spatial attention maps generated by multi-scale inter-layer refinement are upsampled to a resolution matching the decoding output of each layer, so as to align with the pseudo-labels at the corresponding scale.

[0231] The single-scale intra-block refinement submodule and the multi-scale inter-layer refinement submodule are two optional refinement strategies. One can be enabled according to application requirements, but not both.

[0232] The training and inference module includes:

[0233] Loss Construction Submodule. A composite loss is applied simultaneously to the final prediction and the hierarchical supervision sidepath. The main loss consists of binary cross-entropy and Dice loss:

[0234]

[0235] in The principal loss is the final predicted crack probability map ψ and the actual segmentation mask φ; For the prediction of the l-th layer Corresponding scale-inflated pseudo-labels Auxiliary losses. The total loss is defined as...

[0236]

[0237] Where η and λ (l) To balance the weights of the primary and secondary losses, in practice, η = 1. λ (l) ∈[0,1] are used as learnable parameters to balance multi-scale loss and satisfy ∑ l＝1 λ (l) =1.

[0238] The training stability submodule, including an adaptive optimizer and learning rate scheduling, combines gradient clipping, mixed precision, and regularization strategies to improve convergence stability against strongly disturbed backgrounds and extremely small targets during the training phase.

[0239] The quantization and deployment submodule exports the trained network to a deployment format, performs graph-level optimization and operator fusion, and enables half-precision or integer quantization inference on resource-constrained platforms. Quantization calibration uses representative road surface samples to reduce distribution drift. To ensure boundary quality, higher numerical accuracy is maintained primarily at the decoding end and edge extraction subunits, while low-bit inference is used for the remaining layers to reduce latency and memory usage, thereby achieving real-time segmentation output in vehicle / device scenarios.

[0240] The main contributions of this invention are as follows: (1) Proposing an integrated dynamic state modeling real-time segmentation framework for fine-grained targets: Constructing an end-to-end system of "lightweight detailed feature extraction - dual-scale selective state scanning - dynamic cross-scale aggregation - edge extraction and decoding - morphological guidance hierarchical supervision", which simultaneously takes into account global dependency modeling and fine-grained structure fidelity under low computation / low latency constraints, and is applicable to multiple scenarios such as intelligent ore sorting, industrial surface defect detection, and road disease identification. (2) Dual-scale selective state scanning method: Dividing features into fine and coarse branches, the fine branch adopts "intra-block sequential scanning + horizontal / vertical bidirectional cross scanning" to strengthen local continuity, and the coarse branch performs global cross scanning to aggregate long-range context; the two branch sequences enter the selective state space to realize input dependency memory / forgetting update, alleviate long-range information "forgetting" with near-linear complexity, and improve the coherent recognition of slender structures and weak edges. (3) Dynamic cross-scale feature aggregation: First, dynamic recalibration of fine / coarse branches is achieved through scale-adaptive weights. Then, path attention based on relative deviation is constructed on the parallel scanning path for selective filtering, suppressing redundancy and invalid correlations, improving cross-scale semantic alignment, and enhancing robustness in complex backgrounds. (4) Morphology-guided pseudo-label hierarchical supervision and deployable training scheme: Based on morphological dilation, a coarse-to-fine pseudo-label sequence is constructed, hierarchical supervision is achieved on the decoder side branch, and background noise is suppressed by pseudo-label attention. During the training phase, the BCE+Dice combined loss and learnable multi-scale weights are combined with mixed precision and mini-batch inference to achieve a comprehensive effect of higher recall, more stable boundaries, and online deployment friendliness for small targets.

[0241] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fine-grained real-time target image segmentation method based on a dynamic state modeling network, characterized in that: include: Step 1, Fine-grained data acquisition: Utilize online platforms for various scenarios, including ore sorting, road inspection, and industrial testing, to collect scene image data containing fine-grained targets in real time, and construct a dataset through data annotation; Step 2, Lightweight detail feature extraction: Using a lightweight feature extraction backbone composed of depthwise separable convolutions, multi-level feature extraction is performed on the input image to obtain multi-scale feature representations including edge and texture priors, and the features are sequentially processed to adapt to subsequent state space modeling. Step 3, Multi-scale semantic feature extraction: Multi-scale selective state blocks are used to realize semantic modeling and long-range dependency capture in the mid-to-deep feature space. Information flow across space and scale is realized through selective state updates, while obtaining effective global context while maintaining linear computational complexity. Step 4, Dual-scale two-dimensional selective scanning: Semantic features are divided into fine-grained branches and coarse-grained branches according to channels. The fine-grained branches perform local region and cross-direction scanning, while the coarse-grained branches perform multi-directional global scanning. The two branches update the state in the selective state space units that depend on the input, thereby realizing the propagation of cross-scale context. Step 5, Dynamic cross-scale feature selection and aggregation: Features from different scales and scanning paths are first reweighted at multiple scales, and then path information is filtered based on statistics to suppress redundant and consistent regions, highlight target responses with significant structural changes, and obtain aggregated features with stronger discriminative power. Step 6, Local-Global Fusion Decoding: At the decoding end, cross-scale state blocks and lightweight convolutional blocks are alternately stacked, upsampled step by step, and fused with the corresponding scale features from the feature extraction side through skip connections; before the final output, an edge extractor composed of differential Gaussian, Laplacian Gaussian and convolution is introduced to enhance boundary details under the condition of no edge annotation. Step 7, Morphology-guided pseudo-label hierarchical supervision: Generating multi-level morphological dilation pseudo-labels from coarse to fine through morphological dilation, and mapping them as spatial weights into the network using a pseudo-label attention mechanism, forming a coarse-to-fine hierarchical supervision, improving structural consistency and recall of small targets. Step 8, Construction and Inference of Training Loss: Construct a training loss system consisting of main loss and multi-level auxiliary loss. The main loss is used to constrain the final segmentation map to achieve a balance between foreground recall and boundary consistency. The auxiliary loss is used to constrain the output of each layer of side path to gradually approximate the morphological prior. In the inference stage, export the model format adapted for deployment, support half-precision or integer quantization inference, and achieve real-time segmentation with low memory and low latency.

2. The method according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Through the imaging platform, imaging sensor components and lighting and environmental control module, stable imaging acquisition is performed on fine-grained target scenes including ore transportation, road cracks and industrial workpieces, and time, position and attitude information are recorded in combination with the synchronization and positioning module. Step 1.2: Perform data cleaning, grayscale normalization, and manual or semi-automatic annotation on the received images and metadata to construct a high-quality training and inference dataset.

3. The method according to claim 1, characterized in that, Step 2 specifically includes: Step 2.1: Construct a training dataset for the target scene and perform data augmentation using random cropping, horizontal flipping, brightness perturbation, noise and color jitter to improve the robustness of the model; Step 2.2: Lightweight depthwise separable convolution is used as the backbone for feature extraction, and each convolution is followed by a batch normalization and corrected linear unit (ReLU) activation function; Step 2.3 involves controlling the computational load and enhancing channel representation through 1×1 convolution channel projection, and sequentializing the features to adapt to subsequent state space modeling.

4. The method according to claim 1, characterized in that, Step 3 includes: Step 3.1 Perform layer normalization and linear transformation on the two-dimensional features from Step 1, rearrange them according to the preset sequence length and channel dimension, and generate a serialized representation that meets the state space input requirements; Step 3.2 Input the sequence into the state space model. During the state update process, divide the feature subflows into different scales according to the receptive field. Perform selective state scanning and aggregation at each scale. Small scale focuses on local details, while large scale focuses on the long-range context. Step 3.3 By inputting relevant weights to generate a network, the output of different scale states is dynamically weighted to enhance the response of significant regions such as crack skeletons and thin boundaries, and to suppress texture noise and background interference. Step 3.4 The fused state sequence is mapped back to a two-dimensional feature map through linear reconstruction and inverse transformation, and residual superposition is performed with the input of this layer. If necessary, one-to-one convolution is used for channel alignment, and the enhanced multi-scale semantic representation is output.

5. The method according to claim 1, characterized in that, Step 4 includes: Step 4.1 Divide the semantic features into fine-grained branches and coarse-grained branches according to the channel ratio. The fine-grained branch uses a depthwise convolution with a unit stride to maintain resolution, while the coarse-grained branch uses a depthwise convolution with a stride of 2 for downsampling. Step 4.2 The fine branch first performs local partitioning and Z-shaped scanning within the region, and then performs cross scanning in the horizontal and vertical directions to achieve the gradual fusion of local to cross-region information; Step 4.3 The coarse branch performs a multi-directional cross-scan to obtain the global context and establish stable dependencies on the large-scale structure; Step 4.4 The two branches enter the selective state space unit that depends on the input for state update and are restored to two-dimensional features by inverse transformation; Step 4.5 aligns the coarse branches to the spatial resolution of the fine branches using interpolation and outputs the result for subsequent cross-scale aggregation.

6. The method according to claim 1, characterized in that, Step 5 includes: Step 5.1 Multi-scale reweighting: Global average pooling and multilayer perceptron mapping are performed on the fusion features of fine branches and coarse branches to obtain adaptive branch weights and channel-level recalibration. The initial fusion features are then back-fused using the residual method. Step 5.2 Multi-path statistical filtering: Calculate cross-path statistics on the multi-scan path output, construct a relative dispersion index to measure path response differences, set an empirical threshold to generate a filtering attention map, and weight and aggregate the features of each path point by point to retain areas with obvious structural changes and suppress redundant and consistent areas.

7. The method according to claim 1, characterized in that, Step 6 includes: Step 6.1 Each decoding layer adopts a cascaded structure of upsampling, cross-scale state blocks and lightweight convolutional blocks, and performs skip connections and fusion with the corresponding scale features of the feature extractor to achieve the synergy of semantic recovery and detail compensation. Step 6.2 Before the final output, set up an edge extraction unit, which generates edge cue features by combining differential Gaussian, Laplacian Gaussian and convolution, and fuses them with the backbone features to improve the separability and stability of fine boundaries.

8. The method according to claim 1, characterized in that, Step 7 includes: Step 7.1 Generate multi-level morphological dilation pseudo-labels using real labels to provide the model with coarse-to-fine structural priors; Step 7.2 Generate spatial attention weights through convolution and activation, inject pseudo-labels into the network, and suppress responses in non-target regions; Step 7.3 Select a single-scale intra-block refinement or multi-scale inter-layer refinement strategy according to application requirements, generate side outputs at the corresponding positions and calculate bypass supervision signals to guide the gradual refinement of representations at different levels.

9. The method according to claim 1, characterized in that, Step 8 includes: Step 8.1 The main loss adopts a composite loss for binary segmentation tasks to simultaneously take into account for foreground recall and boundary consistency; multi-level auxiliary losses are applied to each side-path output to constrain the coarse-to-fine structure learning; the main loss and auxiliary losses are summed according to weights, which can be set according to the importance of the hierarchy and the degree of sample imbalance or adaptively learned. Step 8.2 During the training phase, an adaptive optimizer, learning rate scheduling, gradient pruning, mixed precision and regularization strategy are adopted to improve the convergence stability against strong interference backgrounds and extremely small targets. Step 8.3 Export deployable model files, perform computation graph optimization and normalization fusion, and support half-precision or integer quantization inference; on resource-constrained platforms, prioritize the preservation of higher numerical precision of decoding ends and edge extraction units to ensure boundary quality and detail restoration.

10. A fine-grained real-time target image segmentation system based on a dynamic state modeling network, characterized in that, include: The dataset construction unit is used to collect scene image data containing fine-grained targets online, including images of ore transportation processes, road crack inspection images, and workpiece images of industrial inspection objects, and to construct datasets through annotation management. The dataset construction unit includes: An imaging platform is used to carry the object under test and operate online according to a predetermined motion or positioning method; Imaging sensor components are used to switch between X-ray detectors, industrial cameras, or linear array cameras depending on the scenario to acquire the desired images; The lighting and environmental control module is used to provide stable and controllable lighting in different detection environments, eliminate reflections or noise, and ensure clear visibility of fine-grained targets; The synchronization and positioning module is used to record metadata such as image acquisition time, platform / object position and attitude, and trigger signals, and to control imaging triggering to ensure synchronization between frames and between devices; The data acquisition and annotation management module is used to receive acquired images and associated metadata, perform preprocessing, organize manual or semi-automatic annotation generation, and build high-quality datasets required for training and inference. An image segmentation unit is used to perform fine-grained target segmentation on the image data and output structured results in real time; the image segmentation unit includes: The lightweight detail feature extraction module is used to extract multi-level features from the input image using lightweight depthwise separable convolution and complete sequential processing, outputting a multi-scale feature tensor. The multi-scale semantic feature extraction module is used to obtain mid-to-deep multi-scale semantic representations through input adaptation, selective state modeling, gated fusion and output reconstruction, and write the enhanced semantics back to the backbone in the form of residuals. The dual-scale two-dimensional selective scanning module is used to perform local and global scans of fine and coarse branches, and to update the state in the state space unit that depends on the input, thereby mitigating the long-range information decay in high-resolution scenes. The dynamic cross-scale feature selection and aggregation module is used to complete multi-scale reweighting and multi-path statistical filtering, and output aggregated features with strong discriminative power and low redundancy. The local-global fusion decoding module is used for upsampling and semantic recovery during the decoding stage, and performs detail compensation and boundary stabilization through skip connections and edge extraction subunits; A morphology-guided pseudo-label hierarchical supervision module is used to generate multi-level morphological dilation pseudo-labels and implement hierarchical supervision under two strategies: intra-scale block refinement or inter-scale layer refinement, through a pseudo-label attention mechanism. The training and inference module is used for main and auxiliary loss construction, end-to-end training, model export and quantization deployment, and integrates optimization strategies such as mixed precision and memory reuse to meet the real-time requirements of edge and vehicle platforms.

11. The system according to claim 10, characterized in that: The lightweight detail feature extraction module first performs data augmentation using random cropping, horizontal flipping, brightness and contrast perturbation, mild noise, and color jitter to enhance the model's robustness to changes in lighting, material, and scale. Then, it uses a lightweight extractor composed of depthwise separable convolutions as the backbone to perform multi-level feature extraction on the input image, obtaining multi-scale feature representations that include edge and texture priors, and sequentially processes the features to adapt to subsequent state space modeling.

12. The system according to claim 10, characterized in that: The multi-scale semantic feature extraction module includes an input adaptation submodule, a selective state modeling submodule, a gated fusion submodule, and an output reconstruction submodule. The input adaptation submodule is used to perform layer normalization and linear mapping on the features from the feature extraction end, and rearrange them into a sequence according to a preset length and channels; The selective state modeling submodule is used to divide multi-scale pathways according to receptive domains and perform selective state scanning and aggregation within a state-space framework. The gated fusion submodule is used to dynamically weight the state output at each scale according to the relevant input weights, highlighting significant areas and suppressing background interference. The output reconstruction submodule is used to map the fused sequence back to two-dimensional features through linear reconstruction and inverse transformation, and then superimpose it with the input residual to complete channel alignment.

13. The system according to claim 10, characterized in that: The dual-scale two-dimensional selective scanning module includes two pathways: a fine-branch pathway and a coarse-branch pathway; Fine-branch paths are used to maintain resolution. They first perform local partitioning and intra-block scanning, and then perform cross-scanning in the horizontal and vertical directions to propagate cross-regional context. Coarse-branched pathways are used to expand the global receptive field through downsampling and to perform multi-directional cross-scanning to obtain the global context; Both branch paths are updated through selective state space units and aligned to the spatial resolution after interpolation.

14. The system according to claim 10, characterized in that: The dynamic cross-scale feature selection and aggregation module includes a global reweighting submodule and a multi-path statistical filtering submodule; The global reweighting submodule is used to generate adaptive branch weights through global average pooling and multilayer perceptron, and to perform channel-level recalibration and residual back-fusion. The multi-path statistical filtering submodule is used to calculate the statistical differences of multiple scan paths, generate a filtering attention map, and perform point-by-point weighted aggregation to retain areas with significant structural changes while suppressing redundant and consistent areas.

15. The system according to claim 10, characterized in that: The local-global fusion decoding module includes a cascaded decoding submodule and an edge extraction submodule. The cascaded decoding submodule is used to reconstruct the spatial resolution by progressive upsampling, cross-scale state recovery and lightweight convolution refinement, and to perform skip connections and fusion with the corresponding scale features of the feature extraction side to achieve the synergy of semantic reconstruction and detail compensation. The edge extraction submodule is used to generate edge cue features before the final output. It uses a combination of differential Gaussian, Laplacian Gaussian and convolution, and fuses them with the backbone features to improve the separability and stability of weak boundaries.

16. The system according to claim 10, characterized in that: The morphology-guided pseudo-label hierarchical supervision module includes a pseudo-label generation submodule, a single-scale block intra-refinement submodule, or a multi-scale inter-layer refinement submodule. The single-scale intra-block refinement submodule and the multi-scale inter-layer refinement submodule are two optional refinement strategies. One can be enabled according to application requirements, but they cannot be enabled at the same time. The pseudo-tag generation submodule sets decreasing morphological dilation rates according to different levels, and generates pseudo-tags from coarse to fine step by step. The single-scale block refinement submodule is used to inject spatial weights of morphological dilation pseudo-labels into the final decoding layer, generate side-path outputs and apply auxiliary supervision to enhance macroscopic structural connectivity. The multi-scale interlayer refinement submodule is used to inject pseudo-label weights that match the resolution of each decoding layer, generate side-path outputs in layers, and supervise the gradual convergence of details to improve the recall of small targets and boundary accuracy.

17. The system according to claim 10, characterized in that: The training and inference module includes: The loss construction submodule is used to form the training objective with the main loss and multi-level auxiliary losses, taking into account for foreground recall, region consistency and boundary accuracy. The training stability submodule is used for managing the adaptive optimizer, learning rate scheduling, gradient pruning, mixed precision, and regularization strategies. The quantization and deployment submodule is used for half-precision or integer quantization inference and uses representative data for quantization calibration, prioritizing the numerical accuracy of end-decoding and edge extraction.

Images