Image semantic segmentation method based on gradient split and high receptive field cross-domain feature interaction and application thereof

Abstract

The application provides an image semantic segmentation method based on gradient shunt and high receptive field cross-domain feature interaction and application thereof, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an RGB image; inputting the image into a feature extraction network; separating features into shallow detail flow and deep semantic flow by using a gradient shunt module, so as to avoid deep and shallow layer gradient conflict; performing high receptive field global feature interaction by using a large kernel convolution and a cross-domain deformable cross attention mechanism, and enhancing semantic expression; and finally generating a prediction result by using a segmentation head. The application effectively solves the problems of texture blur, reflection interference and unclear boundary of a water accumulation area in a complex scene, significantly improves segmentation accuracy and robustness, and can be widely used in urban road water accumulation monitoring and traffic safety decision-making.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a deep learning-based image semantic segmentation method, which is particularly suitable for the detection and segmentation of waterlogged areas on urban roads. Background Technology

[0002] In the fields of intelligent transportation and urban safety management, accurate identification of waterlogged areas on urban roads is crucial for drainage scheduling, traffic safety early warning, and emergency management. Traditional waterlogging identification methods mostly rely on manual inspection or threshold algorithms based on color segmentation and edge detection, but they have poor robustness in complex environments such as changes in lighting, reflection interference, and shadows.

[0003] While semantic segmentation methods based on convolutional neural networks (CNNs) and Transformers have made progress with the development of deep learning, they still face significant challenges in handling water accumulation scenes. Firstly, water accumulation areas typically exhibit low texture, strong reflection, and blurred contours, with their shape changing significantly with the environment. Existing conventional networks, mostly based on small receptive field convolutions or local attention mechanisms, struggle to establish cross-regional semantic dependencies, resulting in weak overall shape recognition capabilities for large areas of water accumulation. Furthermore, they are ill-equipped to handle lighting reflection patterns over long distances, easily causing breaks in the segmentation mask over long regions.

[0004] Secondly, during the training of deep neural networks, shallow features are mainly responsible for capturing edges and textures, while high-level features are responsible for global semantics. In conventional network architectures, both typically share the same gradient update path. This mechanism can easily lead to gradients generated by deep semantic loss suppressing the learning of shallow edge features, or noisy gradients in the shallow layers interfering with the convergence of global semantics. This "gradient conflict" phenomenon is particularly evident in water accumulation segmentation tasks, directly resulting in blurred edges and loss of details in the segmentation results, especially when processing small water accumulation areas, making it difficult to achieve high-precision boundary segmentation. Summary of the Invention

[0005] This invention provides an image semantic segmentation method based on gradient splitting and cross-domain feature interaction with a high receptive field, and its application. It addresses the problems of existing technologies when facing road flooding scenes with low texture, strong reflection and blurred boundary features. Due to the lack of global semantic modeling capabilities with a large receptive field and gradient conflicts caused by the sharing of gradient paths between shallow and deep features, there are problems such as inconsistent recognition of the overall structure of the flooded area, blurred edge detail segmentation and high false negative rate.

[0006] The core technology of this invention is to decouple the deep semantic flow from the shallow detail flow by constructing a feature extraction network based on gradient splitting to avoid gradient conflicts, and to combine the XLKA module based on large kernel convolution and deformable cross attention mechanism to achieve cross-domain global feature interaction under high receptive field, thereby enhancing the global semantic modeling of water accumulation area while accurately restoring edge details.

[0007] In a first aspect, the present invention provides an image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction, comprising the following steps:

[0008] Obtain the RGB modal image to be segmented;

[0009] RGB modal images are input into a pre-constructed feature extraction network for multi-scale semantic encoding, outputting a globally enhanced semantic feature representation. The feature extraction network contains several cascaded gradient splitting modules. The gradient splitting module is used to separate the input features into two gradient streams in the channel dimension. The first gradient stream is processed through the deep backbone to extract global features with a high receptive field, and the second gradient stream serves as a jump branch to retain shallow detail features. The processed first gradient stream and the second gradient stream are then fused. The deep backbone contains a high receptive field cross-domain feature interaction submodule, which performs interactive processing on features through large kernel convolution and cross-domain cross-attention mechanism.

[0010] The semantic feature representation is passed to the semantic segmentation head to generate a classification score map of the image, thus obtaining the semantic segmentation result.

[0011] Furthermore, before inputting the RGB modal image into the feature extraction network, the following steps are also included:

[0012] The RGB modal image is mapped to an initial feature tensor through block mapping and embedding layers, and then downsampled through convolutional layers to convert the image into a feature representation.

[0013] Furthermore, the specific processing steps of the gradient splitting module include:

[0014] Perform local convolution processing on the input features; split the locally convolutioned features by channel to obtain the first gradient flow and the second gradient flow;

[0015] The first gradient stream is fed into a deep backbone stack consisting of multiple consecutive high receptive field cross-domain feature interaction sub-modules for processing.

[0016] The output of the main deep stack is concatenated with the second gradient stream along the channel dimension, and the concatenated features are fused using a convolutional layer as the output of the gradient splitting module.

[0017] Furthermore, the high receptive field cross-domain feature interaction submodule adopts a dual-branch parallel architecture, including a first branch and a second branch;

[0018] Both the first and second branches are equipped with large kernel convolutional units, which are used to extract high receptive field features on different feature maps;

[0019] The high receptive field cross-domain feature interaction submodule also includes a cross-domain deformable cross attention unit, which is used to establish pixel-level semantic associations between the first branch and the second branch;

[0020] The processing procedure for the large kernel convolutional unit is as follows:

[0021] The input feature map is fed into two large-kernel convolutional layers with different kernel sizes in succession to extract image features under a large receptive field;

[0022] The processing steps for the cross-domain deformable cross-attention unit include:

[0023] The kernel convolution outputs of the first and second branches are transformed respectively to generate their respective query vectors and key vectors;

[0024] For each query position on the first branch, the sampling offset and attention score are predicted using its query vector;

[0025] Based on the sampling offset, sampling is performed on the key vector feature map of the second branch to obtain the corresponding sampled value;

[0026] The sampled values ​​are weighted and summed based on the attention scores to obtain the cross-domain attention output of the first branch;

[0027] By using symmetric operations, the query vector of the second branch is sampled and weighted on the first branch to obtain the cross-domain attention output of the second branch.

[0028] Furthermore, sampling is performed on the key-value vector feature map of the second branch based on the sampling offset, specifically including:

[0029] The coordinates of the sampling point are obtained by adding the normalized grid reference point coordinates to the predicted sampling offset.

[0030] The feature values ​​at the sampling point coordinates are obtained on the feature map of the second branch using the bilinear interpolation algorithm.

[0031] Furthermore, the high receptive field cross-domain feature interaction submodule also includes feature modulation and fusion steps:

[0032] Linear projection is used to modulate the cross-domain attention outputs of the first and second branches element by element, and the modulation weights are generated by activation functions and applied to the original features.

[0033] The modulated features from the two paths are added together, and then residually connected to the input features of the high receptive field cross-domain feature interaction submodule through the projection layer to obtain the fused output.

[0034] Furthermore, the semantic feature representation is passed to the semantic segmentation head, including:

[0035] The semantic feature representation is restored step by step through the hierarchical upsampling module, which includes alternating convolution and upsampling operations to restore spatial resolution.

[0036] Convolutional layers are used to map the upsampled features into a single-channel classification score map.

[0037] Furthermore, the image semantic segmentation method is applied to urban road water accumulation detection. After obtaining the semantic segmentation results, the following steps are also taken:

[0038] Binarize and threshold the classification score image to obtain the water accumulation prediction mask;

[0039] The percentage of waterlogged areas is calculated based on the waterlogging prediction mask, which is used to assess road accessibility indicators.

[0040] In a second aspect, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction.

[0041] Thirdly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction described above.

[0042] The main contributions and innovations of this invention are as follows:

[0043] 1. Resolves the gradient conflict problem in shallow and deep feature learning: By introducing a gradient splitting mechanism, the feature flow is divided into a shallow detail flow and a deep semantic flow. This ensures that shallow edge information is preserved in an independent branch, avoiding the suppression of shallow edge feature learning by deep semantic gradients during backpropagation. Experiments show that this mechanism significantly reduces the boundary false negative rate (1-Recall index is significantly reduced) and improves the clarity of segmentation boundaries.

[0044] 2. Enhanced global modeling capability for blurred and reflective areas: By designing XLKA modules containing large kernel convolutions (5×5 and 7×7), the effective receptive field of the network is greatly expanded, enabling the capture of a wide range of road structures and lighting consistency; combined with a cross-domain deformable cross-attention mechanism, pixel-level global semantic interaction is achieved. This design effectively solves the segmentation and fragmentation problem caused by weak texture and strong reflectivity in waterlogged areas, ensuring the integrity and coherence of waterlogged area predictions.

[0045] 3. Achieves high accuracy and robustness in complex scenarios: While maintaining a high F1-score (89.71%) and high accuracy (92.28%), this invention significantly improves the stability of boundary prediction. Compared with mainstream models such as Mask2Former and Mobile-SAM, this invention demonstrates a more balanced overall performance, particularly in handling complex weather conditions, shadow interference, and fine-grained boundary recovery, making it suitable for smart city traffic decision-making systems with high reliability requirements.

[0046] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0048] Figure 1 This is a flowchart of an image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction according to an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of the framework structure of a network based on gradient splitting and high receptive field global feature interaction (XLKANet) according to an embodiment of the present invention.

[0050] Figure 3 This is a schematic diagram of the gradient splitting module structure according to an embodiment of the present invention.

[0051] Figure 4 This is a schematic diagram of the Large Kernel Convolutional Cross Attention Module (XLKA) according to an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0054] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0055] Example 1

[0056] like Figure 1 As shown, this invention provides an image semantic segmentation method based on gradient splitting and cross-domain feature interaction with a high receptive field, mainly applied to the detection and segmentation of waterlogged areas on urban roads. This method solves problems such as texture blurring, reflection interference, and gradient conflicts between shallow and deep layers in existing technologies by constructing a specific deep neural network (XLKANet).

[0057] The specific implementation steps of this embodiment are as follows:

[0058] Step S1: Obtain the RGB modal image of the flooded urban road and the corresponding binary semantic mask image.

[0059] First, RGB images of urban road scenes are collected as input data. For the training phase, the RGB modal images need to be processed. Manually label the waterlogged areas to obtain the target polygon bounding boxes. The pixel values ​​within the bounding box are set to 0 (or 1, depending on the specific convention; in this embodiment, the water accumulation area is the foreground), and the remaining background pixels are set to 255 (or 0), thereby generating the corresponding binary semantic mask image. .

[0060] Step S2: Feed the RGB modal image into a feature extraction network based on gradient splitting and cross-domain feature interaction with high receptive field to extract semantic features of the image.

[0061] like Figure 2-4As shown, the network architecture of the present invention includes block embedding (i.e., block and embedding layer), downsampling convolution (i.e., convolutional layer), gradient splitting module (GSM) and high receptive field cross-domain feature interaction module (XLKA).

[0062] (1) Initial feature mapping and downsampling

[0063] Input RGB modal image The input to the network first passes through patching and embedding layers, mapping it to initial feature tensors and transforming it into a form suitable for convolutional neural network processing.

[0064]

[0065] in, Where C is the feature map space size, and C is the number of channels.

[0066] Next, a 3×3 convolutional layer is used for preliminary downsampling and feature encoding to obtain the features. :

[0067]

[0068] in, BN represents the activation function (such as ReLU) and batch normalization. Indicates the convolution kernel weights. This indicates a convolution operation.

[0069] (2) Gradient splitting module (GSM) processing

[0070] To address the problem of deep semantic gradient suppression of shallow edge detail learning, this invention introduces a gradient splitting module. Several downsampling convolutional layers and gradient splitting modules are fed into the system. By providing independent gradient paths for different semantic layers, all features are prevented from sharing the same gradient update, thereby reducing redundant feature learning and improving feature stability in complex waterlogged regions.

[0071] like Figure 3 As shown, in the gradient splitting module at layer t, the processing procedure is as follows:

[0072] First, the input features Perform local feature extraction (compute local convolution):

[0073]

[0074] Subsequently, features were defined along the channel dimension. Split into two independent gradient flows:

[0075]

[0076] in, It is fed into the deep core (containing the XLKA stack) for global semantic and high receptive field feature extraction; As a shallow detail-preserving stream, it is directly transmitted to the end of the module via a jumper connection. In water accumulation detection, shallow details are crucial for identifying the edges, geometric contours, and reflective areas of water accumulation.

[0077] The main deep layer consists of three consecutive cascaded XLKA sub-modules. It uses large-kernel convolution to capture a wide range of road structures, water accumulation shapes, and lighting consistency. It achieves global semantic interaction through cross-road attention, thereby improving the ability to recognize "reflective, blurred, and weakly textured water accumulation areas".

[0078]

[0079] Finally, the three-stage characteristics of the XLKA stack output are presented. With the retention of shallow flow Stitching along the channel dimension:

[0080]

[0081] And use 1×1 convolution for fusion:

[0082]

[0083] Therefore, during backpropagation, the gradient flows back along multiple paths, resulting in shallow flow. Unaffected by gradients from deep and complex transformations, it effectively preserves detailed information about the edges of water accumulation.

[0084] (3) High receptive field cross-domain feature interaction module (XLKA)

[0085] The XLKA module is the core of feature extraction, such as... Figure 4 As shown, a dual-branch parallel architecture is adopted, which combines large kernel convolution (LKA) with cross-domain deformable cross attention.

[0086] A. Large kernel convolution for high receptive field extraction

[0087] Input features The data is fed into two separate branches, passing through large-kernel convolutional modules (LKA) of 5×5 and 7×7 respectively, to obtain an ultra-large receptive field:

[0088]

[0089] Note: Here, v is obtained by further processing based on u. However, in the dual-path interactive design, u and v can be regarded as feature representations under two receptive fields of different scales (or z can be fed into two convolutions with different kernel sizes). In this preferred embodiment, a cascaded approach is used to obtain receptive field features u and v at different levels (both belonging to...). ).

[0090] B. Cross-domain query / key value generation

[0091] Perform a 1×1 convolution transformation on u and v and flatten them to generate Query (Q) and Value (V) for attention:

[0092]

[0093]

[0094] C. Deformable cross-domain cross-attention mechanism

[0095] With position p=(x,y) as the reference point, define the normalized center coordinates:

[0096]

[0097] For the upper-path feature (as a Query), for each query position p, its corresponding... vector Predict the (two-dimensional offset) of S sampling points using a lightweight network (OffsetNet). And attention weights (raw score) :

[0098]

[0099] in, Implemented as a 1×1 convolutional layer. This is a learnable linear mapping. Calculate the normalized coordinates of the sampling points:

[0100]

[0101] At the sampling location At this point, bilinear interpolation is used. In the bottom lane characteristics Get the sampled value (as Value):

[0102]

[0103] Softmax normalization is applied to the predicted attention score (i.e., by...) (Normalized attention weights generated directly on S sampling points)

[0104]

[0105] Calculate the cross-domain attention output at the top lane position p (i.e., obtain the deformable cross-domain attention output at the top lane position p as a weighted sum of the sampled values):

[0106]

[0107] Similarly, perform a symmetric operation, using the lower path v as the query and the upper path u as the value, to compute the downward attention output. .

[0108] D. Feature reshaping and modulation fusion

[0109] Reshape the two attention outputs back to the spatial dimension:

[0110]

[0111]

[0112] By using a gating mechanism with Sigmoid activation to modulate the original features (i.e., using linear projection to modulate the attention outputs of both paths element-wise), background noise is suppressed and the features of the water accumulation area are enhanced.

[0113]

[0114]

[0115] Finally, the two features are fused and residual connections are added to obtain the final output of the XLKA module. :

[0116]

[0117] in, It is a 1×1 convolutional layer. is the residual projection of input z.

[0118] Step S3: Pass the semantic feature representation to the semantic segmentation head to generate a classification score map of the image, thereby accurately predicting the target location.

[0119] Global augmentation features output by the feature extraction network It is fed into the decoder.

[0120] (1) Hierarchical upsampling

[0121] To restore the original resolution of the image, a hierarchical upsampling module is used. :

[0122]

[0123] The specific operation is defined as a multi-iterative process of "convolution-normalization-activation-upsampling":

[0124]

[0125] in, This represents bilinear interpolation upsampling with a scale of 2. It is a 3×3 convolution.

[0126] (2) Segmentation prediction and post-processing

[0127] A single-channel classification score map is generated using a 7×7 convolutional layer (semantic segmentation head):

[0128]

[0129] Generate the final binary water accumulation prediction mask based on the score map. :

[0130]

[0131] Among them, threshold Typically, a value of 0.5 is used. This mask image can accurately identify the area and spatial distribution of water accumulation on urban roads.

[0132] Furthermore, based on the aforementioned prediction mask, road accessibility indices can be calculated. Used to assist in traffic decision-making:

[0133]

[0134] in, This represents the pixel area of ​​the predicted water accumulation area. This represents the pixel area of ​​the entire road region. If If the water level is below a preset value, the system can automatically determine that there is a serious risk of water accumulation on the road section and issue detour instructions or risk warnings through the traffic management platform, thereby realizing intelligent urban road safety decision-making.

[0135] In this embodiment, the model training and loss function are as follows:

[0136] In the training phase, this invention employs a combined loss function to optimize network parameters. For the prediction results With real labels Supervision:

[0137]

[0138] in, and These are the weighting coefficients for the two loss terms, used to balance pixel-level accuracy with overall region consistency. The binary cross-entropy loss... Used for pixel-level classification optimization:

[0139]

[0140] Dice coefficient loss Used to solve the problem of imbalance between positive and negative samples and improve the overlap of regions:

[0141]

[0142] Where N is the total number of pixels. This is a smoothing term. Through joint optimization, the model maintains boundary clarity while ensuring the coherence within the waterlogged area.

[0143] Example 2

[0144] To demonstrate the individual and combined contributions of the two key improvements in this invention (gradient splitting and deformable cross-domain attention) to segmentation performance, especially boundary accuracy, we designed the ablation experiments shown in the table below, in which all models were trained on the same dataset and under the same training protocol to obtain robust statistics.

[0145] (1) Baseline model training

[0146] We construct the most basic network structure, without introducing gradient splitting or deformable cross-domain attention; we only use the standard channel attention module. We keep the backbone, loss function, optimizer, and other configurations completely consistent. We fully train the model and record the F1-score, precision, and 1-recall metrics on the test set.

[0147] (2) Baseline-v2 training (with gradient splitting only)

[0148] A gradient splitting mechanism is added to the baseline structure, changing only the gradient propagation path without introducing any changes to the attention structure. The training period, batch size, and learning rate are kept constant, and the model is retrained. After training, the above metrics are evaluated on the same test set to observe the impact of gradient splitting on the stability and accuracy of boundary prediction.

[0149] (3) Baseline-v3 (with only deformable cross-domain cross attention added) training

[0150] The standard channel attention module is replaced with the deformable cross-domain attention structure proposed in this invention, but gradient splitting is not used. Other training settings remain unchanged. After training, the same metrics are evaluated on the test set to observe the contribution of cross-domain contextual interactions to feature representation capabilities.

[0151] (4) Training of this invention (gradient splitting + deformable cross-domain cross attention)

[0152] Under the same architecture, both gradient splitting and deformable cross-domain attention modules are used to perform complete training. This model is the final version of this invention. Its test set metrics are compared with the aforementioned three groups to demonstrate the performance gain when the two techniques are used in combination.

[0153] All four models above are trained using the same steps:

[0154] ① Consistent data preprocessing, including size normalization and random augmentation;

[0155] ② Use the same combination of loss functions (e.g., BCE+Dice);

[0156] ③ Share a unified optimizer (such as AdamW) and a fixed number of training rounds;

[0157] ④ After training is complete, perform binarized inference with the same threshold;

[0158] ⑤ Use the same evaluation script to calculate F1-score, Precision, and 1-Recall.

[0159] The experimental results are shown in Table 1 below:

[0160] Table 1

[0161]

[0162] The experimental results in the table show that:

[0163] 1. Gradient splitting leads to a significant improvement in boundary accuracy:

[0164] Compared to Baseline, Baseline-v2 reduces 1-Recall from 55.6 to 24.4, indicating that gradient splitting effectively suppresses boundary false negatives.

[0165] At the same time, the F1 score also improved significantly (75.71 → 84.33).

[0166] 2. Deformable cross-domain attention effectively enhances contextual expression capabilities:

[0167] Baseline-v3 improves the F1 score (75.71 → 79.64) by cross-domain feature interaction while keeping the conventional feature extraction unchanged, indicating that the module plays a role in enhancing high-level semantic consistency and target integrity.

[0168] 3. Combining the two yields optimal performance:

[0169] The final model of this invention achieved an F1-score of 89.71, a precision of 92.28, and the lowest 1-recall (12.72), indicating that gradient splitting and deformable cross-domain attention are complementary.

[0170] One improves boundary stability, and the other enhances cross-domain context, enabling the overall model performance to reach its optimal level.

[0171] In summary, the ablation experiments fully verify the positive contribution of the two modules proposed in this invention to the segmentation performance (especially the boundary detection index), and the effect is most significant when the two are combined, which has the technical creativity and substantial progress.

[0172] In this embodiment, to verify the overall effectiveness of the model architecture of this invention, we selected several representative mainstream segmentation models, including Mask2Former based on DETR, the lightweight segmentation model mobile_sam based on SAM distillation, RS-mamba based on the Mamba architecture, and yolov11-seg based on the YOLO architecture, for comparative experiments. All models were trained under the same dataset and training protocol. The experimental results are shown in Table 2 below:

[0173] Table 2

[0174]

[0175] The following conclusions can be drawn from Table 2:

[0176] (1) The model of this invention achieves the highest value in overall segmentation performance (F1-score).

[0177] XLKANet achieved an F1 score of 89.71, the highest among all compared models, significantly outperforming other representative architectures. This demonstrates that the model of this invention has stronger representational capabilities in terms of target region consistency and boundary closure.

[0178] (2) Significantly outperforms other models in precision.

[0179] XLKANet achieves a precision of 92.28, which is also higher than all baseline models, including Mask2Former and mobile_sam. This result indicates that the model has higher localization accuracy and lower false detection rate in the foreground region, making it particularly suitable for scenes with complex boundary structures and weak texture contrast.

[0180] (3) It performs well in terms of boundary prediction stability (1-Recall).

[0181] A lower 1-recall indicates fewer missed detections and a higher degree of boundary closure. XLKANet's 1-recall is 12.72, only slightly higher than the Mask2Former method. In contrast, other mainstream models are less stable in terms of boundary-related metrics, especially methods based on YOLO and Mamba, which show more pronounced missed detections at boundaries.

[0182] (4) The inherent characteristics of different architectures cause differences, but the model of this invention performs more balanced.

[0183] As can be seen from the table:

[0184] Mask2Former (DETR architecture) performs well in large structural areas, but lacks in boundary details;

[0185] mobile_sam (SAM distillation) has a high ability to locate the subject, but it has the problem of missing detection in small-scale boundary areas;

[0186] RS-mamba (Sequence State Model) is effective for long-range modeling, but has limited ability to resolve local structures.

[0187] The yolov11-seg (YOLO architecture) is geared towards real-time optimization, but it is prone to undersegmentation when performing fine-grained segmentation.

[0188] Compared to the biased performance of the aforementioned architectures, the model of this invention achieves simultaneous leadership in three metrics: accuracy, boundary stability, and overall segmentation performance, demonstrating a more balanced and stable performance.

[0189] Example 3

[0190] This embodiment also provides an electronic device, see reference. Figure 5 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0191] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0192] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0193] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0194] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the image semantic segmentation methods based on gradient splitting and high receptive field cross-domain feature interaction in the above embodiments.

[0195] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0196] Transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0197] Input / output device 408 is used to input or output information.

[0198] Example 4

[0199] This embodiment also provides a readable storage medium storing a computer program, the computer program including program code for controlling the process to execute the process, the process including the image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction according to Embodiment 1.

[0200] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0201] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0202] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 1 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0203] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. An image semantic segmentation method based on gradient splitting and high receptive field cross-domain feature interaction, characterized in that, Includes the following steps: Obtain the RGB modal image to be segmented; The RGB modal image is input into a pre-constructed feature extraction network for multi-scale semantic encoding, outputting a globally enhanced semantic feature representation. The feature extraction network includes several cascaded gradient splitting modules. Each gradient splitting module separates the input features into two gradient streams along the channel dimension. The first gradient stream is processed through a deep backbone layer to extract high receptive field global features, while the second gradient stream serves as a jumper branch to retain shallow detail features. The processed first and second gradient streams are then fused. The deep backbone layer includes a high receptive field cross-domain feature interaction submodule, which employs a dual-branch parallel architecture, including a first branch and a second branch. Both the first and second branches are equipped with large-kernel convolutional units for extracting high receptive field features on different feature maps. The high receptive field cross-domain feature interaction submodule also includes a cross-domain deformable cross-attention unit for extracting high receptive field features from the first... A pixel-level semantic association is established between the first and second branches. The processing of the large-kernel convolutional unit involves continuously feeding the input feature map into two large-kernel convolutional layers with different kernel sizes to extract image features within a large receptive field. The processing of the cross-domain deformable cross-attention unit includes: transforming the large-kernel convolutional outputs of the first and second branches respectively to generate their respective query vectors and value vectors; predicting the sampling offset and attention score for each query position on the first branch using its query vector; sampling the value vector feature map of the second branch based on the sampling offset to obtain the corresponding sampled value; weighting and summing the sampled values ​​based on the attention score to obtain the cross-domain attention output of the first branch; and using symmetric operations, sampling and weighting the query vector of the second branch on the first branch to obtain the cross-domain attention output of the second branch. The semantic feature representation is passed to the semantic segmentation head to generate a classification score map of the image, thus obtaining the semantic segmentation result.

2. The image semantic segmentation method as described in claim 1, characterized in that, Before inputting the RGB modal image into the feature extraction network, the following steps are also included: The RGB modal image is mapped to an initial feature tensor through block mapping and embedding layers, and then downsampled through convolutional layers to convert the image into a feature representation.

3. The image semantic segmentation method as described in claim 1, characterized in that, The specific processing steps of the gradient splitting module include: Perform local convolution processing on the input features; split the locally convolutioned features by channel to obtain the first gradient flow and the second gradient flow; The first gradient stream is fed into a deep backbone stack consisting of multiple consecutive high receptive field cross-domain feature interaction sub-modules for processing. The output of the main deep stack is concatenated with the second gradient flow along the channel dimension, and the concatenated features are fused using a convolutional layer as the output of the gradient splitting module.

4. The image semantic segmentation method as described in claim 1, characterized in that, Sampling is performed on the key vector feature map of the second branch based on the sampling offset, specifically including: The coordinates of the sampling point are obtained by adding the normalized grid reference point coordinates to the predicted sampling offset. The feature values ​​at the coordinates of the sampling point are obtained on the feature map of the second branch using the bilinear interpolation algorithm.

5. The image semantic segmentation method as described in claim 1, characterized in that, The high receptive field cross-domain feature interaction submodule also includes feature modulation and fusion steps: Linear projection is used to modulate the cross-domain attention outputs of the first and second branches element by element, and the modulation weights are generated by activation functions and applied to the original features. The modulated features from the two paths are added together, and then residually connected to the input features of the high receptive field cross-domain feature interaction submodule through the projection layer to obtain the fused output.

6. The image semantic segmentation method as described in claim 1, characterized in that, Passing the semantic feature representation to the semantic segmentation head includes: The semantic feature representation is restored step by step through a hierarchical upsampling module, which includes alternating convolution and upsampling operations to restore spatial resolution. Convolutional layers are used to map the upsampled features into a single-channel classification score map.

7. The image semantic segmentation method as described in claim 6, characterized in that, The image semantic segmentation method is applied to urban road water accumulation detection, and after obtaining the semantic segmentation result, it further includes: The classification score map is binarized and thresholded to obtain the water accumulation prediction mask; The percentage of waterlogged area is calculated based on the waterlogging prediction mask and used to assess road passability indicators.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the image semantic segmentation method according to any one of claims 1 to 7.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling the process to execute the process, the process including the image semantic segmentation method according to any one of claims 1 to 7.

Images