An image segmentation method, device and equipment based on channel shuffle and a medium

Abstract

The application discloses a channel shuffle-based image segmentation method and device, equipment and medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a target image to be segmented, and performing down-sampling on the target image to be segmented to obtain a feature map; the feature map is input into a local branch and a global branch of a target neural network in parallel, so as to extract fine-grained features and macro semantic features; the local branch obtains a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtains the fine-grained features based on the multi-scale feature tensor and a channel shuffle technology; the fine-grained features and the macro semantic features are adaptively fused to obtain corresponding target multi-scale features, and the target multi-scale features are up-sampled to perform image segmentation. Through adaptive fusion of the fine-grained features and the macro semantic features, dynamic integration of the two-way features is realized, and the problem of information conflict caused by a semantic gap in static fusion is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image segmentation method, apparatus, device and medium based on channel shuffling. Background Technology

[0002] Image semantic segmentation is a core task in computer vision, aiming to classify each pixel in an image to distinguish different target regions. This technology is widely used in fields such as autonomous driving, medical image analysis, and intelligent security.

[0003] Current image segmentation methods utilize a two-stream network architecture. When fusing features from two branches, existing two-stream networks mostly employ a static fusion approach, such as simple linear addition or channel concatenation. This approach suffers from information conflicts caused by semantic gaps between multi-scale features. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an image segmentation method, apparatus, device, and medium based on channel shuffling, which can achieve dynamic integration of dual-channel features through adaptive fusion of fine-grained features and macro-semantic features, thus solving the information conflict problem caused by semantic gap in static fusion. The specific solution is as follows: In a first aspect, this application discloses an image segmentation method based on channel shuffling, comprising: The target image to be segmented is acquired, and the target neural network is used to downsample the target image to be segmented in order to obtain the feature map corresponding to the target image to be segmented. The feature map is input in parallel to the local and global branches of the target neural network, so that the local branches can extract corresponding fine-grained features and the global branches can extract corresponding macroscopic semantic features. Specifically, the local branches obtain a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensor, the attention weight map corresponding to the feature map, and channel shuffling techniques. The local branches include a first sub-branch and a second sub-branch. The first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolutions of different sizes. Adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features. Then, upsample the target multi-scale features using the target decoder corresponding to the target neural network to obtain a target image mask with the same resolution as the target image to be segmented. Finally, perform image segmentation based on the target image mask. The target weights are determined according to the uncertainty score map corresponding to the macroscopic semantic features.

[0005] Optionally, obtaining the target image to be segmented includes: An initial image to be segmented is obtained using a preset data interface, and the initial image to be segmented is preprocessed to obtain the target image to be segmented; wherein, the preprocessing operations performed on the initial image to be segmented include image noise reduction and image size normalization.

[0006] Optionally, the process of extracting the fine-grained features from the local branches includes: The feature map is input in parallel to the first sub-branch and the second sub-branch, and the corresponding attention weight map is obtained in the first sub-branch using the window self-attention mechanism and the feature map; In the second sub-branch, the feature map is multi-scale convolution is performed using convolution kernels of different sizes, and the outputs of each convolution kernel are concatenated to obtain the multi-scale feature tensor. The fine-grained features are obtained based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique.

[0007] Optionally, obtaining the multi-scale feature tensor based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique includes: The attention weight map is determined as the target gating, and the multi-scale feature tensor is operated on element-wise based on the target gating to obtain the interactive features. The interactive features are then subjected to channel shuffling to obtain the fine-grained features.

[0008] Optionally, the process of extracting the macroscopic semantic features using the global branch includes: The feature map is subjected to dimensionality reduction processing to obtain the corresponding dimensionality-reduced features; The dimensionality-reduced features are subjected to feature smoothing to obtain a first processing result, and the dimensionality-reduced features are subjected to serialization to obtain a second processing result. The first processing result and the second processing result are aggregated to obtain the corresponding aggregated features, and the aggregated features are then subjected to dimensionality-upgrading processing to obtain the macroscopic semantic features.

[0009] Optionally, the step of adaptively fusing the fine-grained features and the macro-semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features includes: Extract the deep semantic features corresponding to the macro semantic features, and obtain the spatial attention map and the uncertainty score map corresponding to the macro semantic features based on the deep semantic features; The target weight is determined based on the spatial attention map and the uncertainty score map. The fine-grained features are then weighted using the target weight to obtain weighted fine-grained features. The weighted fine-grained features are then concatenated with the macro-semantic features to obtain the target multi-scale features.

[0010] Optionally, the upsampling of the target multi-scale features using the target decoder corresponding to the target neural network includes: The target decoder is used to perform stepwise bilinear interpolation upsampling on the multi-scale features of the target. In each upsampling process, the target decoder performs bilinear interpolation upsampling on the current input features, concatenates the corresponding upsampled features with the jump connection features of the corresponding stage of the encoder, and uses the concatenated features as the input features for the next upsampling process.

[0011] Secondly, this application provides an image segmentation apparatus based on channel shuffling, comprising: The image downsampling module is used to acquire the target image to be segmented and to downsample the target image to be segmented using a target neural network to obtain the feature map corresponding to the target image to be segmented. A macro-semantic feature extraction module is used to input the feature map in parallel to the local and global branches of the target neural network, so as to extract corresponding fine-grained features using the local branches and extract corresponding macro-semantic features using the global branches. Specifically, the local branches obtain a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensor, the attention weight map corresponding to the feature map, and channel shuffling technology. The local branches include a first sub-branch and a second sub-branch. The first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolutions of different sizes. The image segmentation module is used to adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features, and to upsample the target multi-scale features using the target decoder corresponding to the target neural network to obtain a target image mask with the same resolution as the target image to be segmented, and to perform image segmentation based on the target image mask; wherein, the target weights are weights determined according to the uncertainty score map corresponding to the macroscopic semantic features.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned channel-wash-based image segmentation method.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the aforementioned channel-wash-based image segmentation method.

[0014] This application first acquires a target image to be segmented, and then downsamples the target image to be segmented using a target neural network to obtain a feature map corresponding to the target image to be segmented. The feature map is then input in parallel into the local and global branches of the target neural network, so that the local branches extract corresponding fine-grained features, and the global branches extract corresponding macroscopic semantic features. Specifically, the local branches perform multi-scale convolution on the feature map to obtain a multi-scale feature tensor, and obtain the fine-grained features based on the multi-scale feature tensor and channel shuffling technology. Finally, the fine-grained features and the macroscopic semantic features are adaptively fused to obtain corresponding target multi-scale features. The target decoder corresponding to the target neural network then upsamples the target multi-scale features to obtain a target image mask with the same resolution as the target image to be segmented, and image segmentation is performed based on the target image mask. Therefore, this application achieves enhanced representation of multi-scale fine-grained features by utilizing local branches to perform multi-scale convolution on feature maps and fusing channel shuffling technology, effectively avoiding the submersion of small target features by background noise; by extracting macroscopic semantic features in parallel through global branches, it achieves the capture of global image dependencies, providing a global contextual prior for subsequent fusion; by adaptively fusing fine-grained features and macroscopic semantic features, it achieves dynamic integration of dual-path features, solving the information conflict problem caused by semantic gap in static fusion. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the image segmentation method based on channel shuffling disclosed in this application; Figure 2 This is a schematic diagram of a cross-branch interaction mechanism disclosed in this application; Figure 3 This is a schematic diagram of a global feature construction process disclosed in this application; Figure 4 This is a schematic diagram of a feature fusion process disclosed in this application; Figure 5 This is a flowchart of an image segmentation method disclosed in this application; Figure 6 This is a schematic diagram of an image segmentation device based on channel shuffling disclosed in this application; Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 As shown, this embodiment of the invention discloses an image segmentation method based on channel shuffling, including: Step S11: Obtain the target image to be segmented, and use the target neural network to downsample the target image to be segmented to obtain the feature map corresponding to the target image to be segmented.

[0019] To address the challenges of existing complex visual tasks where single feature extraction mechanisms struggle to balance fine-grained local information with global macroscopic semantic dependencies, and where information conflicts easily arise during feature fusion due to semantic gaps between multi-scale features, this embodiment proposes a general image segmentation method. This method achieves efficient, robust, and adaptive extraction and fusion of image features through carefully designed local branches, global branches, and an uncertainty-guided fusion module.

[0020] In this embodiment, the process of obtaining the target image to be segmented includes: obtaining the initial image to be segmented using a preset data interface, and preprocessing the initial image to be segmented to obtain the target image to be segmented; wherein, the preprocessing operation performed on the initial image to be segmented includes image noise reduction and image size standardization.

[0021] That is, the network model designed in this embodiment first receives the original input image (i.e., the initial image to be segmented). (Where C, H, and W represent the number of channels, height, and width of the original image, respectively), and then perform normalization preprocessing. The image normalization process includes operations such as image noise reduction and image resizing.

[0022] Subsequently, in order to effectively map low-level pixel information to a high-dimensional semantic feature space and reduce computational complexity, the standardized image (i.e., the target image to be segmented) is first downsampled to obtain a preliminary feature map, which is then fed into the subsequent local and global branches. Image preprocessing ensures the usability of the image.

[0023] Step S12: Input the feature map in parallel to the local and global branches of the target neural network, so as to extract corresponding fine-grained features using the local branches and extract corresponding macroscopic semantic features using the global branches; wherein, the local branches obtain multi-scale feature tensors by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensors, the attention weight map corresponding to the feature map, and channel shuffling technology; the local branches include a first sub-branch and a second sub-branch, the first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolution of different sizes.

[0024] In this embodiment, the process of extracting the fine-grained features in the local branch includes: inputting the feature map in parallel to the first sub-branch and the second sub-branch, and obtaining the corresponding attention weight map in the first sub-branch using the window self-attention mechanism and the feature map; performing multi-scale convolution on the feature map using convolution kernels of different sizes in the second sub-branch, and concatenating the output results of each convolution kernel to obtain a multi-scale feature tensor; and obtaining fine-grained features based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique.

[0025] The process of obtaining fine-grained features based on attention weight maps, multi-scale feature tensors, and channel shuffling techniques specifically includes: determining the attention weight map as the target gating, performing element-wise operations on the multi-scale feature tensor based on the target gating to obtain the interactive features, and performing channel shuffling on the interactive features to obtain fine-grained features.

[0026] Specifically, in the local branches, to address the problem of insufficient feature extraction of small targets due to the limited receptive field of a single convolutional kernel and noise interference, an improved multi-scale channel shuffling attention module is constructed to extract high-precision local fine-grained features L (i.e., fine-grained features). The specific process is as follows: the network splits the input feature map into two parallel sub-branches. The first sub-branch uses a window-based self-attention mechanism to calculate the correlation between pixels within a local window, generating a highly sensitive spatial attention weight map (i.e., attention weight map). The second sub-branch uses three convolutional layers with kernels of 3×3, 5×5, and 7×7 in parallel to capture features at different physical scales. Then, the outputs of these three branches are concatenated along the channel dimension to form a multi-scale feature tensor. The core lies in introducing a cross-branch interaction (CBI) mechanism, such as... Figure 2 As shown, the attention weight map generated by the first sub-branch is used as a gating mechanism to perform element-wise multiplication on the multi-scale features concatenated from the second sub-branch, thereby suppressing background noise and enhancing the response to specific targets. Finally, channel shuffling is performed on the interactive features (i.e., the interactive features) to promote information flow between different groups of features and output highly robust local features L (i.e., fine-grained features).

[0027] In addition, in this embodiment, the process of extracting macroscopic semantic features from global branches includes: performing dimensionality reduction processing on the feature map to obtain corresponding dimensionality reduction features; performing feature smoothing processing on the dimensionality reduction features to obtain corresponding first processing results, and performing serialization processing on the dimensionality reduction features to obtain corresponding second processing results; aggregating the first processing results and the second processing results to obtain corresponding aggregated features, and performing dimensionality increase processing on the aggregated features to obtain macroscopic semantic features.

[0028] Specifically, in the global branch, to overcome the lack of long-range perception capability in local branches and the excessively high computational complexity of traditional global attention mechanisms, this embodiment constructs an efficient architecture based on a State Space Adapter (SSA), aiming to generate macroscopic semantic features G with linear complexity. The architecture is as follows... Figure 3As shown, firstly, a 1×1 convolutional layer projects the input features into a low-rank space to reduce subsequent computational load. Then, the features (i.e., the dimensionality-reduced features) are split into two parallel paths: one path utilizes a 3×3 depthwise separable convolution to provide feature smoothing and optimize training stability; the other core global scanning path introduces the Mini-SS2D operator, which performs multi-directional serialization of features and executes selective scanning with linear complexity to achieve pixel-level long-distance interactions across the entire image. Finally, the outputs of the two paths are aggregated and restored to the original dimension through a 1×1 convolutional layer, completing the construction of the global feature G (i.e., macroscopic semantic features).

[0029] Step S13: Adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features, and use the target decoder corresponding to the target neural network to upsample the target multi-scale features to obtain a target image mask with the same resolution as the target image to be segmented, and perform image segmentation based on the target image mask; wherein, the target weights are weights determined according to the uncertainty score map corresponding to the macroscopic semantic features.

[0030] In this embodiment, fine-grained features and macro-semantic features are adaptively fused according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features. This includes: extracting deep semantic features corresponding to the macro-semantic features, and obtaining spatial attention maps and uncertainty score maps corresponding to the macro-semantic features based on the deep semantic features; determining target weights based on the spatial attention maps and uncertainty score maps; weighting the fine-grained features using the target weights to obtain weighted fine-grained features; and concatenating the weighted fine-grained features with the macro-semantic features to obtain target multi-scale features.

[0031] Specifically, given the significant differences in semantic level and trust level between the macroscopic features G (i.e., macroscopic semantic features) of the global branch output and the fine-grained features L of the local branch output, this embodiment designs an uncertainty-gated fusion module (UGF) for integration. For example... Figure 4As shown, its working logic is as follows: Taking the global feature G as prior input, deep semantics (i.e., deep semantic features) are first extracted through a shared convolutional feature extractor. Then, the features are split into two parallel 1×1 convolutional prediction heads to infer the pixel-level spatial attention map S and uncertainty score map U, respectively. The core lies in constructing a dynamic weight formula W = (1 - U) × S + U, which adaptively adjusts the representation degree of local features L using uncertainty U: in regions with high global confidence (lower U), attention map S is mainly relied upon to accurately filter local details; while in regions with high uncertainty (higher U), the weights are forced to approach 1 to retain more original local information as a supplement. Finally, the local features modulated by weight W are concatenated with the original global feature G to output the final fused features (i.e., target multi-scale features).

[0032] In addition, the target decoder corresponding to the target neural network is used to upsample the target multi-scale features, including: using the target decoder to perform stepwise bilinear interpolation upsampling on the target multi-scale features; wherein, in each upsampling process, the target decoder performs bilinear interpolation upsampling on the current input features, concatenates the corresponding upsampled features with the jump connection features of the corresponding stage of the encoder, and uses the concatenated features as the input features of the next upsampling process.

[0033] That is, the multi-scale features obtained after UGF fusion are input into the decoder. The resolution is gradually restored by adopting a strategy that combines skip fusion and efficient upsampling. Each level of decoding first performs bilinear interpolation upsampling on the current feature, then concatenates it with skip connection features of the same resolution and fuses them through convolution, finally generating a pixel-level prediction mask with the same resolution as the original image.

[0034] In summary, the image segmentation process in this embodiment is as follows: Figure 5As shown, a general image segmentation network based on channel shuffling attention and visual state space guidance is constructed. This network first downsamples the standardized preprocessed input image to initially map high-dimensional semantic features. Then, it enters a dual-path parallel feature extraction backbone network consisting of four stages. Local branches couple window self-attention and multi-scale convolution through a cross-branch interaction mechanism (CBI) to capture robust fine-grained features, while global branches utilize the Mini-SS2D operator in the State Space Adapter (SSA) to establish long-distance pixel interactions with linear complexity. The Uncertainty Gated Fusion (UGF) module uses the spatial attention map S inferred from global features and the uncertainty score U to construct a dynamic weight formula W = (1 - U) × S + U, thereby adaptively adjusting the fusion representation degree of local details and global semantics. Finally, the fused features are input into a decoder employing skip connections and an efficient upsampling strategy, generating the final pixel-level prediction mask through hierarchical feature aggregation and convolutional reconstruction layers.

[0035] In one specific implementation, the general image segmentation method of this embodiment is configured as an in-vehicle visual perception system for high-precision semantic segmentation in complex road environments. .

[0036] Data acquisition and preprocessing: Acquire high-resolution real-time images of complex road scenes (such as urban congested roads and rainy / snowy weather conditions) from the vehicle-mounted vision sensor as the raw input images.

[0037] Small Target and Background Noise Processing (Based on CBI Module): In real-world road scenarios, distant traffic signs or pedestrians typically exhibit features at a small physical scale and are easily "submerged" by complex background noise such as foliage shadows and road surface water. The local branch described in this invention utilizes a cross-branch interaction (CBI) mechanism, employing a weight map generated by window self-attention as a gating mechanism to calibrate features extracted by multi-scale convolution. This process effectively filters background noise such as cluttered road surface textures and significantly enhances the feature response of small targets such as distant traffic lights and traffic cones, avoiding the missed detection problems caused by multiple downsampling in existing FCN or U-Net models.

[0038] Long-distance road topology perception (based on SSA module): For lane lines and road boundaries with strong coherence, global branches are scanned sequentially in multiple directions using the Mini-SS2D operator. This rapidly captures the macroscopic geometric topology of the entire image with linear complexity, providing reliable global prior information for vehicle path planning.

[0039] Adaptive calibration of fuzzy boundaries (based on UGF module): When encountering blurred lane lines and road surface edges due to strong light reflection or rain and snow, the uncertainty-gated fusion module (UGF) outputs a pixel-level uncertainty score U. For fuzzy boundary regions (i.e., regions with high U values), the system forces the weight W to approach 1 using the dynamic weight formula W=(1-U)×S+U, thereby preserving more original details of local branches, effectively preventing the propagation of "confidence" in erroneous semantic features, and outputting sharp-edged and accurately classified road and obstacle masks.

[0040] In another specific implementation, the general image segmentation method in this embodiment is configured as an intelligent security visual analysis system, deployed on edge computing nodes or in the cloud, for high-precision target segmentation and abnormal behavior detection in complex lighting and crowded scenes.

[0041] Tiny Target Capture and Low-Light Noise Suppression (Based on CBI Module): In wide-angle surveillance footage, distant pedestrians, intruders, or dangerous abandoned objects (such as unattended suitcases) are typically weak features at the pixel level. Simultaneously, large areas of low light in nighttime surveillance footage generate severe snow-like noise (background noise). In this system, the local branch utilizes a cross-branch interaction (CBI) mechanism, using the window self-attention weights generated by the first sub-branch as gating, to perform element-wise multiplication calibration on the features extracted by the second sub-branch (multi-scale convolution). This mechanism plays a role in "intelligent noise reduction and micro-feature amplification" in intelligent security scenarios, effectively suppressing large areas of random background noise at night and significantly enhancing the feature response of small targets in low light conditions at a distance, thus preventing the security system from missing detections of distant anomalies.

[0042] Low-computing-power global scene understanding (based on SSA module): Security monitoring not only needs to focus on local targets but also needs to understand the global context (e.g., identifying the spatial separation relationship between a "suitcase" and a "pedestrian leaving in the distance" to determine if it is "left behind"). However, security edge devices typically have limited computing power and cannot run highly complex global attention models. This invention introduces a Mini-SS2D operator based on Visual State Space Adapter (SSA) into the global branch, enabling multi-directional sequential scanning of the entire monitoring image. This allows the monitoring network to establish long-distance pixel interactions across the entire image with extremely low computational load, thereby meeting the stringent real-time processing requirements of security systems for video streams.

[0043] Handling Crowd Occlusion and Blurred Boundaries at Night (Based on UGF Module): In crowded scenes (such as subway stations during rush hour) or with insufficient lighting, pedestrians often occlude each other, and the boundaries between people and the background are often blurred due to motion blur or shadows. In such situations, conventional networks are prone to semantic conflicts or classification confusion. The Uncertainty Gated Fusion (UGF) module in this embodiment can accurately capture the pixel-level inference uncertainty score U at the boundaries of these occlusions and shadows. In areas with high occlusion or blurred boundaries, the system forcibly increases the weight W using the formula W=(1-U)×S+U, adaptively supplementing more fine-grained original information from local branches L. Ultimately, the system can adaptively calibrate the boundary blurring caused by occlusion and low light, outputting pedestrian and target masks with clear boundaries and accurate individual separation, providing highly reliable underlying visual data for subsequent security behavior analysis.

[0044] Therefore, this application achieves enhanced representation of multi-scale fine-grained features by utilizing local branches to perform multi-scale convolution on feature maps and fusing channel shuffling technology, effectively avoiding the submersion of small target features by background noise; by extracting macroscopic semantic features in parallel through global branches, it achieves the capture of global image dependencies, providing a global contextual prior for subsequent fusion; by adaptively fusing fine-grained features and macroscopic semantic features, it achieves dynamic integration of dual-path features, solving the information conflict problem caused by semantic gap in static fusion.

[0045] See Figure 6 As shown, an embodiment of the present invention discloses an image segmentation device based on channel shuffling, comprising: The image downsampling module 11 is used to acquire the target image to be segmented and to downsample the target image to be segmented using the target neural network to obtain the feature map corresponding to the target image to be segmented. The macro-semantic feature extraction module 12 is used to input the feature map in parallel to the local and global branches of the target neural network, so as to extract corresponding fine-grained features using the local branches and extract corresponding macro-semantic features using the global branches; wherein, the local branches obtain a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensor, the attention weight map corresponding to the feature map, and channel shuffling technology; the local branches include a first sub-branch and a second sub-branch, the first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolution of different sizes; Image segmentation module 13 is used to adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features, and to upsample the target multi-scale features using the target decoder corresponding to the target neural network to obtain a target image mask with the same resolution as the target image to be segmented, and to perform image segmentation based on the target image mask; wherein, the target weights are weights determined according to the uncertainty score map corresponding to the macroscopic semantic features.

[0046] In some specific embodiments, the image downsampling module 11 may specifically include: An image preprocessing unit is used to acquire an initial image to be segmented using a preset data interface, and to preprocess the initial image to be segmented to obtain the target image to be segmented; wherein the preprocessing operations performed on the initial image to be segmented include image noise reduction and image size normalization.

[0047] In some specific embodiments, the local branch may specifically include: The weight map acquisition unit is used to input the feature map in parallel to the first sub-branch and the second sub-branch, and to acquire the corresponding attention weight map in the first sub-branch using the window self-attention mechanism and the feature map; The feature tensor acquisition unit is used to perform multi-scale convolution on the feature map using convolution kernels of different sizes in the second sub-branch, and to concatenate the output results of each convolution kernel to obtain the multi-scale feature tensor. The fine-grained feature acquisition submodule is used to acquire the fine-grained features based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique.

[0048] In some specific embodiments, the fine-grained feature acquisition submodule may specifically include: The fine-grained feature acquisition unit is used to determine the attention weight map as the target gating, perform element-wise operations on the multi-scale feature tensor based on the target gating to obtain the interactive features, and perform channel shuffling on the interactive features to obtain the fine-grained features.

[0049] In some specific embodiments, the global branch may specifically include: The feature map dimensionality reduction unit is used to perform dimensionality reduction processing on the feature map to obtain the corresponding dimensionality-reduced features; The feature serialization unit is used to perform feature smoothing on the dimensionality-reduced features to obtain a corresponding first processing result, and to perform serialization on the dimensionality-reduced features to obtain a corresponding second processing result. The macro-semantic feature acquisition unit is used to aggregate the first processing result and the second processing result to obtain the corresponding aggregated features, and to perform dimensionality-upgrading processing on the aggregated features to obtain the macro-semantic features.

[0050] In some specific embodiments, the image segmentation module 13 may specifically include: A semantic feature extraction unit is used to extract deep semantic features corresponding to the macro semantic features, and obtain the spatial attention map and the uncertainty score map corresponding to the macro semantic features based on the deep semantic features; A multi-scale feature acquisition unit is used to determine the target weight based on the spatial attention map and the uncertainty score map, use the target weight to weight the fine-grained features to obtain weighted fine-grained features, and concatenate the weighted fine-grained features with the macro-semantic features to obtain the target multi-scale features.

[0051] In some specific embodiments, the image segmentation module 13 may specifically include: The feature upsampling unit is used to perform stepwise bilinear interpolation upsampling on the target multi-scale features using the target decoder; wherein, in each upsampling process, the target decoder performs bilinear interpolation upsampling on the current input features, concatenates the corresponding upsampled features with the jump connection features of the corresponding stage of the encoder, and uses the concatenated features as the input features for the next upsampling process.

[0052] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0053] Figure 7 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the channel-washing-based image segmentation method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0054] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0055] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0056] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the channel-wash-based image segmentation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0057] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned channel-washing-based image segmentation method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0058] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0059] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0060] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0061] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image segmentation method based on channel shuffling, characterized in that, include: The target image to be segmented is acquired, and the target neural network is used to downsample the target image to be segmented in order to obtain the feature map corresponding to the target image to be segmented. The feature map is input in parallel to the local and global branches of the target neural network, so that the local branches can extract corresponding fine-grained features and the global branches can extract corresponding macroscopic semantic features. Specifically, the local branches obtain a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensor, the attention weight map corresponding to the feature map, and channel shuffling techniques. The local branches include a first sub-branch and a second sub-branch. The first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolutions of different sizes. Adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features. Then, upsample the target multi-scale features using the target decoder corresponding to the target neural network to obtain a target image mask with the same resolution as the target image to be segmented. Finally, perform image segmentation based on the target image mask. The target weights are determined according to the uncertainty score map corresponding to the macroscopic semantic features.

2. The image segmentation method based on channel shuffling according to claim 1, characterized in that, The acquisition of the target image to be segmented includes: An initial image to be segmented is obtained using a preset data interface, and the initial image to be segmented is preprocessed to obtain the target image to be segmented; wherein, the preprocessing operations performed on the initial image to be segmented include image noise reduction and image size normalization.

3. The image segmentation method based on channel shuffling according to claim 1, characterized in that, The process of extracting the fine-grained features from the local branches includes: The feature map is input in parallel to the first sub-branch and the second sub-branch, and the corresponding attention weight map is obtained in the first sub-branch using the window self-attention mechanism and the feature map; In the second sub-branch, the feature map is multi-scale convolution is performed using convolution kernels of different sizes, and the outputs of each convolution kernel are concatenated to obtain the multi-scale feature tensor. The fine-grained features are obtained based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique.

4. The image segmentation method based on channel shuffling according to claim 3, characterized in that, The process of obtaining the fine-grained features based on the attention weight map, the multi-scale feature tensor, and the channel shuffling technique includes: The attention weight map is determined as the target gating, and the multi-scale feature tensor is operated on element-wise based on the target gating to obtain the interactive features. The interactive features are then subjected to channel shuffling to obtain the fine-grained features.

5. The image segmentation method based on channel shuffling according to claim 1, characterized in that, The process of extracting the macroscopic semantic features from the global branch includes: The feature map is subjected to dimensionality reduction processing to obtain the corresponding dimensionality-reduced features; The dimensionality-reduced features are subjected to feature smoothing to obtain a first processing result, and the dimensionality-reduced features are subjected to serialization to obtain a second processing result. The first processing result and the second processing result are aggregated to obtain the corresponding aggregated features, and the aggregated features are then subjected to dimensionality-upgrading processing to obtain the macroscopic semantic features.

6. The image segmentation method based on channel shuffling according to claim 1, characterized in that, The adaptive fusion of the fine-grained features and the macro-semantic features based on the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features includes: Extract the deep semantic features corresponding to the macro semantic features, and obtain the spatial attention map and the uncertainty score map corresponding to the macro semantic features based on the deep semantic features; The target weight is determined based on the spatial attention map and the uncertainty score map. The fine-grained features are then weighted using the target weight to obtain weighted fine-grained features. The weighted fine-grained features are then concatenated with the macro-semantic features to obtain the target multi-scale features.

7. The image segmentation method based on channel shuffling according to claim 1, characterized in that, The upsampling of the target multi-scale features using the target decoder corresponding to the target neural network includes: The target decoder is used to perform stepwise bilinear interpolation upsampling on the multi-scale features of the target. In each upsampling process, the target decoder performs bilinear interpolation upsampling on the current input features, concatenates the corresponding upsampled features with the jump connection features of the corresponding stage of the encoder, and uses the concatenated features as the input features for the next upsampling process.

8. An image segmentation device based on channel shuffling, characterized in that, include: The image downsampling module is used to acquire the target image to be segmented and to downsample the target image to be segmented using a target neural network to obtain the feature map corresponding to the target image to be segmented. A macro-semantic feature extraction module is used to input the feature map in parallel to the local and global branches of the target neural network, so as to extract corresponding fine-grained features using the local branches and extract corresponding macro-semantic features using the global branches. Specifically, the local branches obtain a multi-scale feature tensor by performing multi-scale convolution on the feature map, and obtain the fine-grained features based on the multi-scale feature tensor, the attention weight map corresponding to the feature map, and channel shuffling technology. The local branches include a first sub-branch and a second sub-branch. The first sub-branch is used to generate the attention weight map, and the second sub-branch is used to obtain the multi-scale feature tensor by performing multi-scale convolution on the feature map using convolutions of different sizes. The image segmentation module is used to adaptively fuse the fine-grained features and the macroscopic semantic features according to the target weights corresponding to the fine-grained features to obtain corresponding target multi-scale features, and to upsample the target multi-scale features using the target decoder corresponding to the target neural network to obtain a target image mask with the same resolution as the target image to be segmented, and to perform image segmentation based on the target image mask; wherein, the target weights are weights determined according to the uncertainty score map corresponding to the macroscopic semantic features.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the channel-wash-based image segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the channel-wash-based image segmentation method as described in any one of claims 1 to 7.

Images