A neural image segmentation method based on a dual-task hybrid attention network

Abstract

The application belongs to the technical field of biomedical image processing and deep learning, and provides a neural image segmentation method based on a double-task mixed attention network. The method constructs a double-task parallel network model including a shared encoder, a centerline decoder branch and a region segmentation decoder branch. Through a cross-task attention mechanism, spatial positioning information generated by the centerline decoder branch is used to guide the region segmentation decoder branch, so as to enhance the perception and detection ability of the model to fine nerve fibers. In addition, according to the requirements of different tasks, only cross-task attention is used in the centerline decoder branch to maintain spatial positioning accuracy and prevent fine line blurring. In the region segmentation decoder branch, channel attention is superimposed to enhance semantic feature expression, which can effectively alleviate the problems of slender fiber breakage and dense region adhesion, improve the topological integrity of the segmentation result, and ensure the efficiency of processing neural image data.

Description

Technical Field

[0001] This invention belongs to the field of biomedical image processing and deep learning technology, specifically relating to a neural image segmentation method based on a dual-task hybrid attention network. Background Technology

[0002] Neuromorphological reconstruction is fundamental to the study of brain network connectivity and neural circuit function. Its core task is to accurately extract the morphology of neurons using three-dimensional microscopic imaging data. In the experimental procedure, specific neurons are typically stained using fluorescent labeling techniques, and then high-resolution three-dimensional volumetric data are acquired using a microscopic imaging system (such as the fluorescence microtomography system fMOST). After image stitching and preprocessing, the acquired raw data forms the neurovoxel signals to be analyzed. The quality of these signals directly determines the accuracy of subsequent neuron tracking and circuit modeling.

[0003] Nerve fibers exhibit extremely high structural complexity in three-dimensional space. Morphologically, nerve fibers appear as extremely long and highly branched linear structures, their diameter typically occupying only a tiny proportion of the overall imaging space, resulting in significant sparsity in volumetric data. During imaging, limitations imposed by the uniformity of fluorescent labeling, laser penetration depth, and the point spread function of the optical system often lead to physical characteristics such as uneven signal-to-noise ratios and discontinuous weak signal regions in the acquired neural images. Furthermore, when a large number of nerve fibers traverse dense brain regions (such as areas of cell body aggregation), branches from different sources will spatially intertwine and overlap. These inherent physical properties of the data mean that at the pixel level, nerve fiber signals are often intertwined with complex background noise, posing a significant challenge to automated image interpretation.

[0004] Currently, the analysis of these microscopic images still heavily relies on manual annotation or semi-automated tracking. Professionals must examine the images layer by layer in three-dimensional space, relying on visual experience to define fiber boundaries and reconstruct their topological connections. However, the data volume at the whole-brain scale is enormous, and the pathways of nerve fibers are circuitous and variable. Manual interpretation is not only time-consuming and labor-intensive, but also highly susceptible to subjective fatigue, leading to topological breaks or mis-adhesions. Although automated segmentation techniques have attempted to intervene, existing processing logic often ignores the differences between different pixels in the data. The contradiction between topological continuity and localization accuracy: Pixels in neural images can be divided into background pixels, large fiber pixels, and weak terminal pixels. The centerline, as the core reference for describing fiber topology, focuses on the precise localization of axis points, requiring the model to recognize paths with a single pixel width; while region segmentation focuses on the coverage of fiber entities. In practical data processing, if the structural relationship between the two cannot be established, the model is prone to breakage when faced with weak signals due to the lack of structural priors.

[0005] The difficulty of pixel processing is uneven: In 3D neural images, the vast majority of pixels belong to easily distinguishable background regions, with only a very small portion belonging to target signals or complex fiber overlap areas (difficult pixels). Existing segmentation methods typically use convolution operations with uniform weights to process all spatial locations, which leads to severe computational redundancy on a large number of simple background pixels. Meanwhile, in critical, elongated branches and overlapping regions, the lack of targeted feature enhancement strategies results in insufficient model perception of difficult signals.

[0006] Therefore, there is a need for an intelligent segmentation technology that can fully integrate the topological mechanisms of nerve fibers and differentiate the processing based on the difficulty of pixels, in order to replace the arduous manual image reading work, improve segmentation accuracy, and achieve efficient extraction of complex neural morphologies. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a neural image segmentation method based on a dual-task hybrid attention network. Its core innovation lies in reconciling the inherent contradictions in feature requirements between region segmentation and centerline extraction through an asymmetric hybrid attention combination design. This not only improves segmentation accuracy but also enables efficient extraction of complex neural morphologies, replacing the arduous task of manual image interpretation.

[0008] This invention provides a neural image segmentation method based on a dual-task hybrid attention network, comprising the following steps: Step 1: Construct a dual-task parallel network model. The dual-task parallel network model includes a shared encoder, a centerline decoder branch, and a region segmentation decoder branch. Both the centerline decoder branch and the region segmentation decoder branch include multiple decoding layers. Each decoding layer of the centerline decoder branch includes an upsampling module and a cross-task attention module. Each decoding layer of the region segmentation decoder branch includes an upsampling module, a cross-task attention module, and a channel attention module. Step 2: Input the neural image into the dual-task parallel network model, and downsample the input neural image through a shared encoder to extract multi-scale features of the neural image; Step 3: The centerline decoder branch and the region segmentation decoder branch respectively upsample the multi-scale features through the upsampling module in the first decoding layer to obtain the first-level centerline decoding features and the first-level region segmentation decoding features; Step 4: The centerline decoder branch and the region segmentation decoder branch interact with the first-level centerline decoding features and the first-level region segmentation decoding features through the cross-task attention module in the first-level decoding layer to obtain the first-level enhanced fusion features and the first-level centerline prediction feature map. Step 5: The channel attention module in the first decoding layer of the centerline decoder branch generates the first-level region segmentation prediction feature map based on the first-level added fusion features; Step 6: Input the first-level region segmentation prediction feature map and the first-level centerline prediction feature map into the next layer decoder, and execute steps 3 to 6 until all decoding layers have completed processing and output the final centerline prediction feature map and region segmentation prediction feature map.

[0009] Furthermore, the steps also include: A multi-task joint loss function is used to train the dual-task parallel network model.

[0010] Furthermore, step four includes: Obtain the first-level centerline decoding features and the first-level region segmentation decoding features; and concatenate the first-level centerline decoding features and the first-level region segmentation decoding features along the channel dimension to obtain the first-level concatenated features; The first-level concatenated features are processed by convolutional layers and the Sigmoid activation function to obtain the first-level spatial attention map; Using the first-level spatial attention map Spatially weight the first-level centerline decoding features to obtain the first-level centerline decoding enhancement features; The first-level centerline decoding enhancement feature is concatenated with the first-level region segmentation decoding feature to obtain the first-level enhanced concatenated feature. The first-level enhanced concatenated feature is then fused through a convolutional layer and then residual-connected to obtain the first-level fused enhancement feature. A first-level centerline prediction feature map with an approximate single-pixel width is generated based on the first-level enhanced fusion features.

[0011] Furthermore, step five includes: The first-level channel attention module of the region segmentation decoder branch performs channel recalibration on the first-level enhanced fusion features and generates the first-level region segmentation prediction feature map based on the calibration results; Furthermore, the first-level channel attention module of the region segmentation decoder branch performs channel recalibration on the first-level enhanced fusion features and generates a first-level region segmentation prediction feature map based on the calibration results, including: The channel attention module processes the first-level enhanced fusion features through global average pooling and global max pooling to aggregate spatial information and generate two channel descriptors. and ; Use two channel descriptors and The data are fed into a shared multilayer perceptron with a preset dimensionality reduction ratio, and after ReLU activation and element-wise addition, the first-level channel weights are generated by the Sigmoid function. Based on the generated first-level channel weights, the first enhanced fusion feature is recalibrated to generate the first-level recalibration result; First-level region segmentation prediction feature map is generated based on the first-level calibration results.

[0012] Furthermore, the training of the dual-task parallel network model using a multi-task joint loss function includes: Define the total loss function as: ; in, and These are weight hyperparameters; For the branch loss of the region segmentation decoder; This represents the branch loss of the centerline decoder.

[0013] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the neural image segmentation method based on a dual-task hybrid attention network described above.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: ① By using an asymmetric coupling design of the attention mechanism, the problem of "incompatibility between positioning accuracy and semantic enhancement" in multi-task collaboration is solved. This invention breaks through this limitation through an asymmetric design scheme: In the centerline branch, since its core task is to find the center point of the fiber, and this invention deliberately removes channel attention that may lead to the loss of spatial information, only retaining cross-task attention of the spatial dimension, thereby locking the original spatial resolution; while in the region segmentation branch, channel attention is superimposed to enhance the ability to distinguish complex backgrounds. This precise matching design of "each taking what it needs" ensures that while the centerline branch obtains topological guidance, its positioning accuracy will not be disturbed by semantic enhancement operations, achieving the technical effect of "being able to see clearly and be accurately positioned".

[0015] ② By cascading spatial priors and semantic weights, the robustness of segmentation in complex contexts is improved. In the segmentation branch, the spatial priors provided by cross-task attention act as "noise reduction," while the subsequent channel attention acts as "signal enhancement." This combined design enables the model to first lock onto structural regions and then enhance feature responses when faced with high-noise, low-contrast microscopic images, thereby reducing the false negative rate of slender branches from a mechanistic perspective.

[0016] ③ This design reduces redundancy in the model design and improves computational efficiency. This targeted combination design avoids blindly stacking modules across all branches. Compared to two independent models, the number of parameters in this invention is reduced by approximately 40%, and the simplification of the central branch structure improves the response speed during the inference phase, making it suitable for automated processing requirements. Attached Figure Description

[0017] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0018] Figure 1 A schematic diagram of a dual-task segmentation network structure based on differential attention configuration provided in an embodiment of the present invention; Figure 2 The original images of nerve fibers provided in the embodiments of the present invention; Figure 3 This is a binary image of nerve fiber segmentation provided in an embodiment of the present invention; Figure 4 This is a diagram showing the predicted results of the nerve fiber centerline provided in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] Example 1 This embodiment provides a neural image segmentation method (TSH-Net) based on a dual-task hybrid attention network. Its core mechanism lies in coordinating the differences in feature representation between the region segmentation (SEG) and centerline extraction (CTR) tasks through an asymmetric decoder structure, aiming to improve the segmentation accuracy of slender structures in complex neural images.

[0024] Combined with appendix Figure 1 To be continued Figure 4 The specific implementation steps of the neural image segmentation method based on a dual-task hybrid attention network provided in this embodiment are as follows: S1: Construct a dual-task parallel network model; The dual-task parallel network structure adopts a three-dimensional encoder-decoder parallel paradigm, and the dual-task parallel network structure includes a shared encoder, a centerline decoder branch, and a region segmentation decoder branch. The shared encoder is used for multi-scale feature extraction of the input neural image; The centerline decoder branch is used to generate a predicted feature map of the nerve fiber centerline. The region segmentation decoder branch is used to generate a neural fiber region segmentation prediction feature map. The centerline decoder branch and the region segmentation decoder branch are set up in parallel, and both the centerline decoder branch and the region segmentation decoder branch include m decoding layers, where m is an integer greater than or equal to 3, and the two branches exchange information between their corresponding decoding layers through a cross-task attention module.

[0025] Each decoding layer in the centerline decoder branch includes an upsampling module and a cross-task attention module; each decoding layer in the region segmentation decoder branch includes an upsampling module, a cross-task attention module, and a channel attention module.

[0026] For ease of explanation and understanding, the upsampling module in the nth decoding layer is represented as the nth level upsampling module; the cross-task attention module in the nth decoding layer is represented as the nth level cross-task attention module; and the channel attention module in the nth decoding layer is represented as the nth level channel attention module, where n∈[2,3,……,m].

[0027] S2: Input the neural image into the constructed dual-task parallel network model, and downsample the input neural image through a shared encoder to extract multi-scale features of the neural image.

[0028] The shared encoder includes multi-layer (e.g., three-layer) downsampling modules. Each downsampling module includes two sets of residual convolutional structures. Each set of residual convolutional results includes a 3×3×3 convolution, BN batch normalization, ReLU activation function, and a Dropout operation. Each downsampling module achieves downsampling through convolutions with a stride greater than 1. The initial number of channels in the shared encoder is 64, and the number of channels is doubled layer by layer until the number of channels reaches 512. This allows the shared encoder to downsample the input neural image layer by layer through the multi-layer downsampling modules, forming a multi-scale feature pyramid, thereby obtaining the multi-scale features of the neural image.

[0029] For example, neural images can be in the form of a size of After being processed layer by layer by a shared encoder, the voxels yield semantic features with 512 channels.

[0030] It should be noted that during the downsampling process, the spatial size of the neural image (e.g., D×H×W) gradually decreases, but the extracted semantic information of the features becomes richer and richer. These features at different levels and resolutions together constitute a multi-scale feature pyramid.

[0031] S3: The first-level decoding layer of the centerline decoder branch and the region (SEG) segmentation decoder branch generates the first-level centerline prediction feature map and the first-level region segmentation prediction feature map based on the multi-scale features obtained by the shared encoder. Specifically, the following steps are included: S3-1: The centerline decoder branch and the centerline decoder branch acquire multi-scale features obtained by the shared encoder through the first-level upsampling module in parallel and process them to obtain the first-level centerline decoding features and the first-level region segmentation decoding branch features. Specifically, the first-level upsampling module of the region segmentation decoder branch acquires and processes the multi-scale features obtained by the shared encoder to obtain the first-level region segmentation decoding features, wherein the first-level region segmentation decoding features are represented as follows: , ; The first-level upsampling module of the centerline decoder branch acquires and processes multi-scale features obtained from the shared encoder to obtain the first-level centerline (CTR) decoding features, where the first-level centerline decoding features are represented as follows: , .

[0032] S3-2: The center decoder branch and the region segmentation decoder branch interact with the first-level centerline decoding features and the first-level region segmentation decoding features through the first-level cross-task attention module to obtain the first-level fused features, and generate the first-level centerline prediction feature map based on the first-level fused features; The central decoder branch and the region segmentation decoder branch execute the following steps through the first-level cross-task attention module: S3-2-1: Obtain the first-level centerline decoding features and the first-level region segmentation decoding features; and concatenate the first-level centerline decoding features and the first-level region segmentation decoding features along the channel dimension to obtain the first-level concatenated features; S3-2-2: Through A convolutional layer (which internally compresses the number of channels in the concatenated feature layer to 1 / 4 of the original number of channels) and a sigmoid activation function process the first-level concatenated features to obtain the first-level spatial attention map; its calculation formula is as follows: ; in, represents the Sigmoid activation function, and [·] represents the feature concatenation operation; This is a first-level spatial attention map, and .

[0033] This spatial attention map This reflects the importance of each spatial location to the centerline mission.

[0034] S3-2-3: Utilizing the first-level spatial attention map Spatially weighting the first-level centerline decoding features yields the first-level centerline decoding enhancement features, as shown in the following formula: ; Where ⊙ represents element-wise multiplication; This indicates the first-level centerline decoding enhancement feature.

[0035] S3-2-4: The first-level centerline decoding enhancement feature is concatenated with the first-level region segmentation decoding feature to obtain the first-level enhanced concatenated feature. The first-level enhanced concatenated feature is then processed... After convolutional layer fusion, residual connections are then performed to obtain the first-level fusion enhancement features, as shown in the following formula: ; in, This is a first-level fusion enhancement feature; S3-2-5: Generate a first-level centerline prediction feature map with an approximate single-pixel width based on the first-level enhanced fusion features.

[0036] This embodiment utilizes the CTA module to implement a topology-guided region segmentation (SEG) decoder branch that focuses on slender fiber regions, enhancing the network's ability to capture weak signals. Furthermore, it facilitates the use of spatial attention to strengthen centerline features, thereby improving the response to the center location of neural fibers. The centerline decoder branch employs only cross-task attention mechanisms to maintain spatial resolution and avoid the decrease in localization accuracy caused by global pooling.

[0037] S3-3: The first-level channel attention module of the region segmentation decoder branch performs channel recalibration on the first-level enhanced fusion features and generates the first-level region segmentation prediction feature map based on the calibration results; In the region segmentation decoder branch, each level of cross-task attention module is followed by a channel attention module (CA) designed in the SE-Net style. The channel attention module further enhances the fused features output by the cross-task attention module. Specifically, the channel attention module performs the following operations: S3-3-1: The channel attention module processes the first-level enhanced fusion features through global average pooling (GAP) and global max pooling (GMP) to aggregate spatial information and generate two channel descriptors. and ; S3-3-2: Connect the two channel descriptors and Each is fed into a preset dimensionality reduction ratio (e.g., dimensionality reduction ratio). In the shared multilayer perceptron (MLP), after ReLU activation and element-wise addition, the first-level channel weights are generated using the Sigmoid function, as shown in the following formula: ; in, and These are the weight parameters for the MLP; For the Sigmoid function; First-level channel weights.

[0038] S3-3-3: Based on the generated first-level channel weights, the first enhanced fusion feature is recalibrated to generate the first-level recalibration result. The calculation formula is as follows: ; in, First-level recalibration results.

[0039] S3-3-4: Generate the first-level region segmentation prediction feature map based on the first-level calibration results.

[0040] S4: The nth layer of the centerline decoder branch and the nth layer of the centerline decoder branch generate the nth-level centerline prediction feature map and the nth-level region segmentation prediction feature map based on the n-1th level centerline prediction feature map and the n-1th level region segmentation prediction feature map output by the n-1th layer of the decoder branch, respectively. Since both the centerline decoder branch and the region segmentation decoder branch include m-level decoding layers, the following operation is performed for the n-th level decoding layer (where n∈[2,3,...,m]): S4-1: Both the centerline decoder branch and the centerline decoder branch acquire and process the (n-1)th level centerline prediction feature map and the (n-1)th level region segmentation prediction feature map output by the (n-1)th level decoding layer through the nth level upsampling module to obtain the nth level centerline decoding feature and the nth level region segmentation decoding feature. Specifically, the nth-level upsampling module of the region segmentation decoder branch acquires and processes the (n-1)th-level region segmentation prediction feature map to obtain the nth-level region segmentation decoding feature, where the nth-level region segmentation decoding feature is represented as... , ; The nth-level upsampling module of the centerline decoder acquires and processes the (n-1)th-level centerline prediction feature map acquired by the shared encoder to obtain the nth-level centerline decoding feature, where the nth-level centerline decoding feature is represented as... , ; S4-2: The center decoder branch and the region segmentation decoder branch interact with the nth centerline decoding features and the nth region segmentation decoding features through the nth level cross-task attention module to obtain the nth level fusion enhancement features, and generate the nth level centerline prediction feature map based on the nth level fusion features; Specifically, it includes the following steps: S4-2-1: The nth level cross-task attention module obtains the nth centerline decoding feature and the nth level region segmentation decoding feature; and concatenates the nth centerline decoding feature and the nth level region segmentation decoding feature along the channel dimension to obtain the nth level concatenated feature; S4-2-2: Through A convolutional layer (which internally compresses the number of channels in the concatenated feature layer to 1 / 4 of the original number of channels) and a sigmoid activation function process the concatenated features at the nth level to obtain the nth-level generative spatial attention map; its calculation formula is as follows: ; in, represents the Sigmoid activation function, and [·] represents the feature concatenation operation; This is a first-level spatial attention map, and .

[0041] This spatial attention map This reflects the importance of each spatial location to the centerline mission.

[0042] S4-2-3: Utilizing the nth level spatial attention map Spatially weighting the nth-level centerline decoding features yields the nth-level centerline decoding enhancement features, as shown in the following formula: ; Where ⊙ represents element-wise multiplication; This represents the enhanced feature of the centerline decoding at level n.

[0043] S4-2-4: The nth-level centerline decoding enhancement feature is concatenated with the nth-level region segmentation decoding feature to obtain the nth-level enhanced concatenated feature. The nth-level enhanced concatenated feature is then processed... After convolutional layer fusion, residual connections are then performed to obtain the nth level fused feature, as shown in the following formula: ; in, This represents the nth level of fusion feature; S4-2-5: Generate an nth-level centerline prediction feature map with an approximate single-pixel width based on the nth-level enhanced fusion feature.

[0044] S4-3: The nth-level channel attention module of the region segmentation decoder branch performs channel recalibration on the nth-level enhanced fusion features and generates the nth-level region segmentation prediction feature map based on the calibration results; S4-3-1: The channel attention module processes the nth-level enhanced fusion feature using Global Average Pooling (GAP) and Global Max Pooling (GMP) to aggregate spatial information and generate two channel descriptors. and ; S4-3-2: Connect the two channel descriptors and Each is fed into a preset dimensionality reduction ratio (e.g., dimensionality reduction ratio). In the shared multilayer perceptron (MLP), after ReLU activation and element-wise addition, the nth-level channel weights are generated using the Sigmoid function, as shown in the following formula: ; in, and These are the weight parameters for the MLP; For the Sigmoid function; This represents the channel weight for the nth level.

[0045] S4-3-3: Based on the generated nth-level channel weights, the nth-level enhanced fusion feature is recalibrated to generate the nth-level recalibration result. The calculation formula is as follows: ; First-level recalibration results.

[0046] S4-3-4: Generate the nth level region segmentation prediction feature map based on the nth level impulse calibration results.

[0047] It should be noted that the output of the last level, i.e., the m-th level, is used as the final segmentation prediction feature map.

[0048] In this embodiment, the dual-task parallel network model gradually restores spatial resolution through the parallel decoding of the centerline decoding branch and the region segmentation decoding branch. Both the centerline decoding branch and the region segmentation decoding branch include multi-level (e.g., three-level) cascaded upsampling modules, and each upsampling module uses transposed convolution to gradually restore spatial resolution.

[0049] This process enhances the response of important channels and suppresses irrelevant background features, thereby improving the semantic discriminative ability of region segmentation. As the decoding process proceeds step by step, the centerline decoder branch outputs a probability map with a single pixel width, while the region segmentation decoder branch outputs a segmentation probability map of the neural fiber region.

[0050] In this embodiment, this differentiated configuration allows the CTR branch to retain only the CTA module to maintain high spatial resolution and avoid localization ambiguity caused by spatial pooling operations; while the SEG branch enhances the response to neuronal semantic features through cascaded CA modules and improves the discriminative ability of region segmentation in complex backgrounds by suppressing background-irrelevant channels. Furthermore, utilizing the differentiated combination design described in this embodiment, setting a reasonable threshold (here, 0.6) allows for the determination of the region segmentation ability from the input neural image (…). Figure 2 Simultaneously obtain highly connected segmentation results in ) Figure 3 ) and high-precision centerline skeleton ( Figure 4 Furthermore, it effectively alleviates the optimization conflict between the easy breakage and easy adhesion of slender branches in dense areas, providing a robust data foundation for neuromorphic analysis.

[0051] S5: The dual-task parallel network model is trained using a multi-task joint loss function; To drive the effective learning of the dual-task structure of the dual-task parallel network model (TSH-Net) and ensure that segmentation accuracy and topological integrity mutually reinforce each other during training, this embodiment uses a multi-task joint loss function for calculation. This strategy aims to force the network to maximize the preservation of the global topological structure while optimizing pixel-level classification accuracy through explicit mathematical constraints, thereby solving the common problems of "signal breakage" and "loss of weak structures" in traditional single-task networks, as detailed below: The model is optimized using a multi-task joint loss function, and its total loss function is defined as follows: ; in, and These are weighted hyperparameters that, when set, can guide the network to prioritize optimizing its topology. For the branch loss of the region segmentation decoder; This is the branch loss of the centerline decoder.

[0052] The branch loss of the region segmentation decoder consists of a weighted binary cross-entropy loss and an overlap loss: ; in and This is the balance coefficient; The supervision target of the branch loss of the region segmentation decoder is the neuron voxel segmentation label. By adopting a linear combination of weighted binary cross-entropy loss and Dice loss, it helps to solve the class imbalance problem while taking into account the integrity of the segmentation region.

[0053] The weighted binary cross-entropy loss is defined as: ; Where N is the total number of voxels. For the true label of voxel i, For model prediction probabilities; positive sample weights Set to the ratio of background voxels to foreground voxels to enhance the penalty for sparse foreground.

[0054] Overlapping loss : ; in To smooth out terms and prevent the denominator from being zero.

[0055] The supervision target of the topology branch loss is a single-pixel-wide skeleton label, which is mainly used to enhance the connectivity of the center line.

[0056] Centerline decoder branch loss Branch loss of region segmentation decoder Similarly, it is also a linear combination of weighted binary cross-entropy loss and Dice loss, but the calculation uses a single-pixel-wide skeleton label as the supervision signal. Specifically, ; in and Designed based on data characteristics, and The mathematical form is the same as that of the region segmentation decoder branch, but the skeleton label is used as the supervision signal during the calculation.

[0057] Through this hybrid loss, the network is able to simultaneously locate the skeleton voxels and maintain their topological continuity.

[0058] Example 2: Embodiment 2 of the present invention provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, it implements the steps in the neural image segmentation method based on a dual-task hybrid attention network as described in Embodiment 1 of the present invention.

[0059] The detailed steps are the same as those of the neural image segmentation method based on dual-task hybrid attention network provided in Example 1, and will not be repeated here.

[0060] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0065] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0066] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A neural image segmentation method based on a dual-task hybrid attention network, characterized in that, Includes the following steps: Step 1: Construct a dual-task parallel network model. The dual-task parallel network model includes a shared encoder, a centerline decoder branch, and a region segmentation decoder branch. Both the centerline decoder branch and the region segmentation decoder branch include multiple decoding layers. Each decoding layer of the centerline decoder branch includes an upsampling module and a cross-task attention module. Each decoding layer of the region segmentation decoder branch includes an upsampling module, a cross-task attention module, and a channel attention module. Step 2: Input the neural image into the dual-task parallel network model, and downsample the input neural image through a shared encoder to extract multi-scale features of the neural image; Step 3: The centerline decoder branch and the region segmentation decoder branch respectively upsample the multi-scale features through the upsampling module in the first decoding layer to obtain the first-level centerline decoding features and the first-level region segmentation decoding features; Step 4: The centerline decoder branch and the region segmentation decoder branch interact with the first-level centerline decoding features and the first-level region segmentation decoding features through the cross-task attention module in the first-level decoding layer to obtain the first-level enhanced fusion features and the first-level centerline prediction feature map. Step 5: The channel attention module in the first decoding layer of the centerline decoder branch generates the first-level region segmentation prediction feature map based on the first-level added fusion features; Step 6: Input the first-level region segmentation prediction feature map and the first-level centerline prediction feature map into the next layer decoder, and execute steps 3 to 6 until all decoding layers have completed processing and output the final centerline prediction feature map and region segmentation prediction feature map.

2. The neural image segmentation method based on a dual-task hybrid attention network according to claim 1, characterized in that, The steps also include: A multi-task joint loss function is used to train the dual-task parallel network model.

3. The neural image segmentation method based on a dual-task hybrid attention network according to claim 1, characterized in that, Step four includes: Obtain the first-level centerline decoding features and the first-level region segmentation decoding features; and concatenate the first-level centerline decoding features and the first-level region segmentation decoding features along the channel dimension to obtain the first-level concatenated features; The first-level concatenated features are processed by convolutional layers and the Sigmoid activation function to obtain the first-level spatial attention map; Using the first-level spatial attention map Spatially weight the first-level centerline decoding features to obtain the first-level centerline decoding enhancement features; The first-level centerline decoding enhancement feature is concatenated with the first-level region segmentation decoding feature to obtain the first-level enhanced concatenated feature. The first-level enhanced concatenated feature is then fused through a convolutional layer and then residual-connected to obtain the first-level fused enhancement feature. A first-level centerline prediction feature map with an approximate single-pixel width is generated based on the first-level enhanced fusion features.

4. The neural image segmentation method based on a dual-task hybrid attention network according to claim 3, characterized in that, Step five includes: The first-level channel attention module of the region segmentation decoder branch performs channel recalibration on the first-level enhanced fusion features and generates the first-level region segmentation prediction feature map based on the calibration results.

5. The neural image segmentation method based on a dual-task hybrid attention network according to claim 4, characterized in that, The first-level channel attention module of the region segmentation decoder branch performs channel recalibration on the first-level enhanced fusion features and generates a first-level region segmentation prediction feature map based on the calibration results, including: The channel attention module processes the first-level enhanced fusion features through global average pooling and global max pooling to aggregate spatial information and generate two channel descriptors. and ; Use two channel descriptors and The data are fed into a shared multilayer perceptron with a preset dimensionality reduction ratio, and after ReLU activation and element-wise addition, the first-level channel weights are generated by the Sigmoid function. Based on the generated first-level channel weights, the first enhanced fusion feature is recalibrated to generate the first-level recalibration result; First-level region segmentation prediction feature map is generated based on the first-level calibration results.

6. The neural image segmentation method based on a dual-task hybrid attention network according to claim 2, characterized in that, The training of the dual-task parallel network model using a multi-task joint loss function includes: Define the total loss function as: ； in, and These are weight hyperparameters; For the branch loss of the region segmentation decoder; This represents the branch loss of the centerline decoder.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the neural image segmentation method based on a dual-task hybrid attention network as described in any one of claims 1-6.

Images