Collaborative Optimization Method and Device for Three-Dimensional Tissue Segmentation and Registration of Brain Neuroimaging

Abstract

This invention discloses a collaborative optimization method and apparatus for three-dimensional tissue segmentation and registration of brain neural images. The method includes the following steps: Step S01. Constructing a segmentation and registration collaborative model, including a shared feature encoder, a segmentation path, and a registration path. The segmentation path is used to generate a segmentation probability distribution map, and the registration path is used to generate a deformation field. Step S02. Obtaining a training set of three-dimensional magnetic resonance imaging (MRI) images of the brain. Step S03. Collaboratively training the segmentation and registration collaborative model according to a multi-task collaborative loss function. The loss function includes segmentation loss, registration loss, and a collaborative regularization term. The collaborative regularization term modulates the deformation field gradient penalty term by using a multi-scale boundary weight map and tissue-specific mechanical weights. Step S04. Receiving the image pairs to be registered in real time and inputting them into the trained segmentation and registration collaborative model to obtain the registration result. This invention can significantly improve computational efficiency while ensuring segmentation and registration accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a collaborative optimization method and apparatus for three-dimensional tissue segmentation and registration of brain neural images. Background Technology

[0002] Three-dimensional magnetic resonance imaging (MRI) is a core tool for the precise diagnosis and treatment of brain diseases. Its analytical accuracy highly depends on the automatic segmentation of brain tissue and the accurate registration of multiple images. Current technologies typically treat image segmentation and registration as two separate tasks, implemented using different deep learning models. For example, 3D U-Net is used for image segmentation, and VoxelMorph (voxel deformation model) is used for image registration. However, applying this approach to three-dimensional tissue segmentation and image registration of brain neural images presents the following problems:

[0003] 1. Low execution efficiency and accuracy

[0004] Existing technologies treat segmentation and registration as two independent tasks, ignoring the inherent connection between them. This leads to problems because segmentation and registration are actually coupled. Accurate segmentation provides semantic guidance for registration, while registration prioritizes the alignment of similar structures across images, avoiding noise interference. High-precision registration also provides spatial consistency constraints for segmentation. Performing image segmentation and registration independently fails to fully utilize this inherent connection, resulting in low execution efficiency and accuracy, making it difficult to balance the accuracy and efficiency of segmentation and registration.

[0005] 2. The model is complex and costly.

[0006] Because it requires two independent models for segmentation and registration, the number of model parameters is enormous, resulting in high computational resource consumption. This increases the complexity and maintenance costs of the system, making it difficult to deploy in real-time scenarios or on embedded devices with limited computing power. Existing lightweight techniques, such as network pruning and knowledge distillation, often sacrifice model accuracy, especially the ability to process small structures such as the hippocampus and amygdala, thus still failing to balance the accuracy and efficiency of segmentation and registration. Summary of the Invention

[0007] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems existing in the prior art, the present invention provides a collaborative optimization method and device for three-dimensional tissue segmentation and registration of brain neural images, which can realize the collaborative optimization of three-dimensional tissue segmentation and registration of brain neural images, and significantly improve the computational efficiency while ensuring the accuracy of segmentation and registration.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0009] A collaborative optimization method for three-dimensional tissue segmentation and registration in brain neural images, comprising the following steps:

[0010] Step S01. Construct a segmentation and registration collaborative model. The segmentation and registration collaborative model includes a shared feature encoder, a segmentation path, and a registration path. The shared feature encoder is used to receive the image pairs to be registered, extract features, and output a multi-scale feature map set. The segmentation path is used to segment the highest-level semantic feature map in the multi-scale feature map set to generate a segmentation probability distribution map of each voxel belonging to different brain tissues. The registration path is used to register two or more high-level feature maps in a multi-scale feature map set to generate a differential homeomorphic deformation field.

[0011] Step S02. Obtain a training set consisting of different pairs of three-dimensional magnetic resonance images of the brain, wherein the pairs of three-dimensional magnetic resonance images of the brain include floating images as moving targets and fixed images as alignment references;

[0012] Step S03. Use the acquired training set to perform collaborative training on the segmentation and registration collaborative model according to the multi-task collaborative loss function. The multi-task collaborative loss function includes segmentation loss, registration loss, and collaborative regularization term. The collaborative regularization term uses a multi-scale boundary weight map. and tissue-specific mechanical weights Commonly modulated deformation field gradient penalty term , Represents the deformation field At voxel location Spatial gradient at a given location, the multi-scale boundary weight map Using segmentation probability distribution plots The tissue-specific mechanical weights were calculated. A weighted graph determined based on prior biomechanical information of brain tissue;

[0013] Step S04. Receive the three-dimensional magnetic resonance images of the brain to be registered in real time, input them into the trained segmentation and registration collaborative model, output the final three-dimensional brain tissue segmentation label map from the segmentation path in the model, and output the differential homeomorphic deformation field from the registration path, and obtain the registration result aligned with the fixed image space by resampling the floating image.

[0014] Furthermore, the segmentation path employs an improved 3D U-Net (three-dimensional U-shaped network) architecture for segmentation processing. This improved 3D U-Net architecture is based on the 3D U-Net framework, embedding a multimodal channel attention module after the encoder bottleneck layer to calculate multimodal statistical features. These multimodal statistical features are then fused and input into the MLP for nonlinear transformation. Multi-scale spatial attention modules are embedded at each skip connection in the decoder to apply multi-scale deformable convolutions to the input feature maps and embed multi-map prior information to guide the optimization of boundary details. Finally, the Softmax function outputs the probability distribution map of each voxel belonging to different brain tissues. .

[0015] Furthermore, in the multimodal channel attention module, during the fusion of statistical features from each channel, the contribution of each statistical feature is adjusted using a weight matrix. Channel statistical features include mean, extreme value, variance, and skewness features. The bottleneck dimension of the MLP is determined based on the number of input channels and the network layer depth according to the following formula:

[0016]

[0017] in, Indicates the bottleneck dimension. For network layer depth, Where C is the base compression factor and C is the number of input channels. This indicates the number of channel statistical features.

[0018] Furthermore, the processing flow of the multi-scale spatial attention module includes:

[0019] The input feature map is convolved using multiple deformable convolutional branches of different scales, and the branches are fused using a gating mechanism.

[0020]

[0021] in, The number of convolution branches. The result of branch feature fusion. For gating weights, , The weight matrix is ​​a learnable matrix. For feature maps Perform global average pooling. For the i-th deformable convolution operation, These are the spatial saliency descriptors extracted from the input feature map through global average pooling, global max pooling, global variance pooling, and global entropy pooling, respectively.

[0022] Based on the branch feature fusion results Atlas fusion is performed using an anatomical prior embedding approach:

[0023]

[0024] in, For the results of map fusion, For multi-map priors, K represents the number of brain regions, and D, H, and W represent the depth, height, and width of the feature maps, respectively. For graph attention weights, These are the prior intensity coefficients; based on the spectrum fusion results The final output of the multi-scale spatial attention module is obtained by using gated residual connections:

[0025]

[0026] in, To integrate weights, The weight matrix is ​​a learnable matrix. To perform global average pooling on the input feature map X. For the input feature map, This represents the Sigmoid activation function.

[0027] Furthermore, the improved 3D U-Net architecture uses dynamically grouped convolutions in its decoder, where the grouping strategy is determined based on the greatest common divisor of the input and output channel numbers.

[0028]

[0029]

[0030] in, The average cosine similarity between input channels. For the input feature map, The number of groups is dynamically determined. Number of input and output channels. This indicates finding the greatest common divisor. The preset maximum similarity normalization factor, The i-th and j-th channel features are input to feature map X;

[0031] Grouped convolution operations use channel recalibration:

[0032]

[0033] in, This represents the result of grouped convolution. This is the scaling factor for each set of features. This indicates a grouped convolution operation;

[0034] Cross-group information exchange is performed after grouped convolution:

[0035]

[0036] in, Indicates the results of cross-group information exchange;

[0037] A gradual strategy is used for channel compression:

[0038]

[0039] in, This indicates the first 1×1×1 convolution compression operation. This indicates the second 1×1×1 convolution compression operation. This represents intermediate features before compression. Indicates the intermediate features after compression;

[0040] Finally, the target number of channels is restored through deconvolution.

[0041] Furthermore, the registration path adopts a two-level VoxelMorph architecture. The decoder gradually restores the spatial resolution through transposed convolution, and the final output uses Tanh-activated 3D convolution to generate a differential homeomorphic variable field.

[0042] Furthermore, the calculation expression for the multi-task collaborative loss function is as follows: ,in, , , These are the weighting coefficients and the segmentation loss, respectively. The results are calculated using multi-scale Dice loss and weighted cross-entropy loss. Multi-scale Dice loss is obtained by combining Dice losses calculated at different spatial scales. Weighted cross-entropy loss is a boundary-weighted cross-entropy loss. Structural consistency loss describes the difference in topological features between the predicted results and the true labels. Registration loss... The results are calculated using multi-resolution global cross-correlation loss, anisotropic smoothing constraint, Jacobian determinant regularization term, and differential homeomorphism constraint. The global cross-correlation loss describes the similarity between images; the anisotropic smoothing constraint is a diffusion constraint that integrates differences in mechanical properties in different directions; and the differential homeomorphism constraint is an exponential mapping of the velocity field. The cooperative regularization term... The calculation expression is:

[0043] ,

[0044] Where p represents the position coordinates of a voxel in three-dimensional space. Indicated by A local window centered on the center. The tissue-specific mechanical constraints are calculated by integrating segmentation masks of different brain tissues.

[0045] Furthermore, before inputting the image pair to be registered into the segmentation and registration co-processing model, the process includes preprocessing the floating and fixed images in the image pair separately. The preprocessed floating and fixed images are then stitched together and input into the segmentation and registration co-processing model. The preprocessing includes using a segmentation network to segment the input image and generate a brain tissue probability map. Based on the generated brain tissue probability map Obtaining a binary mask of brain tissue Using a binary mask of brain tissue Brain tissue regions are extracted, and the extracted brain tissue regions are standardized before outputting the standardized results. The standardization process includes estimating the bias map. Intensity correction results were obtained by performing intensity correction on the extracted brain tissue regions. Calculate tissue-specific local mean According to the strength correction results Tissue-specific local mean Calculate the standardized results.

[0046] Furthermore, based on the generated brain tissue probability map A binary mask of brain tissue was generated using adaptive thresholding. The adaptive threshold is based on the probability map of brain tissue. The mean and standard deviation are calculated from this.

[0047] The formula for calculating the standardized result is:

[0048]

[0049] in, This represents the standardized result after standardizing the extracted brain tissue regions. The global intensity standard deviation for the entire brain tissue region. To prevent division by zero of extremely small constants, The local mean value specific to the tissue is expressed by the following formula:

[0050]

[0051] in, For A local window centered on the center. This is a probability weighting graph for the organization.

[0052] It also includes spatial normalization achieved using differential homeomorphic registration, the calculation expression of which is:

[0053]

[0054] in, This indicates that it includes a similarity measure and a differential homeomorphism regularization term. Standard brain atlas image, This represents the deformation field from the input image space to the standard atlas space. The template deformation field after spatial standardization;

[0055] A computer device includes a processor and a memory, the memory being used to store a computer program and the processor being used to execute the computer program to perform the method described above.

[0056] Compared with the prior art, the advantages of the present invention are as follows:

[0057] 1. This invention constructs a dual-path collaborative segmentation and registration model. It utilizes a shared feature encoder to extract multi-scale features from the image pairs to be registered. The highest-level feature map is segmented using a segmentation path to output a probability distribution, while the higher-level feature map is registered using a registration path to generate a deformation field. After collaborative optimization training of the segmentation and registration model, the optimized model can achieve collaborative three-dimensional tissue segmentation and registration of brain neural images. It can fully explore the intrinsic relationship between segmentation and registration tasks. Through shared features and joint learning mechanisms, it can significantly improve computational efficiency while ensuring accuracy, achieving an organic unity of accuracy and efficiency.

[0058] 2. The segmentation and registration collaborative model in this invention, by adopting a shared encoder and dual-path design, can also avoid the waste of resources in building models independently for two tasks, achieve high reuse of feature calculation, reduce the complexity of the model and the implementation cost, and can be easily deployed on edge computing devices or clinical workstations, etc.

[0059] 3. In the model training process, this invention achieves optimization through a multi-task collaborative loss function. The collaborative regularization term uses the segmentation probability map to generate boundary-aware weights and combines tissue-specific mechanical constraints to modulate the smoothness of the deformation field. This allows reasonable degrees of freedom of deformation within the tissue during the registration process, while applying stronger smoothness to maintain structural integrity when crossing anatomical boundaries. This information feedback mechanism allows the segmentation task to benefit from more accurate spatial alignment, while the registration task can obtain anatomical priors. Compared with the traditional approach of treating segmentation and registration as isolated tasks, this invention can fully explore the anatomical relationship between segmentation and registration, and promote deep interaction between segmentation and registration. Attached Figure Description

[0060] Figure 1 This is a schematic diagram illustrating the implementation process of the collaborative optimization method for three-dimensional tissue segmentation and registration of brain neural images in this embodiment.

[0061] Figure 2 This is a detailed flowchart illustrating the collaborative optimization of three-dimensional tissue segmentation and registration in brain neural images in this embodiment. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0063] like Figure 1 As shown, the steps of the collaborative optimization method for three-dimensional tissue segmentation and registration of brain neural images in this embodiment include:

[0064] Step S01. Construct a segmentation registration collaborative model, which includes a shared feature encoder. Segmentation path and registration path, shared feature encoder This is used to receive image pairs to be registered, extract features, and output a multi-scale feature map set. The segmentation path is used to segment the highest-level semantic feature map in the multi-scale feature map set, generating a segmentation probability distribution map of each voxel belonging to different brain tissues. The registration path is used to register two or more high-level feature maps in a multi-scale feature map set to generate a deformation field.

[0065] Step S02. Obtain a training set consisting of different pairs of three-dimensional magnetic resonance images of the brain, wherein the pairs of three-dimensional magnetic resonance images of the brain include floating images as moving targets and fixed images as alignment references;

[0066] Step S03. Use the acquired training set to perform collaborative training on the segmentation and registration collaborative model according to the multi-task collaborative loss function. The multi-task collaborative loss function includes segmentation loss, registration loss, and collaborative regularization term. The collaborative regularization term uses a multi-scale boundary weight graph. and tissue-specific mechanical weights Commonly modulated deformation field gradient penalty term Where p represents the position coordinates of a voxel in three-dimensional space. Represents the deformation field At voxel location Spatial gradient at that location Represents spatial gradient The square of the L2 norm is used to measure the deformation field. The magnitude of local changes; multi-scale boundary weighting diagram Using segmentation probability distribution plots The tissue-specific mechanical weights were calculated. A weighted map determined based on biomechanical prior information (such as relative stiffness differences) of brain tissues (such as white matter, gray matter, cerebrospinal fluid, etc.) to apply differentiated deformation constraints to different tissue regions.

[0067] Step S04. Receive the three-dimensional magnetic resonance images of the brain to be registered in real time, input them into the trained segmentation and registration collaborative model, output the final three-dimensional brain tissue segmentation label map from the segmentation path in the model, and output the deformation field from the registration path, and obtain the registration result aligned with the fixed image space by resampling the floating image.

[0068] This embodiment uses a shared feature encoder. The segmentation path and registration path are used to construct a dual-path collaborative segmentation and registration model, which utilizes a shared feature encoder. Multi-scale feature extraction is performed on the image pairs to be registered. The highest-level feature map is segmented using a segmentation path to output a probability distribution, and the higher-level feature map is registered using a registration path to generate a deformation field. After co-optimizing and training the segmentation and registration co-model, the optimized segmentation and registration co-model can achieve the synergy of three-dimensional tissue segmentation and registration of brain neural images. It fully explores the intrinsic relationship between segmentation and registration tasks, so that accurate segmentation results can provide strong semantic constraints for registration and improve the anatomical rationality of the deformation field. High-precision registration results can provide spatial alignment training data for the segmentation model and enhance its robustness. Through feature sharing and joint learning mechanisms, computational efficiency can be significantly improved while ensuring accuracy, achieving an organic unity of accuracy and efficiency.

[0069] Meanwhile, the segmentation and registration collaborative model, by adopting a shared encoder and dual-path design, can also avoid the waste of resources in building models independently for two tasks, achieve high reuse of feature computation, reduce model complexity and implementation cost, and facilitate deployment on edge computing devices or clinical workstations.

[0070] On the other hand, this embodiment optimizes the model training process through a multi-task collaborative loss function. The collaborative regularization term uses the segmentation probability map to generate boundary-aware weights and combines tissue-specific mechanical constraints to modulate the smoothness of the deformation field. First, it uses the segmentation probability distribution map generated by the segmentation path. Calculate the multi-scale boundary weight map It also incorporates weights corresponding to tissue-specific mechanical prior information (such as the differential stiffness of white matter, gray matter, and cerebrospinal fluid). Generate a composite weighted graph of boundary sensing and biomechanical constraints. This, in turn, forms a gradient penalty term for the deformation field by jointly and dynamically modulating the smoothness constraint of the deformation field. This allows for reasonable degrees of freedom of deformation within the tissue during the registration process, while providing greater smoothness to maintain structural integrity when crossing anatomical boundaries. This type of information feedback mechanism allows the segmentation task to benefit from more accurate spatial alignment, while the registration task can obtain prior knowledge of anatomical structures. Compared to the traditional approach of treating segmentation and registration as isolated tasks, this approach can fully explore the anatomical connections between segmentation and registration, promoting deep interaction between the two processes.

[0071] In this embodiment, the shared depth feature encoder Employing a residual connection structure, each stage contains two consecutive 3D residual blocks. The input image is downsampled four times using max pooling (stride=2), outputting a multi-scale feature map set. The deepest semantic feature map output by the encoder The input segmentation path is processed to generate a segmentation probability distribution map of each voxel belonging to different brain tissues. The high-level feature map output by the encoder and The input registration path is processed to generate the deformation field. .

[0072] In this embodiment, the segmentation path uses an improved 3D U-Net architecture to perform segmentation processing, generating a segmentation probability distribution map of each voxel belonging to different brain tissues. The improved 3D U-Net architecture is based on the 3D U-Net architecture. A multimodal channel attention module is embedded after the encoder bottleneck layer to calculate multimodal statistical features. These features are then fused and input into an MLP (Multilayer Perceptron) for nonlinear transformation. A multi-scale spatial attention module is embedded at each skip connection in the decoder to perform multi-scale deformable convolution on the input feature map and embeds multi-map prior information to guide the optimization of boundary details. Finally, the Softmax function outputs the probability distribution map of each voxel belonging to different brain tissues. This embodiment integrates a multimodal channel attention module and a multi-scale spatial attention module on the basis of the 3D U-Net architecture to form an improved 3D U-Net architecture. The improved architecture is embedded in the multimodal channel attention module. By utilizing the multimodal channel attention mechanism and the multi-scale deformable spatial attention mechanism, it is possible to accurately capture global semantics and local boundaries without increasing significant computational overhead, thereby enhancing feature representation.

[0073] In this embodiment, the multimodal channel attention module employs a multimodal channel statistical fusion and adaptive bottleneck adjustment mechanism. The channel statistical features include mean, extreme value, variance, and skewness features. During the fusion of these features, the contribution of each channel statistical feature is adjusted using a weight matrix. By introducing channel distribution moment features and dynamic weight learning on top of traditional dual pooling, the multimodal channel attention module significantly improves the ability to model inter-channel dependencies.

[0074] Specifically, let the input feature map be... The multimodal channel attention module extracts feature information with different statistical properties through four-channel statistical features: mean feature (first moment). Extreme value characteristics Variance features used to reflect the degree of dispersion of a feature distribution The second moment and the skewness feature (third moment) used to describe the asymmetry of the feature distribution are then combined to obtain the fused feature by fusing the four channel statistics of mean, extreme values, variance, and skewness. It also dynamically adjusts the contribution of the four statistics through learnable attention weights. (i=1,2,3,4) to implement a multi-statistic adaptive fusion mechanism based on attention weights, then the fused features It can be represented as:

[0075] (1)

[0076] in, , To enable soft selection of statistics by learning weight matrices, , This represents the i-th and j-th channel descriptors.

[0077] MLP employs a two-layer perceptron with an adaptive bottleneck dimension, where the bottleneck dimension is determined based on the number of input channels and the network layer depth, for example, it can be adaptively determined using the following formula:

[0078] (2)

[0079] in, Indicates the bottleneck dimension. For network layer depth, Based on the compression factor, This represents the number of channel statistical features, where C is the number of input channels.

[0080] Furthermore, the MLP computation process also includes introducing residual connections to prevent gradient vanishing:

[0081] (3)

[0082] Among them, is the channel attention weight vector, , , 、 are learnable weight matrices, represents the ReLU activation function.

[0083] In this embodiment, the multi-modal channel attention module enhances the global feature representation by adopting a multi-statistical descriptor (mean, maximum, variance, skewness, etc.) and an adaptive weight fusion mechanism. Introducing high-order moment statistics (variance, skewness) can also capture richer channel distribution information. The dynamic weight mechanism can realize the content-adaptive selection of statistics, and the bottleneck dimension adjustment of the MLP can fit the information density requirements of different-level features, which can significantly improve the modeling ability of the inter-channel dependence relationship.

[0084] In this embodiment, the multi-scale spatial attention module adopts a hybrid attention mechanism of multi-scale deformable convolution and anatomical prior embedding, and realizes accurate spatial feature recalibration through geometric adaptive receptive fields and expert knowledge guidance. The specific processing flow of the multi-scale spatial attention module includes:

[0085] 1) Spatial saliency feature extraction adopts a multi-granularity pooling strategy: a multi-granularity pooling strategy is adopted for the input feature map to extract four spatial saliency descriptors as the guiding information for adaptively adjusting the sampling positions of subsequent deformable convolution branches. Specifically, it includes: obtaining the average spatial response of the feature map through global average pooling ; highlighting the most significant spatial position features through global maximum pooling ; obtaining a descriptor representing spatial consistency by calculating the variance over the channel dimension and pooling ; obtaining a descriptor for measuring the richness of spatial information by calculating the information entropy at spatial positions and pooling .

[0086] 2) Use multiple deformable convolution branches with different scales to perform convolution operations on the input feature map. The deformable convolution operation of each branch uses the extracted specific spatial descriptor to predict the spatial offset field through a lightweight convolution layer, so that the sampling position of the convolution kernel can be adapted to the geometric structure of the input feature, enhancing the model's representation ability for complex anatomical boundaries. The specific branch configuration is as follows: Local detail branch: Adopt deformable convolution, and its offset is obtained by processing the most significant position descriptor through a convolution layer to focus on the local high-response area; Context-aware branch: Adopt The offset of the dilated deformable convolution (dilation rate = 2) Through a Convolutional layer processing is based on average response and consistency descriptor The concatenated features are used to capture a wider range of contextual structure information; global structure branch: employing Deformable convolution, its offset Through a Convolutional layers process information entropy descriptors This allows us to perceive the global spatial information distribution pattern. These calculated offsets are directly built into their respective deformable convolution operations. In this process, it is used to adaptively sample and convolve input features using irregular grid points, thereby generating a geometry-aware feature map. .

[0087] 3) Adaptive fusion of each convolutional branch using a gating mechanism achieves content-aware feature selection. The calculation expression is:

[0088] (4)

[0089] in, The number of convolution branches. The result of branch feature fusion. For gating weights, , The weight matrix is ​​a learnable matrix. For feature maps Perform global average pooling. For the i-th deformable convolution operation, These are the spatial saliency descriptors extracted from the input feature map using global average pooling, global max pooling, global variance pooling, and global entropy pooling, respectively.

[0090] 4) Based on the branch feature fusion results Atlas fusion employs an anatomical prior embedding approach, embedding multi-channel prior information representing the spatial distribution of different anatomical regions, obtained from standard brain atlas templates, into the features to guide network optimization and focus on important brain region boundaries. Specifically, multi-atlas priors... For a predefined set of depth-aligned features that are aligned with the input feature map space ,high ,width ) probability map or binary mask, where Here, represents the number of brain regions, and each channel represents the spatial location information of a specific anatomical region (such as the hippocampus, amygdala, cortical regions, etc.). The computational expression for atlas fusion using anatomical prior embedding can be expressed as:

[0091] (5)

[0092] in, For the results of map fusion, For multi-map priors, K represents the number of brain regions, and D, H, and W represent the depth, height, and width of the feature maps, respectively. These are attention weights for the brain map, used to adaptively adjust the importance of prior knowledge in different brain regions. The learnable prior strength coefficient controls the strength of the prior information injection; This represents the Sigmoid activation function, which combines anatomical priors as strong guiding signals with learned features, enabling the network to more accurately locate and segment complex brain tissue structures during the decoding process.

[0093] 5) Based on the map fusion results The final output of the multi-scale spatial attention module is obtained by using gated residual connections:

[0094] (6)

[0095] in, To integrate weights, The weight matrix is ​​a learnable matrix. To perform global average pooling on the input feature map X. For the input feature map, ( ) represents the Sigmoid activation function.

[0096] This embodiment embeds a multi-scale spatial attention module at each level of skip connection in the decoder. This module uses deformable convolution and anatomical prior guidance mechanism to optimize boundary details and force the network to pay attention to the boundaries of important brain regions.

[0097] Furthermore, in this embodiment, the convolution operation in the decoder of the improved 3D U-Net architecture uses dynamic grouped convolution, in which the grouping strategy is determined based on the greatest common divisor of the number of input and output channels according to feature similarity:

[0098] (7)

[0099] (8)

[0100] in, The number of groups is dynamically determined. Number of input and output channels. This indicates finding the greatest common divisor. The preset maximum similarity normalization factor, Let i be the i-th and j-th channel features of the input feature map X.

[0101] In this embodiment, the grouped convolution result is calculated based on the input feature map G and the grouping strategy G to perform channel recalibration on the grouped convolution operation. The calculation expression is as follows:

[0102] (9)

[0103] in, This represents the result of grouped convolution. This is the scaling factor for each set of features. This indicates a grouped convolution operation;

[0104] Furthermore, a cross-group feature interaction module is introduced to perform [further processing] based on the group convolution results after group convolution. Cross-group information exchange:

[0105] (10)

[0106] in, This operation represents the result of cross-group information exchange. It achieves inter-group information flow through a 1×1×1 convolution, with only a small number of parameters. .

[0107] Based on the results of cross-group information exchange A gradual strategy is used for channel compression:

[0108] (11)

[0109] in, This indicates the first 1×1×1 convolution compression operation. This indicates the second 1×1×1 convolution compression operation. This represents intermediate features before compression. This represents the intermediate features after compression. For example, the number of channels in the two compressions are respectively... and Finally, the target number of channels is restored through deconvolution.

[0110] This embodiment replaces standard convolution with dynamic group convolution based on feature similarity. Instead of fixed grouping, it adaptively determines the grouping strategy according to the mathematical characteristics of the number of input and output channels. While ensuring necessary interaction between channels, it achieves extreme compression of the number of parameters, which can significantly reduce the number of parameters.

[0111] In this embodiment, the registration path adopts a two-stage VoxelMorph architecture, which simplifies the four-stage codec to two stages and removes skip connections to reduce complexity. The decoder gradually restores spatial resolution through transposed convolution, and the final output uses Tanh-activated 3D convolution to generate a differential homeomorphic transformation field. .

[0112] In this embodiment, the multi-task collaborative loss function adopts a multi-scale spatially weighted composite loss architecture. The training process is optimized through anatomical structure-aware weight mapping. The calculation expression for the multi-task collaborative loss function is as follows:

[0113] (12)

[0114] in, , , These are the weighting coefficients and the segmentation loss, respectively. The results are calculated using multi-scale Dice loss and weighted cross-entropy loss. Multi-scale Dice loss is obtained by combining Dice losses calculated at different spatial scales. Weighted cross-entropy loss is a boundary-weighted cross-entropy loss. Structural consistency loss describes the topological feature differences between the predicted results and the true labels. Multi-scale processing adapts to different cerebellar and cerebellar structures. Boundary-sensitive design enhances edge segmentation accuracy. Topological loss ensures anatomical rationality, and spatial weighting achieves regional differentiation optimization. Registration loss... The results are calculated using multi-resolution global cross-correlation loss, anisotropic smoothing constraint, Jacobian determinant regularization term, and differential homeomorphism constraint. The global cross-correlation loss describes the similarity between images; the anisotropic smoothing constraint is a diffusion constraint that integrates differences in mechanical properties in different directions; and the differential homeomorphism constraint is an exponential mapping of the velocity field. The cooperative regularization term... This enables the deep transfer of anatomical structural information to the registration task; the calculation expression is as follows:

[0115] (13)

[0116] Where p represents the position coordinates of a voxel in three-dimensional space. Indicated by A local window centered on the center. The tissue-specific mechanical constraints are calculated by integrating segmentation masks of different brain tissues. By measuring the deformation field The partial derivatives in the three spatial directions are squared and summed to obtain the result, which serves as the basis for measuring the smoothness of the deformation field.

[0117] Specifically, the segmentation loss term introduces multi-scale Dice loss, calculating the Dice loss at three spatial scales:

[0118] (14)

[0119] in, To downsample the true segmentation labels to After resolution, the first The label value of the individual element. For scale weights, This indicates that the model is at the corresponding scale. The first prediction The segmentation probability value of a single element. denoted as the minimum constant of the smoothing term, and s represents the index of the spatial scale. Representing scale The total number of prime numbers.

[0120] The weighted cross-entropy loss is extended to a boundary-sensitive form, and its calculation expression is:

[0121] (15)

[0122] in, Let i be the distance mapping from the voxel i to the nearest boundary. For boundary enhancement coefficient, C represents the organization category index and the total number of categories. This represents the boundary distance value at voxel i. Represents the total number of prime numbers. This represents the true label (one-hot form) of voxel i in class c. This represents the weight of category c. Represents the boundary enhancement coefficient. This represents the probability that voxel i is predicted to be of class c.

[0123] The formula for calculating structural consistency loss is:

[0124] (16)

[0125] in, Let K be the k-th Betti number used to characterize topological invariants, where K represents the number of Betti numbers selected. , Represents the true label Y and the predicted label. The kth Betti number.

[0126] The final joint loss is obtained by integrating spatially adaptive weights to obtain the segmentation loss. :

[0127] (17)

[0128] in, This is a spatial weight matrix based on anatomical atlases, with higher weights assigned to key brain regions.

[0129] This embodiment employs the aforementioned segmentation loss. Multi-scale processing can adapt to different brain structures, and the use of boundary-sensitive weighted cross-entropy loss can enhance edge segmentation accuracy. Structural consistency loss can ensure anatomical rationality, and combined with spatial weighting, it can also achieve regional differentiation optimization.

[0130] The registration loss term employs a composite loss architecture with multi-resolution differential homeomorphism constraints, and a physically inspired regularization term ensures the biomechanical rationality of the deformation field. The image similarity loss is extended to a multi-resolution adaptive weighted form, calculating the globally normalized cross-correlation loss at three different resolution levels:

[0131] (18)

[0132] in, and These are downsampled versions of the fixed image and the registered image at the l-th pyramid level, respectively. For hierarchical weights, satisfying , The statistical relevance measure at level l is calculated using the following expression:

[0133] (19)

[0134] in This indicates the inner product operation. This statistical correlation measure is invariant to linear transformations of grayscale values ​​and can handle intensity differences caused by different scanning protocols well.

[0135] The deformation field smoothing loss introduces anisotropic diffusion constraints to account for the differences in the mechanical properties of brain tissue in different directions. The calculation expression for the anisotropic smoothing constraints is as follows:

[0136] (20)

[0137] in These are directional weighting coefficients, set according to the stiffness characteristics of brain tissue in different directions. For example, they can be set to... To reflect the difference in axial stiffness.

[0138] The Jacobi regularization loss is extended to a regularization term based on statistical distribution. This not only penalizes negative Jacobi determinants but also constrains the overall distribution of determinant values. The calculation expression can be represented as:

[0139] (twenty one)

[0140] in, and These are the weight coefficients for the negative Jacobian penalty term and the distribution deviation penalty term, respectively. Let represent the set of voxel locations in the deformation field where the Jacobian determinant is negative. express The total number of voxels in the set. Represents the deformation field At voxel location The determinant of the Jacobian matrix at a given location represents the rate of change of local volume. Represents the positions of all voxels in the deformation field. The distribution of values. This indicates that the mean is 1 and the variance is... Gaussian distribution, This is the preset variance value. Let KL represent the Kullback-Leibler divergence, used to measure the difference between two probability distributions. The first term of this loss function specifically penalizes non-physical deformations that lead to local volume reversals (i.e., negative Jacobian determinants); the second term constrains the volume change distribution of the entire deformation field to approximate a Gaussian distribution with a mean of 1 through the KL divergence, thereby encouraging volume preservation properties overall and preventing excessive expansion or contraction.

[0141] This embodiment uses the aforementioned Jacobian determinant regularization term to directly monitor and constrain the local geometric properties of the deformation field. The first part of this term... Used to punish voxel locations with negative Jacobian determinants, this can prevent non-physical deformations such as "tissue folding." (Part Two) By constraining the distribution of the Jacobian determinant of the entire deformation field by KL divergence to approximate a Gaussian distribution with a mean of 1, the overall "volume retention" is encouraged, which can suppress unreasonable local overexpansion or contraction.

[0142] Furthermore, an exponential mapping is used to form a differential homeomorphic constraint to ensure topological preservation and thus guarantee the invertibility of the deformation field:

[0143] (twenty two)

[0144] in, For the velocity field, the deformation field is ensured through integral equations. Differential homeomorphism property.

[0145] In this embodiment, at the end of the registration path, a differential homeomorphic deformation field generation mechanism based on velocity field exponential mapping, as shown in Equation (22), is used to generate the final deformation field. This ensures that the generated deformation field is differential homeomorphic, i.e., smooth and reversible, and that the determinant of the Jacobian matrix is ​​positive everywhere. This effectively ensures the preservation of the topological structure and fundamentally avoids tissue tearing or penetration.

[0146] The corresponding regularization term is:

[0147] (twenty three)

[0148] In this embodiment, a differential homeomorphism constraint term as shown in equation (23) is used to directly act on the velocity field. This constraint term regularizes the smoothness and amplitude of the velocity field itself during the training process, which can ensure the stability of the exponential mapping process and the quality of the generated deformation field.

[0149] The final joint loss function yields the registration loss. :

[0150] (twenty four)

[0151] In this embodiment, the collaborative regularization term A composite constraint mechanism integrating multi-scale boundary sensing and mechanical property fusion is employed to achieve deep collaborative optimization of segmentation and registration tasks. Specifically, boundary strength quantization introduces multi-scale gradient magnitude fusion to calculate segmentation probability maps in different scale spaces. gradient information :

[0152] (25)

[0153] in The standard deviation is Gaussian kernels are used to achieve multi-scale boundary detection. Indicates the index of the scale space. This represents a segmentation probability graph.

[0154] Calculate segmentation probability maps for spaces at different scales gradient information Boundary-aware weight map is generated through nonlinear transformation. :

[0155] (26)

[0156] in, This indicates a preset control coefficient to control the overall weight intensity. For multi-scale fusion weights, The function ensures that the weight values ​​are bounded.

[0157] In this embodiment, tissue-specific mechanical constraints The regularization strength can be adjusted based on the biomechanical characteristics of different brain tissues; for example, it can be obtained by combining segmentation masks for white matter, gray matter, and cerebrospinal fluid.

[0158] (27)

[0159] in These are segmentation masks for white matter, gray matter, and cerebrospinal fluid, respectively. It reflects the relative stiffness of different structures.

[0160] To further conform the deformation field to the mechanical properties of biological tissues, a curl constraint term is introduced based on the basic form, forming an extended composite cooperative regularization term, defined as follows:

[0161] (28)

[0162] in, These are the weighting coefficients for the curl constraint term. (This is achieved through the curl term.) By penalizing non-conservative deformation fields, we encourage deformation fields to be derived from potential functions as much as possible, thereby better simulating the mechanical properties of biological tissues that tend to maintain local rotational invariance during deformation.

[0163] This embodiment employs the aforementioned collaborative regularization term. Multi-scale boundary detection can enhance the sensitivity to blurred boundaries, tissue-specific weights can be used to introduce prior biomechanical knowledge, and curl constraints can make the deformation field more in line with physical laws, thus realizing end-to-end collaborative optimization from image features to physical constraints.

[0164] In this embodiment, before inputting the image pair to be registered into the segmentation and registration collaborative model (including the model training stage and the model real-time application stage), the method further includes preprocessing the floating image and the fixed image in the image pair respectively. The preprocessed floating image and fixed image are then stitched together and input into the segmentation and registration collaborative model. The preprocessing includes using a segmentation network to segment the input image to generate a brain tissue probability map. Based on the generated brain tissue probability map Obtaining a binary mask of brain tissue Using a binary mask of brain tissue Brain tissue regions are extracted, and the extracted brain tissue regions are standardized before outputting the standardized results. The standardization process includes estimating the bias map. Intensity correction results were obtained by performing intensity correction on the extracted brain tissue regions. Calculate tissue-specific local mean According to the strength correction results Tissue-specific local mean The final standardized result is calculated. The preprocessing stage employs a deep learning-based segmentation and adaptive standardization process, which can more accurately eliminate non-brain tissue interference and correct intensity inhomogeneities compared to traditional methods.

[0165] In this embodiment, an improved Z-score normalization process is adopted. Based on the Z-score normalization process, off-field correction and local intensity adaptive mechanisms are introduced to preprocess both floating and fixed images. This eliminates intensity differences and achieves spatial normalization. Specific steps include:

[0166] 1) Based on the generated brain tissue probability map A binary mask of brain tissue was generated using adaptive thresholding. The adaptive threshold is based on the probability map of brain tissue. The mean and standard deviation are calculated from this.

[0167] Specifically, precise brain tissue extraction is achieved using a cascaded segmentation network, with an initial brain tissue probability map generated using a lightweight 3D U-Net network:

[0168] (29)

[0169] in, , for pre-trained brain tissue segmentation networks, These are the network parameters. An accurate binary mask is generated through adaptive thresholding:

[0170] (30)

[0171] Among them, adaptive threshold According to the probability diagram The mean and standard deviation are calculated, for example. , and These are probability graphs. The mean and standard deviation, These are adjustable parameters. Compared to traditional morphological manipulations, the above method can more accurately preserve the details of brain tissue boundaries.

[0172] 2) Introducing a multimodal adaptive correction mechanism to achieve intensity normalization: Based on traditional Z-score normalization, off-field correction and modality-specific normalization are added to eliminate image intensity inhomogeneity and preserve key anatomical contrast.

[0173] (31)

[0174] in This is the bias field map estimated using the N4 bias field correction algorithm.

[0175] The final standardized result is obtained based on the improved Z-score standardization process:

[0176] (32)

[0177] in, This represents the standardized result after standardizing the extracted brain tissue regions. The global intensity standard deviation for the entire brain tissue region. To prevent division by zero of extremely small constants, The local mean value specific to the tissue is expressed by the following formula:

[0178] (33)

[0179] in, For A local window centered on the center. The organization probability weight map can be obtained through a pre-trained organization segmentation model. Represents a local window voxel index within.

[0180] Furthermore, to achieve alignment of brain images from different individuals in anatomical space, spatial standardization is required after intensity standardization. Spatial standardization employs differential homeomorphic registration to ensure that the topological structure is not destroyed. A fast differential homeomorphic registration algorithm is used to process each intensity-standardized sample. Registered to a standard brain map template space, the computational expression is:

[0181] (34)

[0182] in, This indicates that it includes a similarity measure and a differential homeomorphism regularization term. Standard brain atlas image, This represents the deformation field from the input image space to the standard atlas space. This is the spatially standardized template deformation field.

[0183] The images preprocessed in this embodiment have unified anatomical spatial coordinates. Both floating and fixed images are converted into a unified and standardized intensity and spatial coordinate system, providing a standardized basis for subsequent analysis. Furthermore, the cascaded deep learning segmentation method has higher boundary accuracy than traditional methods. Multimodal intensity can be used to correct and eliminate scanning protocol differences. Tissue-specific normalization preserves the contrast information of anatomical structures, and the differential homeomorphism criterion can ensure that spatial standardization does not destroy the topological structure of brain tissue.

[0184] After training the segmentation and registration co-training model constructed above, for a new input image pair, the final three-dimensional brain tissue segmentation label map can be output from the segmentation path in the model, and the differential homeomorphic deformation field output from the registration path can be obtained by resampling the floating image to obtain the registration result aligned with the fixed image space.

[0185] like Figure 2 As shown in the detailed process of achieving collaborative optimization of three-dimensional tissue segmentation and registration in brain neural images using the above-mentioned method of the present invention in a specific application embodiment, the process is as follows:

[0186] Step 1: Acquire 3D MRI images of the brain to be registered, including floating images as moving targets. and a fixed image as an alignment reference ;

[0187] Step 2: Preprocess the floating and fixed images separately. This preprocessing process generates a brain tissue probability map using a pre-trained 3DU-Net segmentation network and combines it with an adaptive thresholding algorithm to generate an accurate binary mask for brain tissue. The threshold is dynamically adjusted based on the local statistical characteristics of the probability plot. Then, using this mask, voxel-by-voxel multiplication is performed. Pure brain tissue regions were extracted to exclude interference from non-target tissues such as the skull. The extracted brain tissue regions were then subjected to improved Z-score normalization, which incorporates off-field correction and local intensity adaptation mechanisms: the off-field map was estimated using the N4 off-field correction algorithm. Perform strength correction Calculate tissue-specific local mean Standardization To eliminate intensity differences caused by different scanning devices or protocols and improve model robustness; to standardize each sample space to the standard template space through the differential homeomorphic registration algorithm to ensure the spatial consistency of anatomical structures;

[0188] Step 3: Concatenate the preprocessed floating image and the fixed image along the channel dimension to form a two-channel tensor. Input shared deep feature encoder The encoder employs a residual connection structure, with each stage containing two consecutive 3D residual blocks. It performs four levels of downsampling through max pooling (step size = 2) to output a multi-scale feature map set. ;

[0189] Step 4: Extract the deepest semantic feature map from the encoder output. The input segmentation path is processed; the segmentation path adopts an improved 3D U-Net architecture, embedding a multimodal channel attention module after the encoder bottleneck layer, and enhancing global feature representation through a fusion mechanism of four statistical descriptors (mean, maximum, variance, and skewness) and adaptive weights; multi-scale spatial attention modules are embedded at the skip connections of each level of the decoder, and deformable convolution and anatomical prior guidance mechanisms are used to optimize boundary details; at the same time, dynamic group convolution based on feature similarity is used to replace standard convolution, significantly reducing the number of parameters; finally, the probability distribution map of each voxel belonging to different brain tissues is output through the Softmax function. ;

[0190] Step 5: Output the high-level feature map from the encoder. and The input registration path is processed; the registration path adopts the structurally compressed VoxelMorph architecture, which simplifies the original four-level encoder-decoder into a two-level structure and removes skip connections to reduce computational complexity; the decoder gradually restores spatial resolution through transposed convolution, and the final output uses Tanh-activated 3D convolution to generate a dense deformation field. ;

[0191] Step 6: Use the multi-task collaborative loss function End-to-end training is performed; the segmentation loss is used in this process. Combining multi-scale Dice loss with boundary-weighted cross-entropy loss; registration loss Includes multi-resolution global cross-correlation loss, anisotropic smoothing constraint, Jacobian determinant regularization, and differential homeomorphism constraint; cooperative regularization term. By segmenting the probability graph Calculate the multi-scale boundary weight map And combined with tissue-specific mechanical weights The gradient penalty term of the deformation field is jointly modulated:

[0192] This enables the deep transfer of anatomical structural information to the registration task;

[0193] Step 7: After the model training is completed, for a new input image pair, the segmentation path outputs the final three-dimensional brain tissue segmentation label map, and the registration path outputs the differential homeomorphic deformation field and obtains the registration result aligned with the fixed image space by resampling the floating image.

[0194] To implement the above method, this embodiment further constructs a collaborative optimization system, including:

[0195] The data preprocessing module is configured to perform preprocessing operations on the input floating and fixed 3D magnetic resonance images of the brain. Specifically, this module generates a brain tissue probability map through a pre-trained 3D U-Net segmentation network and combines it with an adaptive thresholding algorithm to generate an accurate binary mask of the brain tissue, wherein the threshold is dynamically adjusted according to the local statistical characteristics of the probability map. The pure brain tissue region is then extracted using this mask to exclude interference from non-target tissues. An improved Z-score normalization method is used to normalize the intensity of the extracted brain tissue region. Offset field correction and local intensity adaptation mechanisms are introduced. The offset field map is estimated by the N4 offset field correction algorithm for intensity correction. The tissue-specific local mean is calculated for standardization to ensure that image data from different sources have a consistent intensity distribution, providing standardized input for subsequent networks.

[0196] The shared feature encoding module contains a deep 3D convolutional neural network encoder based on residual connections. This module receives a normalized two-channel image tensor output by the data preprocessing module. It extracts and outputs a set of multi-scale three-dimensional feature maps with rich semantic information through its four-level downsampling structure. Each downsampling module of the encoder consists of two consecutive residual blocks. Each residual block contains two 3×3×3 convolutional layers, a batch normalization layer, and a ReLU activation function. By using 1×1×3 convolutions on skip connections to match the number of channels with the resolution, the gradient flow is effectively preserved while increasing the network depth, thus enhancing the feature representation capability.

[0197] The path segmentation module incorporates an improved 3D U-Net architecture as a decoder. This module receives the deepest feature map output by the shared feature encoding module. As input, a multimodal channel attention module is embedded after the encoder bottleneck layer, which enhances global feature representation through a fusion mechanism of multiple statistical descriptors (mean, maximum, variance, skewness, etc.) and adaptive weights. A multi-scale spatial attention module is embedded at each skip connection of the decoder, employing deformable convolution and anatomical prior guidance mechanisms to optimize boundary details. Simultaneously, standard convolution operations in the decoder are replaced by a dynamically grouped convolution module based on feature similarity. This module dynamically determines the grouping strategy based on the greatest common divisor of the input and output channel numbers, and combines cross-group feature interaction and progressive channel compression to significantly reduce the number of model parameters. Finally, this module outputs a probability distribution map of each voxel belonging to different brain tissue categories through the Softmax function. ;

[0198] The registration path module includes a lightweight VoxelMorph architecture as the decoder. This module receives the high-level feature map output by the shared feature encoding module. and As input, the traditional four-level encoding / decoding structure is compressed into a two-level structure, and skip connections are removed to reduce computational complexity and memory consumption. This module gradually restores spatial resolution through transposed convolution, and the final output uses Tanh-activated 3D convolution to generate a dense three-dimensional deformation field from a floating image space to a fixed image space. And through exponential mapping Ensure the differential homeomorphism of the deformation field;

[0199] The co-training module is configured to compute a comprehensive joint loss function during the model training phase. The network parameters are then collaboratively optimized using the backpropagation algorithm; the joint loss function consists of a segmentation loss term. Registration loss item and co-regularization terms The weighted summation constitutes, i.e. This module establishes a deep connection between the segmentation task and the registration task through the collaborative regularization term, generates multi-scale boundary-aware weights using the probability map output by the segmentation path, and applies enhanced smoothness constraints to the deformation field in the anatomical boundary region in combination with tissue-specific mechanical constraints, thereby driving the two tasks to promote each other and optimize collaboratively.

[0200] The result generation and output module is configured to, during the model inference stage, integrate the final segmentation probability map output by the segmentation path module with the deformation field predicted by the registration path module, generate a three-dimensional brain tissue segmentation label map by taking the maximum probability value of the segmentation probability map; at the same time, use a spatial transformation network combined with the deformation field to resample the floating image, generate a registration result that is precisely aligned with the fixed image in space, and output the above segmentation and registration results.

[0201] This embodiment also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method as described above.

[0202] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0203] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0204] This invention can be flexibly applied to the segmentation and registration of three-dimensional tissues in various brain images, and can also be applied to the segmentation and registration of images of the heart, liver, etc.

[0205] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0206] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A collaborative optimization method for three-dimensional tissue segmentation and registration of brain neural images, characterized in that the steps include: Comprise: Step S01. Construct a segmentation and registration collaborative model, which comprises a shared feature encoder, a segmentation path and a registration path, the shared feature encoder is used for receiving an image pair to be registered for feature extraction, outputting a multi-scale feature map set, the segmentation path is used for segmenting the highest layer semantic feature map in the multi-scale feature map set to generate a segmentation probability distribution map of each voxel belonging to different brain tissues, and the registration path is used for registering two or more high layer feature maps in the multi-scale feature map set to generate a deformation field; Step S02. Obtain a training set composed of different brain three-dimensional magnetic resonance image pairs, the brain three-dimensional magnetic resonance image pair comprises a floating image as a moving target and a fixed image as an alignment reference; Step S03. Collaborative training the segmentation-registration collaborative model using the obtained training set according to a multi-task collaborative loss function, the multi-task collaborative loss function including a segmentation loss, a registration loss, and a collaborative regularization term, in which the collaborative regularization term is determined by using a multi-scale boundary weight map and a tissue-specific mechanics weight jointly modulate a deformation field gradient penalty term , representing a deformation field at a voxel position , the multi-scale boundary weight map is calculated using a segmentation probability distribution map , the tissue-specific mechanics weight is a weight map determined according to biomechanical prior information of brain tissue; Step S04. Real-time receive brain three-dimensional magnetic resonance image pairs to be registered, input into the trained segmentation and registration collaborative model, output the final three-dimensional brain tissue segmentation label map from the segmentation path in the model, and output the deformation field from the registration path, and obtain the registration result aligned with the fixed image space by resampling the floating image; The segmentation path adopts an improved 3D U-Net architecture to realize segmentation processing, the improved 3D U-Net architecture is based on a 3D U-Net architecture, a multi-modal channel attention module is embedded after an encoder bottleneck layer, so as to calculate multiple channel statistical features, and after the multiple channel statistical features are fused, the multiple channel statistical features are input into an MLP for nonlinear transformation; a multi-scale space attention module is embedded at each level of a decoder jump connection, so as to adopt multi-scale deformable convolution on an input feature map, and multi-atlas prior information is embedded to guide optimization of boundary details, and finally a Softmax function is used to output a probability distribution diagram of each voxel belonging to different brain tissues ; The registration path adopts a two-level structure of VoxelMorph architecture, the decoder restores the spatial resolution step by step through transposed convolution, and the final output end uses a 3D convolution with Tanh activation to generate a differential homeomorphism deformation field.

2. The method of co-optimization of three-dimensional tissue segmentation and registration of brain neuroimages according to claim 1, characterized in that, In the multi-modal channel attention module, the contribution of each heavy channel statistical quantity feature is adjusted by a weight matrix during the fusion process of each heavy channel statistical quantity feature, and the channel statistical quantity feature includes mean value feature, extreme value feature, variance feature and skewness feature; The bottleneck dimension of the MLP is determined according to the input channel number and network layer depth according to the following formula: wherein, denotes the bottleneck dimension, is the network depth, is the base compression factor, C is the number of input channels, denotes the number of channel statistics features.

3. The method of co-optimization of three-dimensional tissue segmentation and registration of brain neuroimages according to claim 1, wherein, The processing flow of the multi-scale spatial attention module comprises: A plurality of deformable convolution branches of different scales are used to perform convolution operation on the input feature map, and each convolution branch is fused through a gating mechanism: wherein, is the number of convolution branches, is the branch feature fusion result, is the gating weight, , is the weight matrix, is the global average pooling on the feature map , is the i-th deformable convolution operation, respectively are spatial saliency descriptors extracted from the input feature map by global average pooling, global max pooling, global variance pooling, and global entropy pooling. According to the branch feature fusion result Atlas fusion is performed in an anatomical prior embedding manner: wherein, is the atlas fusion result, is the multi-atlas prior, K is the number of brain regions, D, H, and W are the depth, height, and width of the feature map, respectively, is the atlas attention weight, is the prior intensity coefficient; According to the atlas fusion result The final output of the multi-scale spatial attention module is obtained by using a gated residual connection: wherein, is a fusion weight, is a weight matrix, is a global average pooling on the input feature map X, is an input feature map, denotes a Sigmoid activation function.

4. The method of co-optimization of three-dimensional tissue parcellation and registration of brain neuroimages of claim 1, wherein, In the decoder of the improved 3D U-Net architecture, the convolution operation uses dynamic group convolution, and in the dynamic group convolution, the grouping strategy is determined according to the greatest common divisor of the input and output channel numbers: wherein, is the average cosine similarity between input channels, is the input feature map, is the determined number of groups, is the number of input and output channels, denotes finding the greatest common divisor, is a preset maximum similarity normalization factor, is the i-th and j-th channel feature of the input feature map X; The grouping convolution result is calculated according to the input feature map G and the grouping strategy G: wherein, denotes a grouped convolution result, is a scaling factor for each group of features, denotes a grouped convolution operation; According to the grouping convolution result Cross-group information exchange: wherein, represents the cross-group information exchange result; According to the cross-group information exchange result Channel compression is performed using a progressive strategy: wherein, denotes a first 1x1x1 convolutional compression operation, denotes a second 1x1x1 convolutional compression operation, denotes an intermediate feature before compression, denotes an intermediate feature after compression; Finally, the target channel number is restored through deconvolution.

5. The method of co-optimization of three-dimensional tissue segmentation and registration of brain neuroimages according to any one of claims 1 to 4, characterized in that, The calculation expression of the multi-task synergistic loss function is wherein, , , are weight coefficients, respectively, segmentation loss is calculated using a multi-scale Dice loss and a weighted cross-entropy loss, the multi-scale Dice loss is obtained by synthesizing Dice losses calculated at different spatial scales, the weighted cross-entropy loss is a boundary-weighted cross-entropy loss, the structural consistency loss is used to describe the difference in topological features between the prediction result and the real label, and the registration loss is calculated using a multi-resolution global cross-correlation loss, an anisotropic smoothing constraint, a Jacobian determinant regularization term, and a differential homeomorphism constraint, the global cross-correlation loss is used to describe the similarity between images, the anisotropic smoothing constraint is obtained by synthesizing the diffusion constraints of the mechanical property differences in different directions, and the differential homeomorphism constraint is an exponential mapping on the velocity field; the calculation expression of the synergistic regularization term is: ， wherein, represents a local window centered at is a tissue-specific mechanical constraint, the tissue-specific mechanical constraint is calculated by integrating different brain tissue segmentation masks, and the segmentation probability map is calculated for different scale spaces gradient information of a boundary-aware weight map is generated by a nonlinear transformation : , represents a scale space index, and L represents the number of scale spaces, represents a preset control coefficient, is a multi-scale fusion weight.​ 6. The method of co-optimization of three-dimensional tissue segmentation and registration of brain neuroimages according to any one of claims 1 to 4, characterized in that, Before inputting the image pair to be registered into the segmentation registration collaborative model, the floating image and the fixed image in the image pair are respectively preprocessed, and the preprocessed floating image and the fixed image are spliced and then input into the segmentation registration collaborative model, the preprocessing includes segmenting the input image using a segmentation network to generate a brain tissue probability map , obtaining a brain tissue binary mask according to the generated brain tissue probability map , extracting a brain tissue region using the brain tissue binary mask , outputting a standardization result after standardization processing of the extracted brain tissue region, the standardization processing includes: estimating a bias field , performing intensity correction on the extracted brain tissue region to obtain an intensity correction result , calculating tissue-specific local mean , calculating the standardization result according to the intensity correction result , and the tissue-specific local mean .​ 7. The method of co-optimization of three-dimensional tissue segmentation and registration of brain neuroimages according to claim 6, characterized in that, According to the generated brain tissue probability map The brain tissue binary mask is generated using adaptive thresholding , the adaptive threshold being computed according to the mean and standard deviation of the brain tissue probability map . The calculation expression of the standardization result is: wherein, denotes a normalized result after the extracted brain tissue region is normalized, is a global intensity standard deviation of the entire brain tissue region, is a minimum constant for preventing division by zero, denotes a tissue-specific local mean, and the calculation expression is: wherein, a local window centered at a local window centered at is an organizational probability weight map; The spatial standardization is realized by using differential homeomorphism registration, and the calculation expression is: wherein denotes a similarity measure and a differential homeomorphism regularization term, is a standard brain atlas image, is a deformation field from the input image space to the standard atlas space, is a spatially normalized template deformation field.

8. A computer apparatus comprising a processor and a memory storing a computer program, characterised in that, The processor is used to execute a computer program to perform the collaborative optimization method for three-dimensional tissue segmentation and registration of brain neural images according to any one of claims 1-7.

Images