Multi-query adaptive segmentation network for cross-center medical images

Abstract

The application belongs to the field of medical image processing, and discloses a multi-query adaptive segmentation network for cross-center medical images. The network comprises a semantic encoder, a multi-query initialization decoder, a regional heterogeneity mapper and a collaborative constraint adapter. The semantic encoder extracts a multi-scale feature map. The multi-query initialization decoder generates a semantic query vector, a boundary-oriented query vector and a true and false positive query vector. The regional heterogeneity mapper outputs a regional heterogeneity modulation field according to the multi-scale feature map. The collaborative constraint adapter generates a prediction mask according to the feature map and the multiple query vectors, and calculates a semantic loss, a boundary loss and a true and false positive loss in combination with a real label. Through an adaptive dynamic modulation mechanism, the application significantly improves the boundary accuracy and the false positive suppression capability of medical image segmentation in a cross-center scenario.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and deep learning technology, specifically to a multi-query adaptive segmentation network for cross-center medical images. Background Technology

[0002] In clinical medical image analysis, image segmentation is a crucial step in extracting organs, lesions, or tissue regions from images such as magnetic resonance imaging (MRI) and computed tomography (CT) scans. Its accuracy directly impacts subsequent disease diagnosis and treatment planning. With the maturity of deep learning technology, segmentation models based on convolutional neural networks (CNNs) or Transformers have achieved excellent performance on single-center datasets. However, medical image data is typically obtained from different hospitals, using different scanning equipment, and employing different acquisition protocols, resulting in significant domain shifts in contrast, texture, and noise distribution. When a model trained on one central dataset is directly applied to data from other central datasets, this domain shift causes a sharp drop in segmentation performance, especially as target boundary regions tend to become blurred or even disappear, while background textures frequently generate false positive segmentation blocks.

[0003] Existing techniques typically mitigate domain shift through data augmentation, domain adaptation, or transfer learning. However, these methods focus primarily on global or overall distribution alignment, lacking the ability to handle fine-grained boundary and easily confused regions. Some solutions introduce multi-branch query mechanisms to separately represent semantics, boundaries, or suppress false positives; however, the optimization process for each query uses fixed loss weights or static interaction methods, failing to adaptively adjust the learning intensity for different regions based on the characteristics of the input image itself. This makes it difficult for the model to simultaneously address boundary sharpness and false positive suppression when facing new, unknown domain images, resulting in unsatisfactory robustness of cross-center segmentation results.

[0004] Therefore, there is an urgent need for an adaptive segmentation scheme that can dynamically adjust the multi-query optimization intensity based on the inherent features of the image, in order to improve the boundary clarity and false positive suppression capability in cross-center scenes. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a multi-query adaptive segmentation network for cross-center medical images, comprising: A semantic encoder is used to extract multi-scale features from input medical images and output multi-scale feature maps. A multi-query initialization decoder, electrically connected to the semantic encoder, is used to generate semantic query vectors, boundary-guided query vectors, and true / false positive query vectors based on the multi-scale feature map. A region heterogeneity mapper, electrically connected to the semantic encoder, is used to measure region heterogeneity based on the multi-scale feature map and output a region heterogeneity modulation field corresponding to the spatial size of the multi-scale feature map. A collaborative constraint adapter is electrically connected to the semantic encoder, the multi-query initialization decoder, and the region heterogeneity mapper, respectively. Based on the multi-scale feature map, the semantic query vector, the boundary-guided query vector, and the true / false positive query vector, the collaborative constraint adapter generates enhanced semantic query vectors, enhanced boundary-guided query vectors, and enhanced true / false positive query vectors. It also generates semantic prediction masks, boundary prediction masks, and true / false positive prediction masks. Furthermore, it calculates semantic loss by combining pre-acquired real semantic labels, calculates boundary loss by combining reference boundary labels generated from the real semantic labels, and calculates true / false positive loss by combining reference true / false positive labels dynamically generated through joint processing of the semantic prediction mask and the real semantic labels. It receives the region heterogeneity modulation field, uses the region heterogeneity modulation field as modulation weights, and performs pixel-wise weighting of the boundary loss and the true / false positive loss. Finally, it generates a collaborative constraint total loss by combining the semantic loss. This collaborative constraint total loss is used to update the parameters of the multi-query initialization decoder during backpropagation.

[0006] Preferably, the regional heterogeneity mapper includes a gradient calculation unit, a local standard deviation statistics unit, and a normalization fusion unit. The gradient calculation unit is used to calculate the feature gradient norm of the multi-scale feature map at each spatial location. The local standard deviation statistics unit is used to calculate the feature value standard deviation of the multi-scale feature map within a local window centered at each spatial location. The normalization fusion unit is used to perform scale normalization fusion processing on the feature gradient norm and the feature value standard deviation to generate and output the regional heterogeneity modulation field.

[0007] Further preferably, the scale normalization fusion process performed by the normalization fusion unit includes: performing a scale normalization operation on the feature gradient norm and the feature value standard deviation, and deriving the modulation value of the regional heterogeneous modulation field based on the normalized ratio, wherein the scale normalization operation includes a combination of division operation and exponential function operation.

[0008] More preferably, the cooperative constraint adapter includes a cooperative modulation factor generator, which is used to perform nonlinear remapping on the regional heterogeneous modulation field to generate a cooperative gradient modulation field, and uses the cooperative gradient modulation field as a pixel-wise weight for the boundary loss and the true / false positive loss.

[0009] More preferably, the nonlinear remapping of the regional heterogeneous modulation field by the cooperative modulation factor generator includes: performing a power function operation on the modulation value of the regional heterogeneous modulation field and multiplying it by a power function operation on the complementary modulation value, and applying a bias processing to the product result to obtain the modulation factor of the cooperative gradient modulation field.

[0010] More preferably, the collaborative constraint adapter receives the regional heterogeneous modulation field, uses the regional heterogeneous modulation field as modulation weight, performs pixel-wise weighting on the boundary loss and the true / false positive loss, and generates the total collaborative constraint loss by combining the semantic loss. Specifically, this includes: multiplying the modulation factor of the collaborative gradient modulation field by the sum of the boundary loss and the true / false positive loss at the corresponding pixel position, summing and averaging over all pixels, and then adding it to the semantic loss.

[0011] More preferably, the collaborative constraint adapter includes a cascaded mask self-attention unit, a query-to-image attention unit, a feedforward network unit, and an image-to-query attention unit. The mask self-attention unit is used to constrain the interaction between the semantic query vector, the boundary-guided query vector, and the true / false positive query vector to limit information interference between different query targets.

[0012] More preferably, the reference boundary label that the collaborative constraint adapter combines to calculate the boundary loss is generated by processing the real semantic label through morphological dilation and morphological erosion, and then smoothing the difference between the two.

[0013] More preferably, the reference true and false positive labels that the collaborative constraint adapter combines to calculate the true and false positive loss are dynamically generated in each training iteration after determining the predicted foreground region based on the current semantic prediction mask and jointly processing the predicted foreground region with the real semantic labels.

[0014] More preferably, the semantic encoder is a backbone network based on a convolutional neural network architecture, and the medical image is one of magnetic resonance imaging image, computed tomography image, and ultrasound image.

[0015] Technical Effects: The multi-query adaptive segmentation network for cross-center medical images provided by this invention introduces a regional heterogeneity mapper to extract heterogeneity information such as local gradient norm and standard deviation from the multi-scale feature map output by the semantic encoder itself, generating a regional heterogeneous modulation field without any additional annotation. A collaborative constraint adapter uses this modulation field as dynamic weights to precisely control the boundary loss and true / false positive loss at the pixel level. This allows the model to automatically enhance the learning intensity of target boundaries and easily confused background regions during training, solving the problem that fixed loss weights or static query interactions in existing technologies cannot adapt to cross-center offsets. This collaborative mechanism simultaneously improves the sharpness of boundary segmentation and the suppression of false positive regions, significantly improving the generalization ability and robustness of cross-center medical image segmentation. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the hierarchical structure of the multi-query adaptive segmentation network architecture for cross-center medical images of the present invention; Figure 2 This is a schematic diagram of the regional heterogeneity mapping calculation of the present invention; Figure 3 This invention relates to a multi-query vector generation and interactive data flow graph; Figure 4 This is a schematic diagram of the closed-loop calculation of collaborative constraint loss in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] The existing technology has technical problems: In the existing cross-center medical image segmentation scheme, the multi-query mechanism usually adopts a preset fixed loss weight or a static inter-query interaction method, which fails to adaptively adjust the optimization intensity of different regions according to the inherent feature distribution of the input image. This results in blurred target boundary segmentation and insufficient suppression of false positive regions in scenarios with significant domain offset.

[0019] Based on this, please refer to Figures 1-4This embodiment provides a multi-query adaptive segmentation network for cross-center medical images, including a semantic encoder, a multi-query initialization decoder, a region heterogeneity mapper, and a collaborative constraint adapter. The semantic encoder extracts multi-scale features from the input medical image and outputs a multi-scale feature map. The multi-query initialization decoder is electrically connected to the semantic encoder and generates semantic query vectors, boundary-guided query vectors, and true / false positive query vectors based on the multi-scale feature map. The region heterogeneity mapper is electrically connected to the semantic encoder and measures region heterogeneity based on the multi-scale feature map, outputting a region heterogeneity modulation field corresponding to the spatial size of the multi-scale feature map. The collaborative constraint adapter is electrically connected to the semantic encoder, the multi-query initialization decoder, and the region heterogeneity mapper, respectively. The collaborative constraint adapter measures the region heterogeneity based on the multi-scale features... The graph, semantic query vector, boundary-oriented query vector, and true / false positive query vector are used to generate enhanced semantic query vector, enhanced boundary-oriented query vector, and enhanced true / false positive query vector. Semantic prediction mask, boundary prediction mask, and true / false positive prediction mask are also generated. Semantic loss is calculated by combining pre-acquired real semantic labels, boundary loss is calculated by combining reference boundary labels generated from real semantic labels, and true / false positive loss is calculated by combining reference true / false positive labels dynamically generated by joint processing of semantic prediction mask and real semantic labels. A regional heterogeneous modulation field is received and used as modulation weights to weight the boundary loss and true / false positive loss pixel-by-pixel. Combined with the semantic loss, a cooperative constraint total loss is generated, which is used to update the parameters of the multi-query initialization decoder during backpropagation.

[0020] The semantic encoder employs a backbone network based on a convolutional neural network architecture, specifically the ResNet-50 structure. This structure has been widely validated in the field of medical image analysis for its stable feature extraction capabilities and good transfer characteristics. The ResNet-50 consists of an initial 7×7 convolutional layer followed by four residual stages, each composed of a stacked number of bottleneck residual blocks. Each bottleneck residual block is composed of three concatenated sub-layers: a 1×1 convolutional layer for dimensionality reduction, a 3×3 convolutional layer for spatial feature extraction, and a 1×1 convolutional layer for dimensionality increase. Each convolutional sub-layer is followed by a batch normalization layer and a ReLU nonlinear activation function. The input of the residual block is added to the output of the third convolutional sub-layer via an identity skip connection, and then activated by ReLU to obtain the output. The feature maps output by the four residual stages are downsampled by 4x, 8x, 16x, and 32x relative to the input image, respectively. This invention uses the 32x downsampled feature map output from the fourth residual stage as the multi-scale feature map, whose spatial dimension is... ,in and These are 1 / 32 of the height and 1 / 32 of the width of the input image, respectively. Before being fed into the semantic encoder, the input medical image undergoes size normalization preprocessing, uniformly adjusting to [size]. Pixel space size and the number of channels can be 1 or 3 depending on the modality. Magnetic resonance imaging (MRI) images are typically single-channel grayscale data. Computed tomography (CT) images, after window width and level adjustments, are also single-channel grayscale data. Ultrasound images are usually single-channel grayscale data, while color Doppler ultrasound images are three-channel RGB data. Preprocessing also includes pixel intensity normalization, linearly mapping the original pixel values ​​to... The interval is transformed into a distribution with zero mean and unit standard deviation through Z-score normalization to eliminate the intensity distribution offset caused by differences in acquisition parameters from different devices. During the forward propagation of the semantic encoder, shallow convolutional kernels learn low-level visual primitives such as edges, corners, and texture gradients; mid-level convolutional kernels respond to local shapes and component-level structures; and deep convolutional kernels form semantic-level category discrimination representations. The number of channels in the final output multi-scale feature map is determined by the number of output channels in the fourth residual stage of ResNet-50, which is 2048. It is compressed to 256 dimensions through a 1×1 convolutional layer to reduce the computational cost of subsequent modules. This 256-dimensional feature map is simultaneously transmitted in three paths to the multi-query initialization decoder, the region heterogeneity mapper, and the collaborative constraint adapter, which are used for query vector generation, heterogeneity measurement, and collaborative interaction decoding, respectively.

[0021] The multi-query initialization decoder internally maintains a learnable embedding map layer, which consists of three independent linear transformation matrices, denoted as follows: , and ,in , and The preset quantities are, in order, semantic query vectors, boundary-oriented query vectors, and true / false positive query vectors. The feature channel count is 256. The three sets of matrix elements are randomly initialized using a Xavier uniform distribution at the start of training, ensuring that the query vectors are significantly different from each other initially, but with controlled numerical ranges. In each forward propagation, the multi-query initialization decoder first performs global average pooling on the multi-scale feature maps, compressing all feature vectors in the spatial dimension into a 256-dimensional global context vector. Then, a two-layer multilayer perceptron maps this global context vector into three sets of conditional offsets, each offset having the same number of rows as the corresponding embedding matrix and a dimension of 256. The mapped conditional offsets are then element-wise added to the three sets of embedding matrices to form the image-conditionalized initial query vector. The preset number of semantic query vectors is specified. Set to 10, each semantic query vector is responsible for aggregating visual prototypes related to a certain semantic subject in the target region from the feature map during the decoding process. For example, in prostate MRI segmentation, these would correspond to feature patterns of sub-regions such as the central zone, peripheral zone, and anterior fibromuscular matrix. This is the preset number of boundary-guided query vectors. The number is set to 20, significantly higher than the number of semantic queries. This design consideration is that boundary morphology variations are far richer than the main semantics; the orientation, width, and surrounding contrast of a single boundary region are diverse, requiring more queries to cover the diversity of boundary patterns. Each boundary-oriented query vector is responsible for encoding the feature patterns of a specific local boundary morphology. The preset number of true and false positive query vectors... The value is set to 10. Each true / false positive query vector is responsible for encoding the discriminative features between the true foreground and the false positive regions, capturing the distribution differences between the two types of regions in the semantic encoder feature space. The dimension of each query vector is... All are fixed at 256, consistent with the number of channels in the multi-scale feature map, to ensure that the feature dimension of the query matches that of the key in the subsequent cross-attention calculation.

[0022] The region heterogeneity mapper comprises a gradient computation unit, a local standard deviation statistics unit, and a normalization fusion unit, which are cascaded together without feedback loops. The gradient computation unit applies a Sobel operator to the spatial dimensions of the multi-scale feature maps. The Sobel operator consists of two 3×3 convolutional kernels. and , Used to detect changes in feature values ​​in the horizontal direction. Used to detect vertical feature value changes. Two convolutional kernels perform independent two-dimensional convolution operations on each channel of the multi-scale feature map. For spatial location... eigenvectors at location Its gradient L2 norm The specific calculation steps are as follows: convert the feature vector... The forward difference in the horizontal direction is approximated as: The forward difference in the vertical direction is approximately: Then, the elements of the two directional difference vectors are squared, summed, and the square root is taken to obtain the horizontal and vertical scalars. Finally, the L2 norm of these two directional scalars, i.e., the square root of the sum of their squares, is taken to obtain the final gradient norm value. This calculation process can be concisely expressed as: in Representing the eigenvector The Each channel component and The gradient norm is approximated by the horizontal and vertical kernels of the Sobel operator, respectively. It reflects the local rate of change of semantic features at that location. In medical images, at the boundaries between organs and background, and between organs themselves, the feature vector output by the semantic encoder often transitions abruptly from one type of semantic representation to another, with synchronous and significant changes in the feature values ​​of each channel, resulting in a significantly high gradient norm value. However, within the organ parenchyma, the activation values ​​of each channel of the feature vector remain relatively stable, with only minor fluctuations due to subtle texture differences, and the gradient norm value approaches zero. In homogeneous background regions, the gradient norm value also remains low. Since the multi-scale feature map is obtained by downsampling the original image by 32 times, the spatial step size between every two adjacent pixels on the feature map corresponds to a distance of 32 pixels in the original image; therefore, the dimension of the feature gradient norm is the reciprocal of its length. .

[0023] The local standard deviation statistical unit applies a fixed-size sliding window operation to the multi-scale feature map. The window size is 5×5 pixels, corresponding to a receptive field of 160×160 pixels in the original image. This is applied to each spatial location on the feature map. Centered on the target location, feature vectors from 25 spatial locations within the window are extracted. Each feature vector has 256 dimensions, forming a 25×256 local feature matrix. The standard deviation of this matrix is ​​calculated column-wise (i.e., by channel dimension), resulting in 256 channel-level standard deviations. The arithmetic mean of these 256 standard deviations is then taken as the standard deviation of the eigenvalue at the center location. The calculation of standard deviation only involves the second-order statistical moments of the eigenvectors; the operation does not change the dimensions, therefore... With feature vectors Both are dimensionless The standard deviation of this eigenvalue reflects the consistency of semantic features in a local area near the central location. In highly heterogeneous regions, such as when lesion edges are accompanied by edema or artifacts, the feature vectors of different pixels within the window differ significantly due to the mixing of signals from different tissue types, resulting in a higher standard deviation. Conversely, in homogeneous organ parenchyma or a uniform background, the feature vectors of pixels within the window are highly similar, leading to a lower standard deviation. It is worth noting that a high gradient norm does not necessarily indicate a high standard deviation. For example, at sharp organ edges, the large difference in features between the two sides results in a large gradient, but the features within each side of the window are consistent, so the standard deviation may be moderate or even low. A low gradient norm with a high standard deviation is common in regions with messy textures or artifacts, such as the intersection of bronchovascular bundles in lung parenchyma, where there are no clear anatomical boundaries but high feature heterogeneity. This combination of gradient norm and standard deviation information allows the regional heterogeneity mapper to distinguish between "structural boundaries" and "textural heterogeneity."

[0024] The normalization fusion unit takes the characteristic gradient norm output by the gradient calculation unit and the eigenvalue standard deviation output by the local standard deviation statistics unit as dual-channel inputs, and performs scale normalization fusion processing to generate a regional heterogeneous modulation field. A preferred execution method is based on the following regional heterogeneity uncertainty mapping formula: In the formula, Indicates spatial location The modulation value of the output regional heterogeneous modulation field is a dimensionless scalar with a value range of [value range missing]. . For semantic encoder at position The output 256-dimensional feature vector, whose component values ​​are generated by the forward computation of the convolutional network, is obtained because the pixel intensity of the original medical image has already been preprocessed with dimensionless normalization on the network input side, and there are no operations that introduce physical dimensions in the network. It is considered dimensionless. The characteristic gradient norm, with dimensions... . The standard deviation of the eigenvalues, with dimensions... . Let be the scale normalization constant, with dimensions . Its exact value is determined based on the spatial downsampling ratio of the multi-scale feature map relative to the original image. In this embodiment, the feature map size is 32×32, the original image is 256×256, and each pixel on the feature map covers an effective area of ​​8×8 pixels in the original image. Set as ,dimension Numerically, it maintains the same scale as the gradient norm. To prevent zero minimum constant, dimensionless The value is This is used to prevent numerical overflow caused by a denominator of zero. (Fraction) Because the numerator and denominator have the same dimensions Dimensionless Satisfies the exponential function The mathematical requirement is that the independent variable must be dimensionless. In the salient region of the boundary, the gradient norm is large while the local standard deviation is moderate, the fractional value is greater than 1, and the exponential output approaches zero. Approaching 1, it assigns the maximum modulation weight. In heterogeneous regions with cluttered textures, such as artifacts or mottled backgrounds, the gradient norm is small but the local standard deviation is large, the fractional value is less than 1, and the exponential output is between 0.3 and 0.7. The value is set in the middle, assigning a moderate modulation weight. In homogeneous foreground or background regions, the gradient norm and local standard deviation are small, the fractional value is very small, and the exponential output is close to 1. As the value approaches zero, the modulation weight is reduced to a minimum. This formula uses the nonlinear mapping of the exponential function to smoothly interpolate the modulation values ​​in the transition region, avoiding the discontinuities and non-differentiability problems caused by hard thresholding. The heterogeneity mapper for the entire region has no learnable parameters and is derived entirely from the first- and second-order spatial statistics of the semantic encoder feature map. Therefore, it can unbiasedly reflect the degree of local structural disorder in the feature space across different domain images, without relying on domain labels or domain classifiers.

[0025] The introduction of regional heterogeneous modulation fields provides crucial adaptive capabilities for cross-center domain offsets. Under the fixed-weight scheme, the weighting coefficients of boundary loss and false positive loss are applied equally to all pixels, ignoring the spatial inhomogeneity of edge degradation and artifact distribution caused by domain offsets. Regional heterogeneous modulation fields incorporate the local heterogeneity information inherent in the feature maps themselves into the training optimization objective, enabling the network's gradient backpropagation to receive spatially discriminative prior guidance, achieving focused learning that "addresses known difficulties" without adding additional supervision.

[0026] The cooperative constraint adapter is the core cooperative scheduling module of this network. Internally, it includes a cooperative modulation factor generator and cascaded mask self-attention units, query-to-image attention units, feedforward network units, and image-to-query attention units. The cooperative modulation factor generator, located on the loss calculation path, performs a nonlinear remapping of the regional heterogeneity modulation field output by the regional heterogeneity mapper to generate a cooperative gradient modulation field. This cooperative gradient modulation field is then used as a pixel-wise weighted spatial weight map for the boundary loss and the true / false positive loss. (Original modulation value) exist The limited range of variation within the interval means that directly using it as a weight only provides linear scaling, lacking sufficient emphasis on extreme values ​​and suppression of the intermediate range. Nonlinear remapping can stretch or compress the dynamic range of the original modulation value, making the weight distribution more closely match the actual task requirements. A preferred implementation method is based on the following query formula for the cooperative gradient modulation factor: ; In the formula Indicates spatial location The modulation factor of the generated cooperative gradient modulation field is dimensionless and has a lower limit of . . The modulation value of the heterogeneous modulation field in the aforementioned region is dimensionless and has a domain. . This is a high uncertainty control index, dimensionless, with a recommended value range of 0.5 to 3.0. When... Time Perform identity mapping; when Time The region with high values, i.e., the boundary and strong heterogeneity, produces an enhancement effect, and the modulation factor is amplified in this region; when... The response to regions of high uncertainty is moderately compressed. This is a low uncertainty control index, dimensionless, with a recommended value range of 0.5 to 3.0. When... Time factor and They exhibit a symmetrical and complementary relationship; when The region of low uncertainty is The modulation factor near 1 is amplified, which intentionally increases the weight of the background in scenes where a balance between foreground and background attention is needed; when The contribution of the low uncertainty region is compressed, making the model more focused on the high uncertainty part. This is the amplitude scaling factor, dimensionless, used to adjust the overall modulation factor magnitude. Because... The maximum value can be determined according to and The value of is calculated analytically by finding the extreme value, for example, when . The maximum value of the product is At this point, you can set Raise the maximum value of the product to around 1.0; when Time product in The maximum value is obtained at the location. At this point, you can set It rose to around 1.0. The basic bias term is dimensionless, typically ranging from 0.1 to 0.5, and ensures the modulation factor through a one-sided constraint. Maintain at least at all pixel locations Weights of magnitude ensure that even When the gradient signal approaches 0 or 1, it does not completely disappear, maintaining the numerical stability and training convergence of the network. Modulation factor The entire computational process is differentiable, and the gradient can be passed to the semantic encoder parameters through backpropagation, which implicitly drives the feature representation itself to produce more discriminative local statistical properties.

[0027] The collaborative constraint adapter constructs a four-stage pipeline from the initial query vector to the enhanced query vector when performing multi-query interactive decoding. In the first stage, the masked self-attention unit receives the semantic query vector. Boundary-oriented query vector True and false positive query vectors The three are concatenated along the dimension of query quantity to form a total query matrix. Before calculating self-attention, a pre-defined binary mask matrix is ​​first applied. Element-wise product masking is performed. The design principles of the mask matrix are as follows: Within the same type of query, the mask value is 1, allowing complete self-attention interaction; between semantic query vectors and boundary-oriented query vectors, the mask value is 0, blocking interaction and preventing boundary features from contaminating the main semantic aggregation; between semantic query vectors and true / false positive query vectors, the mask value is 0, ensuring the purity of the semantic query is not interfered with by false positive discrimination signals; between boundary-oriented query vectors and true / false positive query vectors, some pre-defined connections have a mask value of 1, for example, only allowing corresponding query interactions in spatially adjacent regions, while the rest are set to 0, so as to utilize the auxiliary information of false positive discrimination in boundary tracking without being covered by false positive features. Self-attention calculation uses standard scaled dot product attention: ; in , It is a learnable linear projection matrix. This represents element-wise multiplication. In the second stage, the image attention unit uses the constrained query matrix output from the first stage as the query, flattens the spatial dimensions of the multi-scale feature map output by the semantic encoder, and linearly projects it into a key matrix and a value matrix, then performs multi-head cross-attention. Each attention head has 8 attention heads, and the query, key, and value dimensions of each head are... In the cross-attention process, semantic query vectors tend to form high attention weights with locations in the multi-scale feature map containing semantic information of the target subject. Boundary-oriented query vectors tend to form high attention weights with locations in the feature map that have boundary transition features. True / false positive query vectors search for discriminative feature patterns between the true foreground and the false positive background. The output of the cross-attention process is summed with the input via residual connections and then normalized to form a query vector that integrates image information. In the third stage, the feedforward network unit independently applies a two-layer fully connected network to each query vector, with ReLU activation between the two layers. The hidden layer dimension is expanded to four times the input dimension, i.e., 1024, and the output layer dimension is restored to 256. The nonlinear transformation of the feedforward network enhances the expressive power of the query features, enabling each query to simulate complex nonlinear segmentation decision boundaries. In the fourth stage, the image-to-query attention unit uses the spatial locations of the multi-scale feature map as queries and the updated query matrix output from the third stage as keys and values, performing inverse cross-attention. This inversely integrates the task-specific information contained in the query vector into the feature map, achieving bidirectional information flow. The number of attention heads in this unit is also set to 8. A complete four-stage pipeline process is called a decoding layer. The collaborative constraint adapter repeatedly stacks two decoding layers. The decoding layers do not share parameters. Finally, at the end of the second decoding layer, the enhanced semantic query vector, the enhanced boundary-guided query vector, and the enhanced true / false positive query vector are exported in parallel.

[0028] After generating the enhanced multi-class query vectors, the collaborative constraint adapter incorporates three linear prediction heads. The semantic prediction head is a linear layer that generates the enhanced semantic query vectors. The mapping is the probability that each pixel belongs to the target region; the mapping process is as follows: Generate a semantic prediction mask with a size of 256×256 and a value range of Boundary prediction header to enhance post-boundary guided query vector. With feature map The inner product of the two pixels is activated by a sigmoid function to obtain a boundary prediction mask, also with a size of 256×256. Each value represents the probability that the pixel is located on the target boundary. The true / false positive prediction head is then used to enhance the true / false positive query vector. With feature map The inner product of the two pixels is activated by sigmoid to obtain a true / false positive prediction mask, which is 256×256. Each value represents the confidence that the pixel predicted as the foreground belongs to the real target.

[0029] The collaborative constraint adapter incorporates pre-acquired real semantic labels during the loss calculation stage. Calculate semantic loss . The input image is a binary mask with the same spatial dimensions, where the target region has a pixel value of 1 and the rest are 0. The semantic loss is a weighted combination of Dice loss and binary cross-entropy loss, with the Dice loss being... Cross-entropy loss is The two are added together with equal weights: All are dimensionless. Boundary loss. The calculation is based on the reference boundary label. To monitor the target. Generate The method is to retrieve the actual semantic tags. An expansion mask is obtained by performing a morphological expansion using a 3×3 circular structuring element. Then, using the same structuring element, perform another morphological erosion to obtain the erosion mask. Calculate the boundary zone A Gaussian filter with a window size of 5×5 and a standard deviation of 1.0 is applied to the boundary band to smooth the boundary transition, resulting in the reference boundary label. Boundary loss For pixel-wise binary cross-entropy: Dimensionless. True and false positive loss. The calculation uses dynamically generated reference true and false positive labels. For supervision purposes. In each training iteration, the current semantic prediction mask... The predicted foreground region is obtained by binarization with a threshold of 0.5. Candidate region selection Within the candidate region, if a pixel satisfies... and ,but If satisfied and ,but The remaining pixels are not included in the loss calculation. True / false positive loss. Binary cross-entropy: Dimensionless. Dynamic reference labels are updated automatically as the model's predictive capabilities evolve, preventing mismatches between fixed false positive masks and the model's state in later training stages.

[0030] Cooperative constraint adapter receives cooperative gradient modulation field The boundary loss and true / false positive loss are weighted pixel-wise and combined with the semantic loss to generate the total collaborative constraint loss. A preferred execution method is based on the following region-modulated total collaborative constraint loss function: ; In the formula The total loss is constrained by the network optimizer and is dimensionless, serving as the minimization objective. This is a semantic loss, dimensionless. In this embodiment, the total number of pixels in the input image is used as an example. , dimensionless, is used to average the summation results so that the range of the total loss does not drift with the scaling of the image space. is the modulation factor of the cooperative gradient modulation field, which is dimensionless. This is the boundary loss value, which is dimensionless. The true / false positive loss value is dimensionless. It is a pixel-wise weighted average. The summation and averaging operation is equivalent to applying spatially resolved gradient modulation to each pixel in the loss space. During backpropagation training, At larger pixels, the gradient contributions of boundary loss and true / false positive loss to network parameters are amplified, driving the model to prioritize correcting prediction errors in these challenging regions; At smaller pixels, gradient contributions are compressed, and the model only needs to maintain the existing correct segmentation. This formula design allows the semantic loss to always maintain its inherent global optimization pressure, while the boundary loss and true / false positive loss are spatially selectively modulated. The three work together under a unified objective function without generating competing gradient conflicts among themselves.

[0031] The entire network was trained end-to-end, with the training dataset consisting of annotated medical images collected from at least three different medical centers and two or more different equipment models. The AdamW optimizer was used during training, with an initial learning rate of [value missing]. The learning rate decays to a certain value using a cosine annealing strategy. The weight decay coefficient is The batch size is set to 8 or 16 based on the GPU memory capacity. In each training iteration, the region heterogeneity mapper recalculates the modulation field and co-modulation factor in real time based on the feature map under the current network parameters, without caching historical values, ensuring that the modulation signal is strictly synchronized with the current model state. During the inference phase, the test image is input into the network, using only the forward path of the semantic encoder, multi-query initialization decoder, and co-constraint adapter, outputting a semantic prediction mask as the final segmentation result; the region heterogeneity mapper does not participate in inference computation.

[0032] The backbone network of the semantic encoder can be replaced with a Transformer architecture, such as the Swin-Tiny variant of the Swin Transformer. Its hierarchical structure provides multi-level features at downsampling rates of 4x, 8x, 16x, and 32x. The highest-level features at 32x downsampling are compressed via linear projection to obtain the multi-scale feature map. The number of stacked decoding layers within the collaborative constraint adapter can be increased to three or four layers depending on accuracy requirements, thereby enhancing the interaction depth between queries and features. The preset number of semantic query vectors, boundary-guided query vectors, and true / false positive query vectors can be adjusted according to the anatomical complexity of the target organ and the number of foreground categories. In multi-organ joint segmentation scenarios, the total number of semantic queries is set to 5 times the number of categories, and the total number of boundary-guided queries is set to 10 times the number of categories. The medical image input size can be adjusted to 512×512 or 128×128 according to the spatial resolution requirements of the specific modality, with corresponding scale normalization constants. The value is adjusted synchronously with the downsampling rate. All variants follow the same core design idea, which is to generate a regional heterogeneous modulation field by using the local gradient and dispersion statistics of the semantic encoder feature map itself, and then perform spatial adaptive weighting on the multi-task loss after nonlinear remapping to achieve synergistic optimization of boundary sharpening and false positive suppression in cross-center segmentation.

[0033] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multi-query adaptive segmentation network for cross-center medical images, characterized in that, include: A semantic encoder is used to extract multi-scale features from input medical images and output multi-scale feature maps. A multi-query initialization decoder, electrically connected to the semantic encoder, is used to generate semantic query vectors, boundary-guided query vectors, and true / false positive query vectors based on the multi-scale feature map. A region heterogeneity mapper, electrically connected to the semantic encoder, is used to measure region heterogeneity based on the multi-scale feature map and output a region heterogeneity modulation field corresponding to the spatial size of the multi-scale feature map. A collaborative constraint adapter is electrically connected to the semantic encoder, the multi-query initialization decoder, and the region heterogeneity mapper, respectively. Based on the multi-scale feature map, the semantic query vector, the boundary-guided query vector, and the true / false positive query vector, the collaborative constraint adapter generates enhanced semantic query vectors, enhanced boundary-guided query vectors, and enhanced true / false positive query vectors. It also generates semantic prediction masks, boundary prediction masks, and true / false positive prediction masks. Furthermore, it calculates semantic loss by combining pre-acquired real semantic labels, calculates boundary loss by combining reference boundary labels generated from the real semantic labels, and calculates true / false positive loss by combining reference true / false positive labels dynamically generated through joint processing of the semantic prediction mask and the real semantic labels. It receives the region heterogeneity modulation field, uses the region heterogeneity modulation field as modulation weights, and performs pixel-wise weighting of the boundary loss and the true / false positive loss. Finally, it generates a collaborative constraint total loss by combining the semantic loss. This collaborative constraint total loss is used to update the parameters of the multi-query initialization decoder during backpropagation.

2. The multi-query adaptive segmentation network for cross-center medical images according to claim 1, characterized in that, The regional heterogeneity mapper includes a gradient calculation unit, a local standard deviation statistics unit, and a normalization fusion unit. The gradient calculation unit is used to calculate the feature gradient norm of the multi-scale feature map at each spatial location. The local standard deviation statistics unit is used to calculate the eigenvalue standard deviation of the multi-scale feature map within a local window centered at each spatial location. The normalization fusion unit is used to perform scale normalization fusion processing on the feature gradient norm and the eigenvalue standard deviation to generate and output the regional heterogeneity modulation field.

3. The multi-query adaptive segmentation network for cross-center medical images according to claim 2, characterized in that, The scale normalization fusion process performed by the normalization fusion unit includes: performing scale normalization operation on the feature gradient norm and the feature value standard deviation, and obtaining the modulation value of the regional heterogeneous modulation field based on the normalized ratio, wherein the scale normalization operation includes a combination of division operation and exponential function operation.

4. The multi-query adaptive segmentation network for cross-center medical images according to claim 1, characterized in that, The cooperative constraint adapter includes a cooperative modulation factor generator, which is used to perform nonlinear remapping on the regional heterogeneous modulation field to generate a cooperative gradient modulation field, and uses the cooperative gradient modulation field as a pixel-wise weight for the boundary loss and the true / false positive loss.

5. The multi-query adaptive segmentation network for cross-center medical images according to claim 4, characterized in that, The nonlinear remapping of the regional heterogeneous modulation field by the cooperative modulation factor generator includes: performing a power function operation on the modulation value of the regional heterogeneous modulation field and multiplying it with a power function operation on the complementary modulation value, and applying a bias processing to the product result to obtain the modulation factor of the cooperative gradient modulation field.

6. The multi-query adaptive segmentation network for cross-center medical images according to claim 5, characterized in that, The collaborative constraint adapter receives the regional heterogeneous modulation field, uses the regional heterogeneous modulation field as modulation weight, and performs pixel-wise weighting on the boundary loss and the true / false positive loss. It then combines the semantic loss to generate the total collaborative constraint loss. Specifically, this includes multiplying the modulation factor of the collaborative gradient modulation field by the sum of the boundary loss and the true / false positive loss at the corresponding pixel position, summing and averaging over all pixels, and then adding this average to the semantic loss.

7. The multi-query adaptive segmentation network for cross-center medical images according to claim 1, characterized in that, The collaborative constraint adapter includes a cascaded mask self-attention unit, a query-to-image attention unit, a feedforward network unit, and an image-to-query attention unit. The mask self-attention unit is used to constrain the interaction between the semantic query vector, the boundary-guided query vector, and the true / false positive query vector to limit information interference between different query targets.

8. The multi-query adaptive segmentation network for cross-center medical images according to claim 7, characterized in that, The reference boundary label, which is used by the collaborative constraint adapter to calculate the boundary loss, is generated by processing the real semantic label through morphological dilation and morphological erosion, and then smoothing the difference between the two.

9. The multi-query adaptive segmentation network for cross-center medical images according to claim 7, characterized in that, The reference true and false positive labels that the collaborative constraint adapter combines with the true and false positive loss are dynamically generated in each training iteration by determining the predicted foreground region based on the current semantic prediction mask and jointly processing the predicted foreground region with the real semantic labels.

10. The multi-query adaptive segmentation network for cross-center medical images according to claim 1, characterized in that, The semantic encoder is a backbone network based on a convolutional neural network architecture, and the medical image is one of magnetic resonance imaging, computed tomography, and ultrasound images.

Images