Heterogeneous feature conflict perception fusion method and device for three-dimensional medical image segmentation

Abstract

This invention relates to a heterogeneous feature conflict-aware fusion method and apparatus for 3D medical image segmentation, belonging to the fields of computer vision and 3D medical image segmentation technology. The method includes: preprocessing and feature extraction of the 3D medical image to be segmented based on a convolutional neural network and attention mechanism to obtain a first source feature representing local detail information and a second source feature representing global context information; aligning the two features for comparability and constructing a conflict metric map representing the degree of conflict; constructing a consistency response map based on the conflict metric map and generating dynamic selective fusion control weights; and performing dynamic selective fusion of the two features according to the control weights and the conflict metric map to obtain fused features and perform segmentation and decoding processing. This invention can explicitly characterize the degree of conflict between heterogeneous features and dynamically adjust the fusion method according to the degree of conflict, reducing the erroneous superposition of conflicting features and improving the accuracy and stability of 3D medical image segmentation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D medical image segmentation technology, and in particular to a heterogeneous feature conflict perception fusion method and apparatus for 3D medical image segmentation. Background Technology

[0002] In deep learning-based automatic analysis and recognition of 3D medical images (such as CT and MRI) segmentation tasks, to obtain segmentation results that are both accurate in detail and conform to the overall anatomical structure, it is usually necessary to utilize two key pieces of information simultaneously: first, local detail information that characterizes tissue boundaries and textures; and second, global contextual information that expresses spatial relationships and long-distance dependencies between organs. To this end, existing solutions generally employ network architectures with multiple branches or paths. For example, convolutional neural networks (CNNs) are used to extract local detail features, while visual Transformers or pyramid pooling branches are used to extract global contextual features, aiming to improve model performance by fusing these two heterogeneous features. Feature fusion is a crucial step in achieving the above goals. Currently, common fusion methods include direct concatenation, element-wise addition, or the introduction of attention mechanisms (such as channel attention and spatial attention) to weight and fuse features. These methods achieve information aggregation to a certain extent. All of the above fusion operations are executed in the computing units of graphics processing units (GPUs) or dedicated image processors, achieving multi-channel feature mapping through tensor operations. However, these fusion strategies based on fixed weights or simple tensor concatenation lack explicit identification mechanisms for semantic conflicts between heterogeneous features, and are essentially a matter of indiscriminate mixing and superposition of local detail features and global context features.

[0003] However, local detail features extracted by convolutional neural networks (CNNs) and global context features extracted by visual Transformers have fundamentally different focuses. The former is sensitive to edges and high-frequency changes, while the latter emphasizes semantic consistency and structural relationships. The activation response maps generated by forward propagation through deep learning models inherently differ in spatial location or feature channels. This difference manifests as complementarity in some regions, but as contradiction or conflict in others. For example, the boundaries of local texture cues may not align with the structural contours inferred from global semantics. Existing fusion strategies, whether simple concatenation, addition, or attention weights generated based on the importance of single-path features, focus on combining or enhancing features. Essentially, this is a form of indiscriminate mixing or weighted superposition of heterogeneous features. When two feature paths fundamentally conflict in their responses in a specific region, this direct mixing operation can lead to the erroneous superposition of conflicting signals, which are then transmitted to subsequent network layers, becoming a source of noise. Specifically, this manifests as follows: in areas with blurred organ boundaries, complex structures, or small target regions, segmentation results are prone to discontinuous, broken, or misjudged boundaries; at the same time, the training process may become unstable due to such unmanaged conflicts, and the model's generalization ability will decrease.

[0004] In summary, there is a pressing technical problem in existing deep learning-based image analysis solutions: during the heterogeneous feature fusion process of 3D medical image segmentation, the computer system lacks an explicit measurement and adaptive suppression mechanism for response conflicts between heterogeneous features of convolutional neural networks and Transformers. This raises the question of how to effectively identify and properly handle response conflicts between features from different sources to avoid erroneous superposition of conflicting features during fusion, thereby improving the accuracy of segmentation results, boundary smoothness, and overall robustness of the model. Summary of the Invention

[0005] Based on this, it is necessary to address the problems of existing solutions, such as the difficulty in explicitly characterizing conflicts during heterogeneous feature fusion and the lack of adaptive adjustment capabilities in the fusion method. This paper proposes a heterogeneous feature conflict perception fusion method and device for 3D medical image segmentation, which can realize the conflict perception and dynamic selective fusion of local detail features and global context features in the 3D medical image to be segmented, thereby improving the accuracy of segmentation results, boundary smoothness and overall robustness of the model.

[0006] This invention provides a heterogeneous feature conflict-aware fusion method for three-dimensional medical image segmentation, the method comprising: After preprocessing the three-dimensional medical images to be segmented, standardized image data is obtained and feature extraction is performed to obtain a first source feature for representing local detail information and a second source feature for representing global context information. The feature extraction is implemented using a convolutional neural network and a visual Transformer. The first source feature and the second source feature are aligned for comparability, and a conflict metric map is constructed to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment. Based on the conflict metric graph, a consistency response graph is constructed, and based on the consistency response graph, dynamic selective fusion control weights are generated. Based on the dynamic selective fusion control weights and the conflict metric map, dynamic selective fusion is performed on the aligned first source features and second source features to obtain fused features. The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and performing conflict suppression on the weighted results based on the conflict metric map. The fused features are input into the segmentation and decoding process to generate the segmentation result of the three-dimensional medical image.

[0007] In one embodiment, the preprocessing includes: resampling the three-dimensional medical image to unify the voxel spacing, truncating the image grayscale values, normalizing the truncated image, and performing patch sampling on the normalized image during the training phase.

[0008] In one embodiment, the statistical distribution alignment is achieved through normalization.

[0009] In one embodiment, constructing a conflict metric map to characterize the degree of conflict between two features in the channel dimension and / or spatial dimension includes: Let the first source feature after comparability alignment be... The second source is characterized as Construct a conflict measurement graph : , In the formula, Represents a normalized mapping. Represents cosine similarity. and These are non-negative weighting coefficients, and .

[0010] In one embodiment, constructing a consistency response graph based on the conflict metric graph includes: Based on the conflict measurement map Construct a consistent response graph And satisfy: .

[0011] In one embodiment, generating dynamic selective fusion control weights based on the consensus response graph includes: For the consensus response graph Perform global average pooling to obtain channel descriptors; The channel descriptor is input into a gating network, which is a gating network composed of multilayer sensing mechanisms; The output of the gated network is processed by the Sigmoid activation function to generate the dynamic selective fusion control weights. .

[0012] In one embodiment, performing dynamic selective fusion includes: Based on the dynamic selective fusion control weights With the conflict metric graph The selective fusion result is obtained through the following formula. : , In the formula, This represents element-wise multiplication. This is the conflict suppression coefficient; The selective fusion result The fused feature is obtained by adding the residuals of the projected features.

[0013] In one embodiment, the method further includes: During the training phase, conflict constraint loss is introduced. To constrain the dynamic selective fusion control weights With the aforementioned consistency response diagram The matching relationship between them; Total loss function satisfy: In the formula, This represents the main loss term corresponding to the segmentation and decoding process. It is a balancing factor.

[0014] The present invention also provides a heterogeneous feature conflict sensing and fusion device for three-dimensional medical image segmentation, the device comprising: The feature extraction module is used to extract features from the standardized image data obtained after preprocessing the three-dimensional medical image to be segmented, and to obtain a first source feature for representing local detail information and a second source feature for representing global context information. The feature extraction is implemented using a convolutional neural network and a visual Transformer. A conflict metric graph construction module is used to perform comparability alignment on the first source feature and the second source feature, and to construct a conflict metric graph to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment. The fusion control weight generation module is used to construct a consistency response graph based on the conflict metric graph, and generate dynamic selective fusion control weights based on the consistency response graph. The feature fusion module is used to perform dynamic selective fusion on the aligned first source features and second source features according to the dynamic selective fusion control weights and the conflict metric map to obtain fused features. The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and performing conflict suppression on the weighted result based on the conflict metric map. The segmentation result output module is used to input the fused features into the segmentation decoding process to generate the segmentation result of the three-dimensional medical image.

[0015] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation as described above.

[0016] The aforementioned heterogeneous feature conflict-aware fusion method and apparatus for 3D medical image segmentation provides a comparable basis for subsequent quantization by aligning heterogeneous features for comparability. By constructing a conflict metric map, it explicitly, rather than implicitly, quantifies and identifies the degree of conflict between the first and second source features in the channel and / or spatial dimensions, thus concretizing the negative factor of "conflict" into a computable feature map. Furthermore, based on the conflict metric map, a consistency response map is constructed, and dynamic selective fusion control weights are generated, enabling the fusion process to be dynamically adjusted according to the degree of conflict. Finally, during dynamic selective fusion, while weighting the two feature paths according to the weights, a conflict suppression operation is specifically introduced based on the conflict metric map. This actively suppresses rather than simply mixes contradictory feature responses in high-conflict regions, effectively avoiding the problem of erroneous superposition of conflicting features. Overall, this constitutes a complete technical solution from conflict identification and quantification to targeted suppression, thereby reducing boundary segmentation discontinuities and structural misclassifications caused by feature conflicts and improving the accuracy and robustness of 3D medical image segmentation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the heterogeneous feature conflict perception and fusion method for three-dimensional medical image segmentation according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the heterogeneous feature extraction and comparability alignment structure according to an embodiment of the present invention; Figure 3 This is a flowchart of a heterogeneous feature conflict perception and fusion method for three-dimensional medical image segmentation according to another embodiment of the present invention. Figure 4 A schematic diagram of a heterogeneous feature conflict sensing and fusion device for three-dimensional medical image segmentation, as an example. Figure 5 This is an internal structural diagram of an electronic device according to one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The following is combined Figures 1-5 This invention describes a heterogeneous feature conflict perception fusion method and apparatus for three-dimensional medical image segmentation.

[0021] like Figure 1 and Figure 2 As shown, in one embodiment, a heterogeneous feature conflict-aware fusion method for three-dimensional medical image segmentation includes the following steps: Step S110: After preprocessing the three-dimensional medical image to be segmented, standardized image data is obtained and feature extraction is performed to obtain a first source feature for representing local detail information and a second source feature for representing global context information.

[0022] Feature extraction is implemented using convolutional neural networks (CNNs) and visual Transformers. Preprocessing aims to eliminate data heterogeneity caused by differences in imaging devices and parameter settings among different samples, providing a unified input standard for subsequent feature extraction. The feature extraction stage employs a GPU-accelerated deep learning network architecture with multiple branches. For example, CNN branches are used to extract local detail features, while branches such as visual Transformers or pyramid pooling are used to extract global contextual features. The CNNs and visual Transformers constitute the encoder part of the deep learning model, used to perform image recognition and feature extraction tasks. The first branch focuses on capturing high-frequency detail information such as local texture and edges, forming the first source features. The second branch focuses on capturing global contextual information such as long-range dependencies and the overall topological structure of organs to form second-source features. Because local detail features and global context features have fundamentally different focuses—the former is sensitive to edges and high-frequency changes, while the latter emphasizes semantic consistency and structural relationships—their activation response patterns in many spatial locations or feature channels inherently differ. This difference manifests as complementarity in some regions, but may be contradictory or conflicting in others, which is precisely the problem that subsequent conflict-aware fusion needs to address.

[0023] Step S120: Perform comparability alignment on the first source feature and the second source feature, and construct a conflict metric map to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment.

[0024] The comparability alignment specifically includes: performing trilinear interpolation upsampling on the first source feature and max pooling or stride convolution downsampling on the second source feature to align the spatial resolution of the two features to the higher resolution scale of either the first or second source feature, achieving scale alignment; applying a 1×1×1 convolution kernel to linearly transform the scale-aligned first and second source features respectively, and unifying the number of channels of the two features to the number of channels of the one with fewer channels, achieving channel alignment; and using instance normalization to process the channel-aligned first and second source features respectively, so that the numerical distributions of both are normalized to a distribution space with a mean of 0 and a variance of 1, achieving statistical distribution alignment. This instance normalization allows its output to contain negative values, thus ensuring that the aligned first source feature... Second source characteristics Numerical comparability and includes positive, negative and zero elements.

[0025] Since two heterogeneous features often differ significantly in spatial resolution, number of channels, and numerical distribution range, direct comparison or calculation can lead to substantial errors. Therefore, we first use interpolation, convolution transformation, and normalization to map the two features to a unified scale, channel dimension, and distribution space, making them numerically comparable. Based on this, we construct a conflict metric map, which is a mapping matrix related to the feature map size. Its value directly reflects the degree of difference between the two features at corresponding spatial locations or channels. If the responses of the two features differ greatly in a certain region (e.g., one has a strong response while the other has a weak response, or they are in opposite directions), the conflict metric map will show a high value in that region, indicating a high risk of conflict. Step S130: Based on the conflict metric graph, construct a consistency response graph, and based on the consistency response graph, generate dynamic selective fusion control weights. The conflict metric map visually shows where conflicts exist, while the consistency response map is the reverse representation of conflict, indicating where complementarity or consistency exists. By calculating the consistency response map, it can be determined which regions are suitable for feature enhancement. Subsequently, a gating mechanism (such as global pooling combined with a fully connected layer) is used to transform the consistency response map into dynamic selective fusion control weights. These weights are adaptive parameter vectors that dynamically determine the contribution ratio of the two features in the final fusion result based on the current consistency state of the features.

[0026] Step S140: Based on the dynamic selective fusion control weights and the conflict metric map, perform dynamic selective fusion on the aligned first source features and second source features to obtain fused features.

[0027] The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and suppressing the conflict based on the conflict metric graph.

[0028] Traditional fusion methods, whether simple concatenation, element-wise addition, or attention weighting based on the importance of single-path features, all focus on how to combine or enhance features. Essentially, they are indiscriminate mixing or weighted superposition of heterogeneous features, ignoring conflicts between features. This embodiment focuses on the aligned first source features. Second source characteristics When performing dynamic selective fusion, a conflict suppression mechanism is introduced. Based on weighted fusion, a conflict metric map is used to penalize the feature product in high-conflict regions. Specifically, when two features have fundamentally conflicting responses in a certain region, direct mixing would lead to the erroneous superposition of conflicting signals, which would then be transmitted to subsequent network layers, becoming a noise source. In this case, the conflict suppression term comes into play, reducing the fusion strength in that region and thus preventing erroneous superposition. Conversely, in regions with high consistency, the fusion weights tend to retain or enhance effective features. This mechanism achieves on-demand fusion, that is, adaptively adjusting the fusion strategy according to the conflict state of the features.

[0029] Step S150: Input the fused features into the segmentation and decoding process to generate the segmentation result of the three-dimensional medical image.

[0030] The features obtained through dynamic selective fusion retain rich local details, incorporate accurate global context, and effectively suppress conflicts and interference between heterogeneous features. Inputting this fused feature into a segmentation decoder (such as a progressive upsampling network) restores the original image resolution and outputs the final segmentation probability map. Due to the significantly improved quality of the input features and the avoidance of erroneous superposition of conflicting features during fusion, the segmentation results demonstrate significant superiority in terms of boundary continuity, smoothness, and accuracy in areas with blurred organ boundaries, complex structures, or small target regions. Simultaneously, the model's generalization ability is also enhanced.

[0031] This embodiment explicitly quantifies the degree of conflict between features from different sources by constructing a conflict metric map, and generates dynamic selective fusion control weights based on the consistency response map. This allows the fusion method to adaptively adjust according to the current feature state, rather than using indiscriminate mixing or weighted superposition of heterogeneous features, thus improving the expression quality of the fused features. By suppressing high-conflict responses and retaining or enhancing low-conflict responses, it avoids the erroneous superposition of conflict features during fusion, significantly improving the segmentation effect of blurred organ boundaries, complex structures, or small target regions, and reducing boundary discontinuities, breaks, or misjudgments. By introducing a conflict constraint loss, a consistency constraint is established between the fusion control weights and the conflict quantification results, which helps improve the stability of the training process and the model's generalization ability. This invention can be embedded as an independent conflict-aware fusion module into existing 3D medical image segmentation workflows, demonstrating good engineering feasibility and application value.

[0032] It should be noted that the key to the technical solution of this invention lies in ensuring the accuracy of the first source features after alignment by employing normalization processes such as instance normalization, which allow for negative output values, during the comparability alignment stage. Second source characteristics Not all values ​​are non-negative. Under this premise, the conflict suppression term in the fusion formula... It can achieve the expected function: when two features fundamentally conflict at a certain position (i.e., the response directions are opposite, one positive and one negative), the element-wise product... The value will be negative. At this point, even if the degree of conflict is represented... If the value is positive, the product is also negative. The negative suppression term, through the subtraction operation in the formula, is equivalent to enhancing or supplementing the feature at that position, thereby suppressing the tendency for features to cancel each other out due to opposite signs (i.e., semantic conflict). When the two features are highly consistent and both are strong positive or negative responses, The values ​​are relatively large positive, but the consistency is high at this point. When the value approaches 0, the suppression term is minimal and has almost no weakening effect on the strong consistency feature. When one of the two features responds extremely weakly or zero... When the product value is close to 0, the suppression term fails, and the fusion process relies on the dynamic selective fusion control weights. Adaptive weighted selection also avoids erroneous feature superposition. This design enables the invention to proactively suppress directional contradictions between features in high-conflict regions, rather than simply mixing them, effectively preventing the erroneous superposition of conflicting features.

[0033] In one embodiment, preprocessing includes: resampling the 3D medical image to unify the voxel spacing, truncating the image grayscale values, normalizing the truncated image, and performing patch sampling on the normalized image during the training phase.

[0034] The resampling step aims to eliminate spatial resolution inconsistencies between different samples caused by differences in imaging device parameter settings. Since CT or MRI images acquired from different hospitals or using different scanning protocols often have differences in interslice spacing and intraslice resolution, directly inputting them into the network can lead to a mismatch between the receptive field of feature extraction and the actual physical size, thus affecting segmentation accuracy. Optionally, a linear interpolation algorithm can be used to resample all input images to a uniform voxel spacing, such as 1.5mm × 1.5mm × 2.0mm. It should be understood that this voxel spacing is only a preferred value given in this embodiment. In practical applications, it can be adjusted according to the size characteristics of the target organ and memory limitations. For example, for small organ segmentation tasks, a smaller voxel spacing (such as 1.0mm × 1.0mm × 1.0mm) can be used to retain more detail information; for large organ segmentation tasks, the voxel spacing can be appropriately increased to reduce computational load. The resampling operation ensures that the subsequent feature extraction network perceives image content at the same physical scale, laying a spatial foundation for the comparability of heterogeneous features.

[0035] Gray-level truncation is used to remove background noise and high-brightness artifacts from images that are irrelevant to the target segmentation task. Medical images typically have a large gray-level dynamic range; for example, the HU value of CT images can range from -1024 to 3071, but target organs (such as the liver and kidneys) are often only distributed within a specific gray-level range. Without truncation, background noise and high-brightness artifacts will occupy the network's learning resources and interfere with the feature extraction process. Optionally, for abdominal CT images, the gray-level truncation range can be set to [-175, 250] HU. This range can effectively cover the gray-level distribution of abdominal soft tissue organs while removing interference from bone, air, and metal artifacts. The gray-level values ​​of the truncated image are limited to a preset range, which is beneficial for subsequent normalization processing.

[0036] Normalization is used to eliminate differences in grayscale distribution among different samples, giving the input data a uniform numerical scale. Due to variations in patient constitution, contrast agent dosage, and scanning time, even images of the same organ may exhibit significant differences in grayscale distribution. Normalization maps truncated image data to a uniform distribution space, thereby improving the model's generalization ability across different samples. Optionally, Z-score normalization can be used, which involves subtracting the mean from the image data and dividing by the standard deviation to ensure the data follows a standard normal distribution with a mean of 0 and a standard deviation of 1. Alternatively, min-max normalization can be used to linearly map the data to the interval [0, 1] or [-1, 1]. Normalization not only accelerates the network's training convergence process but, more importantly, aligns statistical distributions, reducing differences in numerical distribution among different samples and providing a numerically comparable basis for subsequent heterogeneous feature extraction and conflict measurement.

[0037] Patch sampling is primarily used in the training phase to reduce GPU memory overhead and increase sample diversity. 3D medical images typically have large datasets, and directly inputting the entire image into the network for training is often limited by GPU memory capacity. Patch sampling allows for the random cropping of fixed-size sub-regions from the original 3D image for training. In one implementation, the patch size can be set to 96×96×96 voxels. To address the foreground-background imbalance problem, a foreground-guided sampling strategy can be employed, where the sampling center is positioned within the target organ region with a preset probability (e.g., 0.7), ensuring that each patch contains sufficient foreground information, thereby improving the model's ability to learn features from the target region. During the inference phase, a sliding window strategy can be used for dense prediction of the entire image, and boundary effects can be eliminated through weighted averaging of overlapping regions, ultimately obtaining a complete segmentation result.

[0038] The preprocessing steps in this embodiment are interconnected, forming a standardized data processing flow. Resampling unifies the spatial scale, grayscale truncation removes irrelevant interference, normalization aligns the numerical distribution, and patch sampling optimizes training efficiency. The standardized image data after preprocessing eliminates the heterogeneity caused by non-target factors, allowing the subsequent feature extraction network to focus on extracting effective features relevant to the segmentation task, providing high-quality data input for heterogeneous feature conflict perception fusion.

[0039] In one embodiment, a conflict metric map is constructed to characterize the degree of conflict between two features in the channel dimension and / or spatial dimension, including: Let the first source feature after comparability alignment be... The second source is characterized as Construct a conflict measurement graph : , In the formula, Represents a normalized mapping. Represents cosine similarity. and These are non-negative weighting coefficients, and .

[0040] The design logic of this formula lies in characterizing feature conflict from two orthogonal dimensions: differences in response amplitude and differences in directional consistency. Among these, The absolute difference between the two features at their corresponding spatial locations and channels was calculated, reflecting the degree of difference in feature intensity. If the first source feature response is strong while the second source feature response is weak in a certain region, the difference is large, indicating a conflict at the amplitude level. The operation maps the difference to the [0, 1] interval, eliminating the influence of dimensions and ensuring numerical stability. The term measures conflict from the perspective of vector direction. Cosine similarity measures the angle between the directions of two feature vectors. If the directions are the same (angle is 0), the similarity is 1, and the conflict term is 0; if the directions are opposite or orthogonal, the similarity decreases, and the conflict term increases. This can capture semantic conflicts that amplitude differences cannot express, such as when two feature vectors have similar amplitudes but opposite directions. Parameters and Contribution weights used to balance magnitude and direction conflicts. Preferably, and It can be set to 0.5, meaning both types of conflicts are treated equally; in another implementation sensitive to boundary details, it can be appropriately increased. The weights are adjusted to focus more on abrupt changes in local response strength. It should be understood that... and The specific value can be adaptively adjusted according to the tolerance for false positives or false negatives based on the specific segmentation task; this embodiment does not impose any restrictions on this. Through the above weighted combination, the conflict metric graph... It can generate a mapping matrix with the same size as the feature map space, where high-value regions correspond to high conflict risk and low-value regions correspond to high consistency, thereby achieving explicit and fine-grained quantification of the degree of conflict of heterogeneous features.

[0041] Furthermore, based on the conflict metric graph, a consistency response graph is constructed, including: based on the conflict metric graph... Construct a consistent response graph And satisfy: .

[0042] This process is a reverse mapping of the conflict metric map. If the conflict metric value of a certain area... A higher value indicates a significant difference between the two feature streams, and direct fusion may introduce noise. In this case, the consistency response value... A lower value indicates that the fusion intensity in that area should be reduced or suppressed; conversely, if... A lower value indicates that the two features are complementary or consistent. A higher value indicates that the region is suitable for feature enhancement or preservation. The construction of the consistency response map provides an intuitive reliability reference for the subsequent generation of fusion control weights, transforming the abstract degree of conflict into a concrete numerical indicator.

[0043] Based on the consensus response graph, dynamic selective fusion control weights are generated, including the following steps: Step S310: Perform global average pooling on the consistency response graph to obtain the channel descriptor.

[0044] Global average pooling compresses spatial dimensional information by calculating the average of all spatial locations within each channel, resulting in a vector with the same dimension as the number of channels. This vector is the channel descriptor, which summarizes the average consistency level of each channel across the entire space. Through this operation, the network can assess from a global perspective which channels have high feature consistency and which channels have global conflicts, thus providing a basis for subsequent channel-level weight allocation.

[0045] Step S320: Input the channel descriptor into the gating network, which is a gating network composed of multilayer sensing mechanisms.

[0046] Multilayer perceptrons (MLPs) possess powerful nonlinear fitting capabilities. In one specific implementation, this gated network can contain two fully connected layers with a ReLU activation function in between. The first fully connected layer maps the channel descriptors to a lower dimension (e.g., 1 / 4 of the number of channels), performing feature compression and nonlinear transformation; the second fully connected layer restores them to the original channel dimension, outputting the original weight values ​​for each channel. This "compression-restoration" structure effectively captures the dependencies between channels and learns the differences in the contributions of different channels to the final segmentation task.

[0047] Step S330: The output of the gated network is processed by the Sigmoid activation function to generate dynamic selective fusion control weights.

[0048] The Sigmoid function maps the real values ​​of the MLP output to the (0, 1) interval, resulting in the final normalized weight vector. Weight Each element in the algorithm corresponds to a feature channel, and the numerical value represents the degree to which the features of that channel are preserved during the fusion process. A value close to 1 indicates high consistency of the channel, which should be preserved or enhanced; a value close to 0 indicates severe conflict in the channel, which should be suppressed. Through the above "global pooling-MLP-Sigmoid" process, the system transforms the conflict metric information calculated in the previous steps into control parameters that can be directly used for feature weighting, realizing a logical closed loop from conflict perception to fusion control. This data-driven weight generation method, compared with traditional fixed weights or simple stitching, can dynamically adjust the fusion strategy according to the specific feature distribution of the input image, thereby improving the robustness and segmentation accuracy of the model under different data distributions.

[0049] In one embodiment, performing dynamic selective fusion includes: based on dynamic selective fusion control weights. With conflict measurement map The selective fusion result is obtained through the following formula. : , In the formula, This represents element-wise multiplication. This is the conflict suppression coefficient.

[0050] Specifically, the design logic of this formula lies in decomposing the feature fusion process into two parallel mechanisms: weighted preservation and conflict suppression. The first two terms on the right-hand side of the formula constitute the weighted preservation mechanism: This indicates the use of dynamic selective fusion control weights. The first source features are weighted, and the larger the weight value, the higher the degree of retention of that feature in the fusion result; This indicates the complementary preservation of features from the second source. Because... The value of is restricted to after being processed by the Sigmoid activation function. Within the interval, and and The sum is 1, which ensures that the contributions of the two features in the weighted retention process are complementary. When the consistency response of a certain channel or spatial location is high, When the consistency value approaches 1, the fusion result mainly retains the features of the first source; conversely, if the consistency is low, When the weighted average approaches zero, the fusion result primarily retains the features from the second source. This adaptive weighting mechanism avoids the problem of poor adaptability to different feature distributions in traditional fixed-weight fusion methods.

[0051] The third term of the formula This constitutes a conflict suppression mechanism. Among them, This represents the element-wise product of two feature paths. The product term has a high value when the response directions of the two features are consistent and their magnitudes are large, and a low value when the response directions are opposite or one feature's response is weak. However, the product term alone cannot distinguish between the two distinct situations of high-consistency complementarity and high-conflict superposition. Therefore, this embodiment introduces a conflict metric graph. As an inhibitory factor. When A high value indicates a significant feature conflict at that location, in which case the suppression term... Increasing the absolute value reduces conflicting components in the fusion result, effectively preventing noise amplification or feature cancellation caused by erroneous superposition of heterogeneous features. Parameters This is the conflict suppression coefficient, used to control the suppression strength. Preferably, It can be set to a value between 0.1 and 0.5 to balance the degree of feature preservation and conflict suppression; in another implementation for high-noise data, These parameters can be set as learnable parameters, and the optimal value can be determined adaptively through network training. It should be understood that... The specific value or setting method can be adjusted according to the accuracy requirements and data quality of the actual segmentation task, and this embodiment does not impose any restrictions on this.

[0052] Furthermore, the selective fusion results The residuals from the projection features are added to obtain the fused features. The projection feature generation method is as follows: the 3D medical image to be segmented is processed through an independent shallow embedding network consisting of two consecutive 3×3×3 convolutional layers and a ReLU activation function to generate a shallow embedding feature for the original image. Subsequently, a 1×1×1 convolutional kernel is used to transform this shallow embedding feature to the selective fusion result. With the same spatial resolution and number of channels, the transformed features are called projected features. Residual addition is a common feature optimization technique in deep learning. In this embodiment, projected features can be understood as representations obtained by linearly mapping the original input features or intermediate layer features, with their dimensionality and selective fusion results... Maintain consistency. By Adding the projected features to the fused features not only preserves the conflict suppression benefits of dynamic selective fusion but also introduces gradient direct paths through residual connections, effectively mitigating the vanishing gradient problem in deep network training and improving model optimization stability. Simultaneously, the residual addition operation makes the fused features more smoothly distributed in numerical terms, which is beneficial for feature recovery and detail reconstruction in subsequent segmentation decoders. This triple mechanism of weighted preservation, conflict suppression, and residual stabilization effectively suppresses high-conflict responses and fully preserves low-conflict responses, enhancing the expressive power of fused features in complex boundary regions and small target regions.

[0053] In one embodiment, the invention further includes: introducing a conflict constraint loss during the training phase. To constrain the dynamic selective fusion control weights With the aforementioned consistency response diagram The matching relationship between them; the total loss function satisfy: In the formula, This represents the main loss term corresponding to the segmentation and decoding process. It is a balancing factor.

[0054] During the training process of deep neural networks, the main loss function relies solely on the segmentation results. Supervision often makes it difficult to guarantee the dynamic selective fusion control weights generated by the intermediate layer. It can accurately reflect the true consistency relationship between features. If If the allocation is biased, such as assigning too high a retention weight to a high-conflict region or too low a weight to a high-consistency region, it will directly lead to a decrease in the quality of the fused features, thereby affecting the segmentation accuracy. Therefore, this embodiment introduces a conflict constraint loss. As an auxiliary monitoring signal, it explicitly constrains the optimization direction of the weight generation network.

[0055] Conflict constraint loss The core logic lies in establishing dynamic selective fusion control weights. With Consistency Response Diagram The mapping constraints between them. Specifically, conflict constraint loss. It can be represented as: .in, This indicates a global average pooling operation. This represents the L2 norm. The physical meaning of this formula lies in: the uniformity response diagram. This reflects the true consistency distribution of features across the spatial dimension. Through global average pooling, the average consistency level of each channel in the overall space can be obtained, which forms a reference benchmark for the ideal weight of that channel. Dynamic selective fusion controls the weights. These are predictions generated by the gating network. Conflict constraint loss. By calculating the prediction weights The L2 distance between the network and the ideal baseline forces the network to continuously adjust its parameters during training, resulting in weights that... To approximate the global statistical properties of the consistent response graph as closely as possible. When When the convergence reaches a smaller value, it means that the network has learned to allocate fusion weights according to the actual consistency of features, thereby achieving the expected goal of high retention in high consistency regions and low retention in high conflict regions.

[0056] Main loss item Used to supervise the final segmentation output. Optional, A weighted combination of Dice loss and cross-entropy loss can be used to simultaneously optimize region overlap and pixel-level classification accuracy. (Balance factor) Used to balance the contribution ratios of the main loss term and the constraint loss term. If... If the value is set too low, the conflict constraint effect is weak, and the network may still tend to generate suboptimal weights; if... Setting excessively large or overly strong constraints may limit the network's expressive power, leading to underfitting of the model. In a preferred embodiment, It can be set to a value between 0.1 and 1.0, and the specific value can be fine-tuned based on the segmentation performance on the validation set. It should be understood that... It can also be set as a learnable parameter, which will adaptively adjust as the network trains.

[0057] By introducing a total loss function that includes conflict constraint loss, this embodiment provides explicit supervision signals for the dynamic selective fusion mechanism during the training phase. This design not only accelerates the network's convergence process and improves training stability, but more importantly, it ensures that the dynamic selective fusion control weights generated during the inference phase have clear physical meaning, that is, they truly reflect the degree of consistency between heterogeneous features. This provides a reliable weight basis for subsequent dynamic selective fusion steps, further improving the accuracy and robustness of 3D medical image segmentation.

[0058] In one embodiment, during dynamic selective fusion, feature product Before performing conflict suppression, dynamic sparsification is applied to filter out low-conflict-correlation noise; projected features are generated through cross-scale feature aggregation, including selective fusion results. Adaptive weighted aggregation is performed with encoder features at multiple different scales to generate projected features for residual summation.

[0059] Specifically, in calculating the conflict suppression term When, characteristic product Features often contain a large amount of low-correlation noise with small amplitudes but wide distribution. Although the amplitude of these noises is low at individual points, their cumulative effect in space may interfere with the purity of the fused features. To address this, this embodiment introduces a dynamic sparsity processing mechanism to clean the feature product before performing the suppression operation.

[0060] The specific process of dynamic sparsity processing includes: First, calculating the feature product. The element-wise magnitude values; secondly, based on the conflict metric graph. Adaptive determination of sparse threshold based on statistical properties Finally, a threshold function is applied to sparsify the feature product.

[0061] Preferably, a soft threshold function is applied. Sparsification of feature products, where the sparse threshold is used. With conflict measurement map The value is proportional to the spatial mean. The core idea of ​​the soft threshold function is to reduce the amplitude below the threshold. Elements above the threshold are set to zero, while elements above the threshold are subtracted from the threshold and their signs are retained. This processing method can smoothly remove noise and avoid the gradient discontinuity problem caused by hard truncation. In particular, the sparse threshold in this embodiment... It is not a fixed constant, but rather adaptively determined based on the conflict state of the current features. Specifically, the conflict metric map is calculated. The average conflict level in the feature space is the spatial mean, reflecting the overall average conflict level. A high average conflict level indicates large differences between features and potentially a high amount of low-correlation noise. In this case, the threshold should be adaptively increased. The sparsity threshold is increased when the sparsity is increased, and vice versa, the threshold is decreased to retain more details. This adaptive mechanism ensures that the sparsity operation can be dynamically adjusted according to the actual data characteristics, effectively filtering out noise while avoiding information loss caused by excessive sparsity.

[0062] Alternatively, a hard threshold function can be applied. Furthermore, gradient backpropagation is maintained through a pass-through estimator. While a hard threshold function directly sets elements below the threshold to zero, which has a clear physical meaning, the gradient becomes zero during backpropagation, leading to training difficulties. Therefore, this embodiment introduces a pass-through estimator, performing hard thresholding during forward propagation and approximating a constant gradient of 1 during backpropagation. This bypasses the non-differentiable thresholding operation, ensuring the network can train normally. It should be understood that the choice between soft and hard thresholds depends on the specific task's requirements for feature continuity and sparsity; this embodiment does not impose any restrictions on this.

[0063] Furthermore, regarding the generation of projection features, this embodiment employs a cross-scale feature aggregation strategy. Cross-scale feature aggregation includes: First, a cross-scale feature pyramid is constructed, which includes high-resolution detail features extracted from the shallow encoder, medium-resolution semantic features extracted from the mid-level encoder, and low-resolution contextual features extracted from the deep encoder.

[0064] In 3D medical image segmentation, features at different scales carry information at different levels: shallow features are rich in geometric details such as edges and textures, but lack semantic understanding; deep features have rich semantic information and a global receptive field, but have low spatial resolution and blurred boundaries. By constructing a feature pyramid, this embodiment can comprehensively utilize the advantages of each level to compensate for the shortcomings of single-scale features.

[0065] Secondly, after obtaining multi-scale features, the contribution weights of each scale feature are calculated through a learnable scale selection gating network. The input of the scale selection gating network includes the depth index of the current decoder stage and the similarity measure between the aligned first source feature and the second source feature, so as to dynamically enhance the encoder feature contribution that matches the current segmentation target scale.

[0066] Specifically, the scale-selection gating network is a lightweight sub-network whose input includes not only the depth information of the current decoder layer but also the consistency state of the current features (i.e., similarity metric). The depth index tells the network which stage of decoding it is currently in, thus inferring the resolution level that needs to be recovered; the similarity metric reflects the fusion quality of the current features, helping the network determine which scale of encoder features to rely on more. For example, in the early stages of decoding (deep stages), the network may prefer to utilize deep contextual features; while in the later stages of decoding (shallow stages), it automatically increases its reliance on shallow detail features. The gating network outputs a set of weight coefficients, which are used to weight and sum the features at each scale, thereby achieving adaptive multi-scale feature fusion.

[0067] Finally, the weighted multi-scale features are processed through... Convolutional dimensionality reduction to selective fusion results Using the same dimension as the projection feature, the dimensionality reduction operation ensures the consistency of the projection feature and the selective fusion result in the channel dimension, enabling the two to successfully perform residual addition.

[0068] The projected features generated through the aforementioned cross-scale feature aggregation not only contain rich multi-scale information but also undergo adaptive filtering to remove redundant features irrelevant to the current segmentation target. Adding these features to the residuals of the selective fusion results further enhances the expressive power of the fused features, ensuring that the final input features to the segmentation decoder achieve optimal performance in detail restoration, semantic understanding, and anti-interference capabilities, thereby significantly improving the accuracy and robustness of 3D medical image segmentation.

[0069] To verify the feasibility and effectiveness of the technical solution of this invention in actual medical image analysis, this embodiment takes abdominal CT image segmentation as an example to illustrate in detail the complete process from input image to output segmentation result. It should be understood that abdominal CT images are only an exemplary application scenario; this invention is also applicable to other three-dimensional medical image segmentation tasks such as brain MRI and chest CT, and this embodiment does not constitute a limitation on the scope of protection of this invention.

[0070] In step S110, abdominal CT image data to be segmented is acquired. Since CT images acquired by different medical institutions or scanning equipment vary significantly in slice spacing, intra-slice resolution, and grayscale distribution, directly inputting them into the network will lead to inconsistent feature extraction. Therefore, the original images are first resampled. In one specific implementation, a linear interpolation algorithm is used to resample all input images to a uniform voxel spacing, for example, 1.5mm × 1.5mm × 2.0mm. This voxel spacing setting comprehensively considers the average size of abdominal organs (such as the liver, kidneys, and spleen) and computational resource consumption, preserving sufficient detail while avoiding excessive data volume leading to memory overflow. For segmentation tasks involving smaller organs or lesions, the voxel spacing can be appropriately reduced to 1.0mm × 1.0mm × 1.0mm to capture finer boundary features; for segmentation tasks involving large organs, the voxel spacing can be appropriately increased to 2.0mm × 2.0mm × 3.0mm to improve processing efficiency.

[0071] Subsequently, the resampled images underwent grayscale truncation. The original grayscale value range of CT images is typically -1024 to 3071 HU, but the CT values ​​of abdominal soft tissue organs are mainly concentrated between -100 and 300 HU. To remove interference from irrelevant information such as bone, air, fat, and metal artifacts, the grayscale truncation range was set to [-175, 250] HU. This range effectively covers the grayscale distribution range of the main abdominal organs while eliminating extremely bright bones and extremely low-density air background. The truncated image grayscale values ​​are limited to the preset range, which is beneficial for subsequent normalization processing.

[0072] Next, the truncated images were normalized. Z-score normalization was used, subtracting the mean from the image data and dividing by the standard deviation to ensure the data followed a standard normal distribution with a mean of 0 and a standard deviation of 1. Normalization not only accelerated the network's training and convergence process, but more importantly, it aligned the statistical distributions, reducing the numerical distribution differences between different samples caused by variations in contrast agent dosage and scanning time. This provided a numerically comparable basis for subsequent heterogeneous feature extraction and conflict measurement.

[0073] During the training phase, patch sampling is performed on the normalized images. Due to the massive amount of data in 3D abdominal CT images, direct input into the network is limited by GPU memory capacity. Therefore, a sliding window strategy is adopted, randomly cropping fixed-size sub-regions from the original 3D images for training. In one specific implementation, the patch size is set to 96×96×96 voxels. To address the foreground-background imbalance problem, a foreground-guided sampling strategy is employed, where the sampling center falls within the target organ region with a preset probability (e.g., 0.7), ensuring that each patch contains sufficient foreground information, thereby improving the model's feature learning ability for the target region. During the inference phase, a sliding window strategy is used to perform dense predictions on the entire image, and boundary effects are eliminated through weighted averaging of overlapping regions, ultimately obtaining a complete segmentation result.

[0074] In step S120, feature extraction is performed on the standardized image data. This embodiment employs a dual-branch architecture for heterogeneous feature extraction. The first branch uses a three-dimensional convolutional neural network (such as the encoder part of 3D UNet) to extract high-frequency detail information such as local texture and edges through multi-layer convolution and pooling operations, forming the first source feature. This branch has a small receptive field, enabling it to accurately capture organ boundaries and internal texture changes. The second branch uses a hierarchical visual Transformer structure, generating token representations through patch embedding, and combining spatial downsampling attention mechanisms and positional encoding mechanisms to perform global feature modeling, extracting global contextual information such as long-distance dependencies and the overall topological structure of organs, forming the second source feature. This branch has a global receptive field, enabling it to understand the spatial positional relationships and overall semantic structure between organs. The two heterogeneous features differ significantly in their expression patterns, which is precisely the problem that subsequent conflict perception fusion needs to address.

[0075] In step S130, the first source feature and the second source feature are aligned for comparability, and a conflict metric map is constructed. Since the two features differ in spatial resolution, channel dimension, and numerical distribution, scale alignment, channel alignment, and statistical distribution alignment are first performed through interpolation, convolution transformation, and normalization operations. Subsequently, according to the formula... Constructing a conflict metric map. In abdominal CT image segmentation scenarios, feature conflicts often exist at organ boundary regions. For example, at the junction of the liver and kidney, local feature branches may produce strong responses to boundary textures, while global feature branches may determine whether the region belongs to the liver or kidney based on overall semantics, leading to inconsistencies between the two feature responses. In this case, the conflict metric map shows a high value in this region, indicating a high risk of conflict. Conversely, in homogeneous regions within the organ, the two feature responses are often consistent, and the conflict metric map shows a low value.

[0076] In step S140, a consistency response graph is constructed based on the conflict metric graph, and dynamic selective fusion control weights are generated. (Consistency Response Graph) This visually demonstrates the degree of feature consistency. Through global average pooling and a gating network, the consistent response graph is transformed into a dynamically selective fusion control weight. In the organ boundary region, due to the high conflict metric and low consistency response, the generated weights... The tendency is to reduce fusion strength to avoid erroneous aggregation; however, within organ regions, the consistency response value is high, and the weighting is high. It tends to retain or enhance effective features. Then, according to the formula... Perform dynamic selective fusion. This includes a conflict suppression term. Playing a crucial role in the boundary region, it effectively suppresses noise amplification or feature cancellation caused by erroneous superposition of heterogeneous features, thereby significantly improving the continuity and accuracy of the segmentation boundary. Finally, the selective fusion result is added to the projected features generated through cross-scale feature aggregation using residuals to obtain the final fused feature.

[0077] In step S150, the fused features are input into the segmentation decoder to generate the segmentation result of the 3D medical image. The segmentation decoder adopts a stepwise upsampling structure combined with skip connections to restore the deep fused features to the original image resolution and outputs the final segmentation probability map. Because the input features undergo conflict-aware fusion processing, the conflict interference between heterogeneous features is effectively suppressed, and the segmentation result shows significant advantages in boundary continuity, small target recognition rate, and overall structural integrity. For example, when segmenting small structures such as liver blood vessels, local feature branches can provide accurate boundary details, while global feature branches can provide contextual information about the direction of blood vessels. The conflict-aware fusion mechanism can effectively avoid the conflict between the two at the edge of the blood vessel, thereby achieving more accurate blood vessel segmentation.

[0078] The heterogeneous feature conflict perception and fusion device for three-dimensional medical image segmentation provided by the present invention is described below. The heterogeneous feature conflict perception and fusion device for three-dimensional medical image segmentation described below can be referred to in correspondence with the heterogeneous feature conflict perception and fusion method for three-dimensional medical image segmentation described above.

[0079] like Figure 4 As shown, in one embodiment, a heterogeneous feature conflict perception and fusion device for three-dimensional medical image segmentation includes a feature extraction module 410, a conflict metric map construction module 420, a fusion control weight generation module 430, a feature fusion module 440, and a segmentation result output module 450.

[0080] The feature extraction module 410 is used to extract features from the standardized image data obtained after preprocessing the three-dimensional medical image to be segmented, to obtain a first source feature for representing local detail information and a second source feature for representing global context information. The feature extraction is implemented using a convolutional neural network and a visual Transformer. The conflict metric graph construction module 420 is used to perform comparability alignment between the first source feature and the second source feature, and to construct a conflict metric graph to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment. The fusion control weight generation module 430 is used to construct a consistency response graph based on the conflict metric graph, and generate dynamic selective fusion control weights based on the consistency response graph. The feature fusion module 440 is used to perform dynamic selective fusion on the aligned first source features and second source features according to the dynamic selective fusion control weights and the conflict metric map to obtain fused features. The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and suppressing the conflict based on the conflict metric map. The segmentation result output module 450 is used to input the fused features into the segmentation decoding process to generate the segmentation result of the three-dimensional medical image.

[0081] Figure 5 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 5 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the heterogeneous feature conflict-aware fusion method for three-dimensional medical image segmentation according to any of the above embodiments.

[0082] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0083] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation of any of the above embodiments.

[0084] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements the heterogeneous feature conflict-aware fusion method for three-dimensional medical image segmentation according to any of the above embodiments.

[0085] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0086] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0087] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A heterogeneous feature conflict-aware fusion method for three-dimensional medical image segmentation, characterized in that, The method includes: After preprocessing the three-dimensional medical images to be segmented, standardized image data is obtained and feature extraction is performed to obtain a first source feature for representing local detail information and a second source feature for representing global context information. The feature extraction is implemented using a convolutional neural network and a visual Transformer. The first source feature and the second source feature are aligned for comparability, and a conflict metric map is constructed to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment. Based on the conflict metric graph, a consistency response graph is constructed, and based on the consistency response graph, dynamic selective fusion control weights are generated. Based on the dynamic selective fusion control weights and the conflict metric map, dynamic selective fusion is performed on the aligned first source features and second source features to obtain fused features. The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and performing conflict suppression on the weighted results based on the conflict metric map. The fused features are input into the segmentation and decoding process to generate the segmentation result of the three-dimensional medical image; The construction of a conflict metric map to characterize the degree of conflict between two features in the channel dimension and / or spatial dimension includes: Let the first source feature after the comparability alignment is completed be , and the second source feature be , a conflict measure map is constructed : ， wherein denotes a normalized mapping, denotes a cosine similarity, and are non-negative weight coefficients, and ; The construction of a consistency response graph based on the conflict metric graph includes: According to the conflict measure map , a consistent response map is constructed , and satisfies: ; The generation of dynamic selective fusion control weights based on the consensus response graph includes: to the consistency response graph performing global average pooling to obtain a channel descriptor; The channel descriptor is input into a gating network, which is a gating network composed of multilayer sensing mechanisms; processing the output of the gating network by a Sigmoid activation function to generate the dynamic selection fusion control weight ; The execution of dynamic selective fusion includes: in accordance with the dynamic selective fusion control weight with the conflict metric map , a selective fusion result is obtained by : ， wherein denotes an element-wise multiplication, is a collision suppression coefficient; The selective fusion result is obtained The residual is added to the projection feature to obtain the fusion feature.

2. The heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation according to claim 1, characterized in that, The preprocessing includes: resampling the three-dimensional medical image to unify the voxel spacing, truncating the image grayscale values, normalizing the truncated image, and performing patch sampling on the normalized image during the training phase.

3. The heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation according to claim 1, characterized in that, The statistical distribution alignment is achieved through normalization.

4. The heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation according to claim 1, characterized in that, The method further includes: During the training phase, conflict constraint loss is introduced. To constrain the dynamic selective fusion control weights With the aforementioned consistency response diagram The matching relationship between them; Total loss function satisfy: In the formula, This represents the main loss term corresponding to the segmentation and decoding process. It is a balancing factor.

5. A heterogeneous feature conflict sensing and fusion device for three-dimensional medical image segmentation, characterized in that, The device includes: The feature extraction module is used to extract features from the standardized image data obtained after preprocessing the three-dimensional medical image to be segmented, and to obtain a first source feature for representing local detail information and a second source feature for representing global context information. The feature extraction is implemented using a convolutional neural network and a visual Transformer. A conflict metric graph construction module is used to perform comparability alignment on the first source feature and the second source feature, and to construct a conflict metric graph to characterize the degree of conflict between the two features in the channel dimension and / or spatial dimension. The comparability alignment includes scale alignment, channel alignment and statistical distribution alignment. The fusion control weight generation module is used to construct a consistency response graph based on the conflict metric graph, and generate dynamic selective fusion control weights based on the consistency response graph. The feature fusion module is used to perform dynamic selective fusion on the aligned first source features and second source features according to the dynamic selective fusion control weights and the conflict metric map to obtain fused features. The dynamic selective fusion includes: weighting the aligned first source features and second source features based on the dynamic selective fusion control weights, and performing conflict suppression on the weighted result based on the conflict metric map. The segmentation result output module is used to input the fused features into the segmentation decoding process to generate the segmentation result of the three-dimensional medical image; The construction of a conflict metric map to characterize the degree of conflict between two features in the channel dimension and / or spatial dimension includes: Let the first source feature after comparability alignment be... The second source is characterized as Construct a conflict measurement graph : ， In the formula, Represents a normalized mapping. Represents cosine similarity. and These are non-negative weighting coefficients, and ; The construction of a consistency response graph based on the conflict metric graph includes: Based on the conflict measurement map Construct a consistent response graph And satisfy: ; The generation of dynamic selective fusion control weights based on the consensus response graph includes: For the consensus response graph Perform global average pooling to obtain channel descriptors; The channel descriptor is input into a gating network, which is a gating network composed of multilayer sensing mechanisms; The output of the gated network is processed by the Sigmoid activation function to generate the dynamic selective fusion control weights. ; The execution of dynamic selective fusion includes: Based on the dynamic selective fusion control weights With the conflict metric graph The selective fusion result is obtained through the following formula. : ， In the formula, This represents element-wise multiplication. This is the conflict suppression coefficient; The selective fusion result The fused feature is obtained by adding the residuals of the projected features.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the heterogeneous feature conflict perception fusion method for three-dimensional medical image segmentation as described in any one of claims 1 to 4.

Images