A medical image lesion fine segmentation method based on multi-modal feature alignment

Abstract

The application provides a medical image lesion fine segmentation method based on multi-modal feature alignment, and relates to the technical field of medical image processing. The application first acquires multi-modal medical images corresponding to the same target lesion and generates a modal feature map, generates an initial segmentation result according to the modal feature map and determines a lesion boundary neighborhood; determines local feature similarity, boundary gradient consistency and residual spatial offset in the lesion boundary neighborhood, generates boundary neighborhood alignment weights and performs local weighted fusion to obtain a boundary enhanced fusion feature map to correct the lesion boundary. The method can reduce the interference of local residual misplacement on boundary correction and improve the boundary stability of the fine segmentation result.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for fine segmentation of medical image lesions based on multimodal feature alignment. Background Technology

[0002] Medical image lesion segmentation is used to determine the spatial extent of lesion regions from medical images and is a fundamental step in image-assisted analysis. Different medical image modalities reflect different tissue information; for example, structural images are better suited to displaying anatomical boundaries, while functional images are better suited to reflecting metabolic or perfusion states. Single-modal images are prone to boundary recognition instability when lesion boundaries are unclear, adjacent tissues have similar gray levels, or the internal composition of the lesion is complex. Yang Hongjie et al., in their review of "Research on Multimodal Medical Image Segmentation Based on Deep Learning," documented that multimodal medical image segmentation can utilize complementary information from different modalities to provide information about the lesion region and its surrounding area from multiple levels, thereby improving the accuracy of clinical diagnosis. This type of research indicates that multimodal medical image segmentation has become a common research direction in medical image analysis.

[0003] Existing multimodal medical image segmentation methods typically involve first spatially registering or resampling images from different modalities, then extracting and fusing features from the registered images, and finally outputting lesion segmentation results based on the fused features. Patent document CN111612754A discloses an optimized MRI tumor segmentation method based on multimodal image fusion. This method achieves tumor segmentation by constructing an MRI tumor multimodal image fusion network, a multimodal 3D network, and relevant loss functions, demonstrating that multimodal fusion networks have been used for tumor region segmentation. Zhang Xinyi et al., in their paper "Non-rigid Registration and Multi-resolution Fusion Methods for Multimodal Medical Images," describe how medical image registration seeks transformation relationships for floating images to achieve spatial matching with fixed images; medical image fusion is not a simple superposition of images but requires processing redundant information between different images while preserving complementary information. The aforementioned research and patent documents indicate that registration, feature fusion, and segmentation output constitute common processing methods in multimodal medical image segmentation.

[0004] However, in actual medical image acquisition, different modalities of images may be affected by scanning position, respiratory motion, slice thickness, imaging time difference, and soft tissue deformation. Zhang Xinyi et al. also pointed out that the internal structure of human tissues is complex and time-varying; for example, organs in lung scans may shift with respiration, making rigid body registration difficult to meet complex registration requirements. Chang Qing et al., in "Multimodal Medical Image Registration Based on Multi-level Feature Fusion," also pointed out that multimodal medical images have significant differences in grayscale and texture structure, making it difficult to extract corresponding features and easily affecting registration accuracy. Therefore, even after spatial registration, multimodal medical images may still have local residual misalignments in the vicinity of lesion boundaries. Existing methods often perform overall fusion of registered multimodal features, lacking local alignment constraints for the vicinity of lesion boundaries. This can lead to mutual interference between different modal features near the boundary during fusion, thus affecting the boundary stability of fine lesion segmentation results. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method for fine segmentation of medical image lesions based on multimodal feature alignment. Addressing the issue that local residual misalignment may still exist in the neighborhood of lesion boundaries after spatial processing of multimodal medical images, this invention establishes boundary neighborhood alignment weights constrained by local feature similarity, boundary gradient consistency, and residual spatial offset within the lesion boundary neighborhood. This allows different modal feature maps to participate in fusion at the lesion boundary according to local alignment relationships, reducing the interference of local misalignment features on lesion boundary correction and improving the stability of fine segmentation results in the lesion edge region.

[0006] To achieve the above objectives, the present invention provides the following solution: A method for fine segmentation of medical image lesions based on multimodal feature alignment, comprising: Acquire multimodal medical images corresponding to the same target lesion, and generate modal feature maps corresponding to each modal medical image based on the multimodal medical images; The initial segmentation result of the target lesion is generated based on the modal feature map, and the lesion boundary neighborhood is determined based on the initial segmentation result; Within the neighborhood of the lesion boundary, the local feature regions to be aligned in different modal feature maps are used as alignment objects to determine the local feature similarity, boundary gradient consistency and residual spatial offset of the neighborhood of the lesion boundary. Based on local feature similarity, boundary gradient consistency, and residual space offset, generate boundary neighborhood alignment weights corresponding to each position within the lesion boundary neighborhood; By utilizing boundary neighborhood alignment weights, local weighted fusion of different modal feature maps within the lesion boundary neighborhood is performed to obtain a boundary enhancement fusion feature map. Based on the boundary enhancement fusion feature map, the lesion boundary in the initial segmentation result is corrected to obtain a fine segmentation result of the target lesion.

[0007] Preferably, acquiring multimodal medical images corresponding to the same target lesion, and generating modal feature maps corresponding to each modal medical image based on the multimodal medical images, including: Spatial benchmark unification is performed on multimodal medical images to obtain multimodal medical images with consistent spatial benchmarks; Feature extraction is performed on multimodal medical images with consistent spatial reference to obtain initial modal feature maps corresponding to each modality of medical image; The initial modal feature maps are adjusted to the same feature scale to obtain the modal feature maps corresponding to each modal medical image.

[0008] Preferably, generating an initial segmentation result of the target lesion based on the modal feature map, and determining the lesion boundary neighborhood based on the initial segmentation result, includes: Initial fusion of the modal feature maps is performed to obtain an initial fused feature map; An initial segmentation probability map of the target lesion is generated based on the initial fusion feature map; The initial segmentation result is determined based on the initial segmentation probability map; The lesion boundary neighborhood is determined based on the boundary location between the lesion area and the non-lesion area in the initial segmentation results.

[0009] Preferably, within the neighborhood of the lesion boundary, the local feature regions to be aligned in different modal feature maps are used as alignment objects to determine the local feature similarity, boundary gradient consistency, and residual spatial offset corresponding to the neighborhood of the lesion boundary, including: Select the boundary location within the neighborhood of the lesion boundary; Using the boundary location as the center, determine the reference local feature region in the reference modal feature map; In the non-reference modal feature map, determine the local feature region to be aligned that corresponds to the reference local feature region; Based on the baseline local feature region and the local feature region to be aligned, calculate the local feature similarity, boundary gradient consistency and residual spatial offset respectively.

[0010] Preferably, based on the reference local feature region and the local feature region to be aligned, the local feature similarity, boundary gradient consistency, and residual spatial offset are calculated respectively, including: Extract the reference local feature vector corresponding to the reference local feature region, and extract the local feature vector to be aligned corresponding to the local feature region to be aligned; Local feature similarity is calculated using the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Local feature similarity of a non-benchmark modality feature map; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; The baseline modal feature map at the boundary location The corresponding baseline local feature vector; For the first The non-reference modal feature map at the boundary location The corresponding local feature vector to be aligned; The dot product of the reference local feature vector and the local feature vector to be aligned; The magnitude of the baseline local eigenvector; Let be the magnitude of the local feature vector to be aligned; This is a stable parameter used to prevent the denominator from being zero.

[0011] Preferably, based on the reference local feature region and the local feature region to be aligned, the local feature similarity, boundary gradient consistency, and residual spatial offset are calculated respectively, including: The gradient direction of the reference boundary is determined based on the reference local feature region, and the gradient direction of the boundary to be aligned is determined based on the local feature region to be aligned. Boundary gradient consistency is calculated using the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary gradient consistency of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; The baseline modal feature map at the boundary location The corresponding reference boundary gradient direction; For the first The non-reference modal feature map at the boundary location The corresponding gradient direction of the boundary to be aligned; This is a value representing the directional consistency between the gradient direction of the reference boundary and the gradient direction of the boundary to be aligned.

[0012] Preferably, based on the reference local feature region and the local feature region to be aligned, the local feature similarity, boundary gradient consistency, and residual spatial offset are calculated respectively, including: In the non-reference modal feature map, the candidate offset range is determined with the local feature region to be aligned as the center; Within the candidate offset range, the regional difference values ​​between the baseline local feature region and different candidate local feature regions are calculated respectively; The positional offset of the candidate local feature region with the smallest regional difference value relative to the local feature region to be aligned is determined as the candidate offset amount; The candidate offset and the residual space offset are calculated using the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Candidate offsets of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; For any candidate position offset within the candidate offset range; Boundary position The corresponding candidate offset range; The baseline modal feature map at the boundary location The corresponding local feature region of the reference; For the first The non-reference modal feature map at the boundary location Offset by candidate position The corresponding candidate local feature region; This represents the regional difference value between the baseline local feature region and the candidate local feature region. The boundary location within the neighborhood of the lesion boundary The corresponding number Residual spatial offset of each non-reference modal feature map; The modulus of the candidate offset.

[0013] Preferably, based on local feature similarity, boundary gradient consistency, and residual spatial offset, boundary neighborhood alignment weights are generated for each location within the lesion boundary neighborhood, including: Obtain the local feature similarity, boundary gradient consistency, and residual spatial offset corresponding to the boundary position within the neighborhood of the lesion boundary; The alignment confidence value and boundary neighborhood alignment weight are calculated using the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Alignment confidence values ​​of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Local feature similarity of a non-benchmark modality feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary gradient consistency of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Residual spatial offset of each non-reference modal feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary neighborhood alignment weights of non-benchmark modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Alignment confidence values ​​of non-reference modal feature maps; The summation index of the non-reference modal feature maps participating in the local weighted fusion; This is the sum of the alignment confidence values ​​corresponding to the feature maps of each non-reference modality participating in the local weighted fusion; This is a stable parameter used to prevent the denominator from being zero.

[0014] Preferably, by utilizing boundary neighborhood alignment weights, local weighted fusion of different modal feature maps within the lesion boundary neighborhood is performed to obtain a boundary enhancement fusion feature map, including: Based on the boundary neighborhood alignment weight, determine the local fusion weights corresponding to different modal feature maps within the lesion boundary neighborhood; Based on the local fusion weight, the feature maps of different modalities in the neighborhood of the lesion boundary are weighted. The boundary enhancement fusion feature map at the boundary location is obtained according to the following formula. Boundary enhancement fusion features at the location: in, Enhance the fusion feature map at the boundary location Enhanced fusion features at the boundary; The boundary location within the neighborhood of the lesion boundary; The baseline modal feature map at the boundary location Features of the location; The sequence number of the non-reference mode feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary neighborhood alignment weights of non-benchmark modal feature maps; For the first The non-reference modal feature map at the boundary location via candidate offset Features at the corresponding position after correction; The boundary location within the neighborhood of the lesion boundary The corresponding number Candidate offsets of non-reference modal feature maps; The fused features are obtained by weighting the non-reference modal feature maps participating in local weighted fusion with boundary neighborhood alignment weights; The normalization term corresponds to the baseline modal feature map and the non-baseline modal feature map participating in the local weighted fusion.

[0015] Preferably, the lesion boundaries in the initial segmentation result are corrected based on the boundary enhancement fusion feature map to obtain a refined segmentation result of the target lesion, including: A boundary correction probability map is generated based on the boundary enhancement fusion feature map, and an initial segmentation probability map is generated based on the initial segmentation result; The location of the boundary to be corrected in the initial segmentation result is determined based on the boundary correction probability map; The lesion boundaries in the initial segmentation results are corrected according to the following formula: in, Boundary position The corresponding corrected segmentation probability; The boundary location within the neighborhood of the lesion boundary; The initial segmentation probability map at the boundary location The initial segmentation probability at the location; Correct the probability map at the boundary location The boundary correction probability is used to obtain the fine segmentation result of the target lesion.

[0016] The present invention discloses the following beneficial effects: This invention first generates modal feature maps corresponding to each modality of multimodal medical images, then obtains the initial segmentation result of the target lesion from the modal feature maps, and determines the lesion boundary neighborhood based on the initial segmentation result. This allows subsequent processing to focus on the lesion boundary region, which is easily affected by residual misalignment. Within the lesion boundary neighborhood, the local feature regions to be aligned in different modal feature maps are further used as alignment objects. Boundary neighborhood alignment weights are generated by combining local feature similarity, boundary gradient consistency, and residual spatial offset. This reduces the impact of modal features with large local misalignment or inconsistent boundary directions after registration on the fusion result. After local weighted fusion using the boundary neighborhood alignment weights, a boundary-enhanced fusion feature map is obtained, which is used to correct the lesion boundary in the initial segmentation result. This allows the lesion boundary correction to be based on the local alignment relationship within the boundary neighborhood, rather than relying solely on the overall fusion feature after registration. Under the same training data and segmentation network conditions, this processing can typically reduce the local missegmentation region at the lesion boundary and keep the boundary contour offset within a small range, thereby improving the stability of the fine segmentation result in the lesion edge region. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of local feature alignment within the lesion boundary neighborhood provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the boundary neighborhood alignment weight generation and lesion boundary correction process provided in an embodiment of the present invention; Explanation of reference numerals in the attached figures: 1. Target lesion region; 2. Non-lesion region; 3. Lesion boundary; 4. Lesion boundary neighborhood; 5. Baseline modality feature map; 6. Non-baseline modality feature map; 7. Baseline local feature region; 8. Local feature region to be aligned; 9. Candidate offset range; 10. Candidate offset amount; 11. Local feature similarity; 12. Boundary gradient consistency; 13. Residual space offset; 14. Alignment confidence value; 15. Boundary neighborhood alignment weight; 16. Boundary enhancement fusion feature map; 17. Boundary correction probability map; 18. Boundary position to be corrected; 19. Fine segmentation result. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The purpose of this invention is to provide a method for fine segmentation of medical image lesions based on multimodal feature alignment. The method determines the lesion boundary neighborhood based on the initial segmentation result, and generates boundary neighborhood alignment weights within the lesion boundary neighborhood based on local feature similarity, boundary gradient consistency, and residual spatial offset. This enables the boundary enhancement fusion feature map to more accurately reflect the local correspondence of different modal feature maps at the lesion boundary, reduces the influence of modal features with large residual spatial offset on boundary correction, and thus improves the boundary stability of the fine segmentation result of the target lesion.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, the present invention provides a method for fine segmentation of medical image lesions based on multimodal feature alignment, comprising the following steps.

[0023] Step 100: Acquire multimodal medical images corresponding to the same target lesion, and generate modal feature maps corresponding to each modal medical image based on the multimodal medical images.

[0024] In step 100, the multimodal medical images are at least two types of medical images acquired for the same target lesion. The multimodal medical images may include at least two image types: structural images, functional images, or enhanced images. Different modalities of medical images are used to characterize the target lesion and adjacent tissues from different imaging information levels.

[0025] Spatial benchmark unification is performed on the multimodal medical images to obtain multimodal medical images with consistent spatial benchmarks. Spatial benchmark unification ensures that different modal medical images are located under the same spatial coordinate benchmark and that the same anatomical location in different modal medical images has a corresponding relationship. For the m-th modal medical image, the spatial benchmark unification process is expressed as follows: in, The m-th modal medical image is a spatially consistent medical image obtained after spatial benchmark unification. This is the m-th modal medical image; The spatial transformation relationship corresponding to the m-th modal medical image; This refers to the serial number of the modal medical image.

[0026] Feature extraction is performed on the multimodal medical images with consistent spatial reference to obtain initial modal feature maps corresponding to each modal medical image; the initial modal feature maps are then adjusted to the same feature scale to obtain the modal feature map corresponding to each modal medical image. The modal feature map corresponding to the m-th modal medical image is represented as: in, This is the modal feature map corresponding to the m-th modal medical image; This is the feature extraction mapping corresponding to the m-th modality of medical image; This is the scaling mapping corresponding to the m-th initial modal feature map; This refers to the spatially consistent medical image obtained after spatial benchmark unification of the m-th modal medical image.

[0027] In one embodiment, the multimodal medical images include CT images and MRI images corresponding to the same lesion region. The CT images are used as the first modality of medical images, and the MRI images as the second modality. After spatial benchmark unification, the CT images and MRI images are adjusted to have the same voxel spacing and the same spatial size. After feature extraction from the CT images and MRI images respectively, first modality feature maps and second modality feature maps are obtained, and the two modality feature maps are adjusted to the same feature scale. This embodiment is only used to illustrate the process of generating modality feature maps from different modalities of medical images and does not limit the multimodal medical images to CT images and MRI images.

[0028] Through the above processing, the modal feature maps corresponding to each modality of medical images have a consistent spatial benchmark and feature scale, providing a data foundation for subsequent local feature alignment within the lesion boundary neighborhood.

[0029] Step 200: Generate an initial segmentation result of the target lesion based on the modal feature map, and determine the lesion boundary neighborhood based on the initial segmentation result.

[0030] In step 200, the modal feature maps are initially fused to obtain an initial fused feature map. This initial fusion is used to obtain the overall positional response of the target lesion, without refining local residual misalignments within the lesion boundary neighborhood. The initial fused feature map is represented as follows: in, The initial fused feature at position x; The number of modal medical images; Let x be the modal feature map corresponding to the m-th modal medical image at position x.

[0031] An initial segmentation probability map of the target lesion is generated based on the initial fusion feature map, and the initial segmentation result is determined according to the initial segmentation probability map. The initial segmentation probability map is represented as follows: in, Let x be the initial segmentation probability at position x; It is a probability mapping function; This is the initial segmentation mapping; This is the initial fusion feature map.

[0032] Based on the boundary between the lesion region and the non-lesion region in the initial segmentation result, the lesion boundary is determined, and the lesion boundary neighborhood is determined based on the lesion boundary. The lesion boundary neighborhood is represented as follows: in, The lesion boundary neighborhood; Location in medical imaging; The boundary location on the lesion boundary; This refers to the set of lesion boundary locations determined based on the initial segmentation results; The radius of the neighborhood used to define the extent of the lesion boundary; Let x be the spatial distance between position x and boundary position p.

[0033] In one embodiment, after obtaining the lesion region based on the initial segmentation probability map, the adjacent positions between the lesion region and the non-lesion region are determined as the lesion boundary position set. When the neighborhood radius When using two feature map grid units, the system expands outwards and inwards from the boundary of each lesion to obtain the corresponding lesion boundary neighborhood. This process focuses subsequent calculations on the locations most prone to boundary shifts in the initial segmentation results, rather than performing a local shift search across the entire medical image area.

[0034] Therefore, subsequent local alignment processing is concentrated within the lesion boundary neighborhood, avoiding unnecessary local alignment calculations for the lesion interior and background areas far from the lesion.

[0035] Step 300: Within the neighborhood of the lesion boundary, using the local feature regions to be aligned in different modal feature maps as alignment objects, determine the local feature similarity, boundary gradient consistency, and residual spatial offset corresponding to the neighborhood of the lesion boundary.

[0036] In step 300, a boundary location is selected within the neighborhood of the lesion boundary. Figure 2 This is a schematic diagram illustrating the alignment of local features within the neighborhood of the lesion boundary, provided in an embodiment of the present invention. Figure 2 As shown, the target lesion region 1 is located within the non-lesion region 2. The boundary between the target lesion region 1 and the non-lesion region 2 forms the lesion boundary 3, and the lesion boundary neighborhood 4 is obtained by extending to the adjacent region based on the lesion boundary 3. The reference modal feature map 5 is used to provide a local alignment reference within the lesion boundary neighborhood 4, and the non-reference modal feature map 6 is used to provide supplementary modal features corresponding to the reference modal feature map 5. Within the lesion boundary neighborhood 4, with the boundary position on the lesion boundary 3 as the center, the reference local feature region 7 is determined in the reference modal feature map 5, and the local feature region to be aligned 8 is determined in the non-reference modal feature map 6. A candidate offset range 9 is set around the local feature region to be aligned 8, and the candidate position offset with the smallest difference value from the reference local feature region 7 is searched within the candidate offset range 9. This candidate position offset is determined as the candidate offset amount 10. Thus, Figure 2 The spatial correspondence between the lesion boundary neighborhood 4, the reference local feature region 7, the local feature region to be aligned 8, the candidate offset range 9, and the candidate offset amount 10 is shown.

[0037] The boundary position is derived from the intersection between the lesion region and the non-lesion region in the initial segmentation result, specifically from the lesion boundary 3 between the target lesion region 1 and the non-lesion region 2. To avoid a lack of unified reference between different modal feature maps, this embodiment first determines a baseline modal feature map and uses it as the reference object for local alignment calculation.

[0038] Specifically, a baseline modal feature map is determined based on the average boundary response intensity of each modal feature map within the set of lesion boundary locations. The modal number of the baseline modal feature map is determined according to the following formula: in, The modal index is the reference modal feature map; The sequence number of the modal feature map; This refers to the set of lesion boundary locations determined based on the initial segmentation results; The number of boundary locations in the set of lesion boundary locations; The boundary location is the boundary location within the set of lesion boundary locations; The feature of the m-th modal feature map at boundary position p; Let be the feature gradient of the m-th modal feature map at boundary position p; Let be the magnitude of the feature gradient. After determining the modal index of the reference modal feature map, the th... The modal feature map is denoted as the baseline modal feature map. The remaining modal feature maps are denoted as non-reference modal feature maps. .

[0039] In one embodiment, if the average boundary response intensity of the first modal feature map within the lesion boundary location set is 0.72, and the average boundary response intensity of the second modal feature map within the lesion boundary location set is 0.58, then the first modal feature map is determined as the reference modal feature map, and the second modal feature map is determined as the non-reference modal feature map. Using this method, the modal feature map with a more pronounced boundary response serves as the local alignment reference, reducing the impact of blurred boundary modalities on local offset search results.

[0040] After determining the reference modal feature map, a reference local feature region is determined in the reference modal feature map, centered on the boundary position p, and a local feature region to be aligned, corresponding to the reference local feature region, is determined in the non-reference modal feature map. The reference local feature region and the local feature region to be aligned are respectively represented as follows: in, The reference modal feature map is the reference local feature region at the boundary position p; The local feature region to be aligned is the m-th non-reference mode feature map at boundary position p. This refers to a local region centered at the boundary location p. The location within the local area; The feature of the baseline modal feature map at position y; Let y be the feature of the m-th non-reference mode feature map at position y.

[0041] The reference local feature vector corresponding to the reference local feature region is extracted, and the local feature vector to be aligned corresponding to the local feature region to be aligned is extracted. The reference local feature vector and the local feature vector to be aligned can be obtained by flattening, average pooling, or weighted pooling the features within the corresponding local feature regions. To ensure a definite input for subsequent similarity calculations, this embodiment uses average pooling to obtain the local feature vector, expressed as: in, This is the reference local feature vector corresponding to the reference modal feature map at boundary position p; Let be the local feature vector to be aligned at the boundary position p of the m-th non-reference mode feature map; For local areas Number of positions within.

[0042] The local feature similarity is calculated according to the following formula: in, The local feature similarity of the m-th non-reference modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary; The vector dot product of the reference local feature vector and the local feature vector to be aligned is given. The magnitude of the reference local feature vector; Let the magnitude of the local feature vector to be aligned be denoted as 'x'. This is a stable parameter used to prevent the denominator from being zero.

[0043] In one embodiment, for the location of the lesion boundary The baseline local feature vector is [0.42, 0.31, 0.27], and the local feature vector to be aligned is [0.38, 0.35, 0.22]. After calculation using the local feature similarity formula, the local feature similarity is 0.98. This result indicates that the two modal feature maps are similar at the boundary position. The local responses in the vicinity are relatively close and can be used as a local correspondence with higher confidence to participate in subsequent weight calculations.

[0044] The reference boundary gradient direction is determined based on the reference local feature region, and the boundary gradient direction to be aligned is determined based on the local feature region to be aligned. Specifically, the gradient components of the reference modal feature map and the non-reference modal feature map at the boundary position p along the first spatial direction and the second spatial direction are first calculated, and then the boundary gradient direction is calculated based on the gradient components. The boundary gradient direction is expressed as: in, The boundary gradient direction is the boundary position p of the m-th modal feature map. Let be the gradient component of the m-th modal feature map at boundary position p along the first spatial direction; Let be the gradient component of the m-th modal feature map at boundary position p along the second spatial direction; This is the arctangent function for determining the direction angle based on the two gradient components.

[0045] After obtaining the reference boundary gradient direction and the boundary gradient direction to be aligned, the boundary gradient consistency is calculated according to the following formula: in, The boundary gradient consistency of the m-th non-reference modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary; The reference boundary gradient direction is the reference modal feature map at boundary position p. Let be the gradient direction of the boundary to be aligned corresponding to the m-th non-reference modality feature map at boundary position p. This formula normalizes the orientation consistency to the range of 0 to 1, which facilitates its subsequent participation in the boundary neighborhood alignment weight calculation along with local feature similarity and residual spatial offset.

[0046] In one embodiment, the gradient direction of the reference boundary is 30°, and the gradient direction of the boundary to be aligned is 40°. The difference between the two directions is 10°. After calculation using the boundary gradient consistency formula, the boundary gradient consistency is approximately 0.99. This result indicates that the boundary orientations of the two modal feature maps are relatively consistent near this boundary position, which can reduce the fusion error caused by inconsistent boundary directions.

[0047] In the non-reference modal feature map, a candidate offset range is determined centered on the local feature region to be aligned. The candidate offset range is a set of multiple candidate position offsets. For each candidate position offset... Candidate local feature regions are obtained from the non-reference mode feature map. The region difference value between the candidate local feature region and the reference local feature region is calculated. The region difference value is expressed as: in, For the boundary position p and the candidate position offset Corresponding regional difference values; For any candidate position offset within the candidate offset range; The feature of the baseline modal feature map at position y; For the m-th non-reference mode feature map at position Features of the location; This represents the squared feature difference value at the corresponding location.

[0048] For the same boundary location p and the same non-reference mode feature map The candidate position offsets within the candidate offset range are sorted in ascending order of their regional difference values. The candidate position offset with the smallest regional difference value in the sorted results is determined as the candidate offset. The candidate offset is expressed as: in, The candidate offset of the m-th non-reference modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary; The candidate offset range corresponding to the boundary position p; For the boundary position p and the candidate position offset The corresponding regional difference value.

[0049] When two or more candidate offsets correspond to the same minimum regional difference value, the candidate offset with the smallest modulus is selected as the candidate offset. When the modulus is still the same, the candidate offset that appears earlier in the preset traversal order within the candidate offset range is selected as the candidate offset. Therefore, the candidate offset has a unique result, avoiding the confusion in subsequent calculations caused by multiple candidate offsets corresponding to the same boundary position.

[0050] The residual space offset is calculated using the following formula: in, The residual spatial offset of the m-th non-reference modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary; The modulus of the candidate offset is given.

[0051] In one embodiment, the location of the lesion boundary Centered on, set the candidate offset range to The regional difference values ​​corresponding to the offsets of each candidate position were calculated, and the regional difference values ​​were 0.18, 0.11, 0.15, 0.13, and 0.09, respectively. After sorting the candidate positions by their regional difference values ​​from smallest to largest, the offsets were... The corresponding regional difference value is the smallest, therefore... Determined as a candidate offset And obtain the residual space offset. This result indicates that the non-reference modal feature map has a local residual offset of one grid unit relative to the reference modal feature map near this boundary location.

[0052] Through the above processing, this embodiment does not directly assume that different modal feature maps correspond completely at the lesion boundary. Instead, it sorts and filters candidate position offsets within the lesion boundary neighborhood, and uses a uniquely determined candidate offset to represent the residual spatial offset between different modal feature maps. This processing enables subsequent boundary neighborhood alignment weights to suppress modal features with large local misalignments.

[0053] Step 400: Generate boundary neighborhood alignment weights corresponding to each position within the lesion boundary neighborhood based on the local feature similarity, the boundary gradient consistency, and the residual spatial offset.

[0054] In step 400, Figure 3 This is a schematic diagram illustrating the process of generating boundary neighborhood alignment weights and correcting lesion boundaries, provided in an embodiment of the present invention. Figure 3As shown, local feature similarity 11, boundary gradient consistency 12, and residual spatial offset 13 are determined within the lesion boundary neighborhood, and alignment confidence value 14 is calculated based on these factors. Furthermore, boundary neighborhood alignment weight 15 is generated based on the alignment confidence value 14, and the boundary neighborhood alignment weight 15 is used to locally weight and fuse different modal feature maps to obtain a boundary enhancement fusion feature map 16. A boundary correction probability map 17 is generated based on the boundary enhancement fusion feature map 16, and the position of the boundary to be corrected 18 is determined in conjunction with the initial segmentation result. Finally, the lesion boundary in the initial segmentation result is corrected based on the position of the boundary to be corrected 18 to obtain a refined segmentation result 19. Thus, Figure 3 This illustrates the continuous data processing relationships from local alignment index calculation, boundary neighborhood alignment weight generation, boundary enhancement fusion to lesion boundary correction.

[0055] Obtain the local feature similarity 11, boundary gradient consistency 12, and residual spatial offset 13 corresponding to the same boundary position p, and calculate the alignment confidence value 14 based on the local feature similarity 11, boundary gradient consistency 12, and residual spatial offset 13. Since the value range of the local feature similarity is from negative to positive, the local feature similarity is first converted into a non-negative similarity representation value. The non-negative similarity representation value is expressed as: in, , is the non-negative similarity representation value of the m-th non-benchmark modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary; The local feature similarity is denoted as .

[0056] The alignment confidence value is calculated based on the non-negative similarity representation value, the boundary gradient consistency, and the residual spatial offset. The alignment confidence value is expressed as: in, The alignment confidence value is the m-th non-reference modal feature map corresponding to the boundary position p within the neighborhood of the lesion boundary. This is the non-negative similarity representation value; For the consistency of the boundary gradient; This is the residual space offset.

[0057] For each non-reference modal feature map corresponding to the same boundary position p, they are sorted from largest to smallest according to the alignment confidence value. Non-reference modal feature maps sorted higher indicate higher local feature similarity, higher boundary gradient consistency, and smaller residual spatial offset; non-reference modal feature maps sorted lower indicate lower local alignment reliability at the boundary position. The boundary neighborhood alignment weight 15 is generated based on the sorting result and the alignment confidence value. The boundary neighborhood alignment weight is expressed as: in, The boundary neighborhood alignment weight is the m-th non-reference modal feature map corresponding to the boundary position p within the lesion boundary neighborhood. Let be the alignment confidence value of the m-th non-reference mode feature map; Let be the alignment confidence value of the k-th non-reference mode feature map; This is the set of non-reference modal feature map indices that participate in the local weighted fusion at boundary location p; This is the sum of the alignment confidence values ​​corresponding to the feature maps of each non-reference modality participating in the local weighted fusion; This is a stable parameter used to prevent the denominator from being zero.

[0058] In one implementation, the set of non-reference mode feature map indices It can include all non-reference modal feature maps. To further suppress modal features with large local misalignments, the set of non-reference modal feature map indices... It can also be composed of non-reference modal feature maps that are ranked first. The number of non-reference modal feature maps participating in local weighted fusion is determined by the number of modalities of the multimodal medical image, and includes at least one non-reference modal feature map.

[0059] In one embodiment, for the same lesion boundary location The non-negative similarity values ​​of the two non-reference modal feature maps are 0.99 and 0.76, respectively; the boundary gradient consistency is 0.99 and 0.82, respectively; and the residual spatial offset is 1 and 2, respectively. According to the alignment confidence value formula, the alignment confidence values ​​of the two non-reference modal feature maps are 0.490 and 0.208, respectively. After sorting by alignment confidence value from largest to smallest, the first non-reference modal feature map is ranked first. If both non-reference modal feature maps participate in local weighted fusion, the boundary neighborhood alignment weight corresponding to the first non-reference modal feature map is greater than that corresponding to the second non-reference modal feature map. This ranking and weighting result allows modal features with smaller local misalignment and more consistent boundary orientation to participate more in subsequent boundary enhancement fusion.

[0060] Through the above calculations, the boundary neighborhood alignment weight is jointly determined by local feature similarity, boundary gradient consistency, and residual spatial offset. Higher local feature similarity, higher boundary gradient consistency, smaller residual spatial offset, and higher alignment confidence value result in a larger corresponding boundary neighborhood alignment weight. This weight calculation method allows more modal features with more stable correspondences within the lesion boundary neighborhood to participate in the fusion, reducing the interference of misaligned features on lesion boundary correction.

[0061] Step 500: Using the boundary neighborhood alignment weight, perform local weighted fusion of different modal feature maps within the lesion boundary neighborhood to obtain a boundary enhancement fusion feature map.

[0062] In step 500, local fusion weights corresponding to different modal feature maps within the lesion boundary neighborhood are determined based on the boundary neighborhood alignment weights. For example... Figure 3 As shown, the boundary neighborhood alignment weight 15 is used to control the contribution of different modal feature maps to the boundary enhancement fusion feature map 16. For the reference modal feature map, the features at the boundary position p are retained. For the m-th non-reference mode feature map, based on the candidate offset... Determine the sampling location after calibration And obtain non-reference mode features at the corrected sampling position.

[0063] When the corrected sampling position is located on a discrete grid point, the feature of the m-th non-reference mode feature map at the corrected sampling position is directly read; when the corrected sampling position is located between discrete grid points, interpolation is performed based on the feature values ​​of neighboring grid points to obtain the feature of the m-th non-reference mode feature map at the corrected sampling position. Therefore, It has a definite value selection method in both continuous and discrete offset cases.

[0064] The boundary enhancement fusion feature at boundary location p is obtained according to the following formula: in, The boundary enhancement and fusion feature map is the boundary enhancement and fusion feature at boundary location p; The feature of the baseline modal feature map at boundary position p; This is the set of non-reference modal feature map indices that participate in the local weighted fusion at boundary location p; The boundary neighborhood alignment weight is the m-th non-reference modal feature map corresponding to the boundary position p within the lesion boundary neighborhood. For the m-th non-reference mode feature map at boundary position p, the candidate offset is... Features at the corresponding position after correction; The fused features are the non-reference modal feature maps participating in local weighted fusion after being weighted by the boundary neighborhood alignment weights; The normalization term is the corresponding to the baseline modal feature map and the non-baseline modal feature map participating in the local weighted fusion.

[0065] In one embodiment, the reference modal feature map is located at the boundary. The feature value at the first non-reference modal feature map is 0.70, the feature value at the corresponding position after candidate offset correction is 0.66, and the feature value at the corresponding position after candidate offset correction is 0.52. If the boundary neighborhood alignment weights corresponding to the first and second non-reference modal feature maps are 0.70 and 0.30 respectively, the boundary enhancement fusion feature is approximately 0.65. Compared with direct fusion without candidate offset correction, this processing reduces the influence of modal features with large residual spatial offsets on the fusion result, making the boundary enhancement fusion feature closer to the boundary response with stable local correspondences.

[0066] For locations within the lesion boundary neighborhood that are not selected as boundary positions but belong to the lesion boundary neighborhood, interpolation can be performed based on the boundary enhancement fusion features corresponding to neighboring boundary positions. Alternatively, this location can be used as a new local processing location to repeat the aforementioned local weighted fusion process. For locations outside the lesion boundary neighborhood, the features corresponding to the initial fusion feature map or initial segmentation result obtained in step 200 are retained. Thus, the boundary enhancement fusion feature map mainly strengthens the correspondence of local features within the lesion boundary neighborhood without changing the segmentation basis for regions far from the lesion boundary.

[0067] Through the above fusion method, the reference modal feature map provides a stable boundary space reference, while the non-reference modal feature maps participate in the fusion after candidate offset correction. Non-reference modal feature maps with smaller residual spatial offsets and consistent boundary directions receive larger boundary neighborhood alignment weights, while non-reference modal feature maps with larger residual spatial offsets receive smaller boundary neighborhood alignment weights, thereby reducing the impact of misaligned modal features on the boundary enhancement fused feature map.

[0068] Step 600: Based on the boundary enhancement fusion feature map, the lesion boundary in the initial segmentation result is corrected to obtain the fine segmentation result of the target lesion.

[0069] In step 600, a boundary correction probability map is generated based on the boundary enhancement fusion feature map. For example... Figure 3 As shown, the boundary enhancement fusion feature map 16 is used to generate the boundary correction probability map 17, which represents the corrected probability that each location within the lesion boundary neighborhood belongs to the target lesion. The boundary correction probability map is represented as follows: in, The boundary correction probability of the boundary correction probability map at boundary position p; Correct the mapping for the boundary; The boundary enhancement fusion feature map; This is a probability mapping function.

[0070] The location of the boundary to be corrected in the initial segmentation result is determined based on the boundary correction probability map. Specifically, for each boundary location p within the neighborhood of the lesion boundary, the probability difference between the boundary correction probability and the initial segmentation probability is calculated. The probability difference value is expressed as: in, This represents the probability difference value corresponding to the boundary position p; The boundary correction probability of the boundary correction probability map at boundary position p; Let p be the initial segmentation probability of the initial segmentation probability map at the boundary position.

[0071] The boundary positions within the lesion boundary neighborhood are sorted from largest to smallest according to the probability difference value, and the boundary positions that are ranked first and located within the lesion boundary neighborhood are determined as the boundary positions to be corrected 18. The boundary positions to be corrected 18 are used to limit the spatial range of lesion boundary correction, so that boundary correction is concentrated on positions where the initial segmentation probability and the boundary correction probability differ significantly. If it is necessary to avoid introducing a specific number limit, the boundary positions whose probability difference value is greater than the average probability difference value within the lesion boundary neighborhood can also be determined as the boundary positions to be corrected. The average probability difference value is expressed as: in, This represents the average probability difference within the neighborhood of the lesion boundary; This is a set of locations representing the boundaries of the lesions. The number of boundary locations in the set of lesion boundary locations; Let p be the probability difference value corresponding to the boundary position.

[0072] For the boundary positions to be corrected, the lesion boundaries in the initial segmentation results are corrected according to the following formula: in, The corrected segmentation probability corresponding to the boundary position p; The initial segmentation probability of the initial segmentation probability map at boundary position p; The boundary correction probability is the boundary correction probability at the boundary position p of the boundary correction probability map; the corrected segmentation probability is used to obtain the fine segmentation result of the target lesion 19.

[0073] In one embodiment, for the location of the lesion boundary The initial segmentation probability is 0.48, the boundary correction probability is 0.64, and the probability difference value is 0.16. If the average probability difference value within the neighborhood of the lesion boundary is 0.10, then the boundary position... The location is identified as the boundary position to be corrected. The boundary position is calculated according to the corrected segmentation probability formula. The corresponding corrected segmentation probability is 0.56. This corrected segmentation probability is closer to the boundary response after local alignment and fusion than the initial segmentation probability, thus moderately correcting the lesion boundary towards the target lesion at that location.

[0074] For locations not identified as boundary positions to be corrected but within the neighborhood of the lesion boundary, the initial segmentation probability is retained or a smooth transition is performed based on the corrected segmentation probability of neighboring boundary positions to be corrected; for locations outside the neighborhood of the lesion boundary, the initial segmentation result is retained. Based on the corrected segmentation probability and the retained initial segmentation result, a refined segmentation result for the target lesion is obtained.

[0075] Through the above processing, lesion boundary correction does not completely replace the initial segmentation result, but focuses on locations with significant probability differences within the neighborhood of the lesion boundary. This method can preserve the stability of the main region of the initial segmentation result while using boundary enhancement fusion feature maps to correct boundary locations that are significantly affected by local residual missegmentation, thereby reducing local missegmentation at the boundary.

[0076] This implementation determines the lesion boundary neighborhood based on the initial segmentation result, and generates boundary neighborhood alignment weights within the lesion boundary neighborhood based on local feature similarity, boundary gradient consistency, and residual space offset. This enables the boundary enhancement fusion feature map to reflect the local correspondence of different modal feature maps at the lesion boundary, reduces the influence of modal features with large residual space offset on lesion boundary correction, and thus improves the boundary stability of the fine segmentation result of the target lesion.

[0077] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0078] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for fine segmentation of medical image lesions based on multimodal feature alignment, characterized in that, include: Acquire multimodal medical images corresponding to the same target lesion, and generate modal feature maps corresponding to each modal medical image based on the multimodal medical images; An initial segmentation result of the target lesion is generated based on the modal feature map, and the lesion boundary neighborhood is determined based on the initial segmentation result; Within the neighborhood of the lesion boundary, the local feature regions to be aligned in different modal feature maps are used as alignment objects to determine the local feature similarity, boundary gradient consistency and residual space offset corresponding to the neighborhood of the lesion boundary. Based on the local feature similarity, the boundary gradient consistency, and the residual spatial offset, generate boundary neighborhood alignment weights corresponding to each position within the lesion boundary neighborhood; Using the boundary neighborhood alignment weight, local weighted fusion of different modal feature maps within the lesion boundary neighborhood is performed to obtain a boundary enhancement fusion feature map; Based on the boundary enhancement fusion feature map, the lesion boundary in the initial segmentation result is corrected to obtain a fine segmentation result of the target lesion.

2. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, The step of acquiring multimodal medical images corresponding to the same target lesion and generating modal feature maps corresponding to each modal medical image based on the multimodal medical images includes: Spatial benchmark unification is performed on the multimodal medical images to obtain multimodal medical images with consistent spatial benchmarks; Feature extraction is performed on the multimodal medical images with consistent spatial reference to obtain the initial modal feature map corresponding to each modal medical image; The initial modal feature maps are adjusted to the same feature scale to obtain the modal feature maps corresponding to each modal medical image.

3. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, The step of generating an initial segmentation result of the target lesion based on the modal feature map, and determining the lesion boundary neighborhood based on the initial segmentation result, includes: The modal feature maps are initially fused to obtain an initial fused feature map; An initial segmentation probability map of the target lesion is generated based on the initial fusion feature map; The initial segmentation result is determined based on the initial segmentation probability map; The lesion boundary neighborhood is determined based on the boundary position between the lesion area and the non-lesion area in the initial segmentation result.

4. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, Within the lesion boundary neighborhood, using the local feature regions to be aligned in different modal feature maps as alignment objects, the local feature similarity, boundary gradient consistency, and residual spatial offset corresponding to the lesion boundary neighborhood are determined, including: Select the boundary location within the neighborhood of the lesion boundary; Using the boundary position as the center, a reference local feature region is determined in the reference modal feature map; In the non-reference modal feature map, determine the local feature region to be aligned that corresponds to the reference local feature region; Based on the reference local feature region and the local feature region to be aligned, the local feature similarity, the boundary gradient consistency, and the residual spatial offset are calculated respectively.

5. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 4, characterized in that, The step of calculating the local feature similarity, the boundary gradient consistency, and the residual spatial offset based on the reference local feature region and the local feature region to be aligned includes: Extract the reference local feature vector corresponding to the reference local feature region, and extract the local feature vector to be aligned corresponding to the local feature region to be aligned; The local feature similarity is calculated according to the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Local feature similarity of a non-benchmark modality feature map; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; The baseline modal feature map at the boundary location The corresponding baseline local feature vector; For the first The non-reference modal feature map at the boundary location The corresponding local feature vector to be aligned; The vector dot product of the reference local feature vector and the local feature vector to be aligned is given. The magnitude of the reference local feature vector; Let the magnitude of the local feature vector to be aligned be denoted as 'x'. This is a stable parameter used to prevent the denominator from being zero.

6. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 4, characterized in that, The step of calculating the local feature similarity, the boundary gradient consistency, and the residual spatial offset based on the reference local feature region and the local feature region to be aligned includes: The reference boundary gradient direction is determined based on the reference local feature region, and the boundary gradient direction to be aligned is determined based on the local feature region to be aligned. The boundary gradient consistency is calculated using the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary gradient consistency of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; The baseline modal feature map at the boundary location The corresponding reference boundary gradient direction; For the first The non-reference modal feature map at the boundary location The corresponding gradient direction of the boundary to be aligned; This is a value representing the directional consistency between the gradient direction of the reference boundary and the gradient direction of the boundary to be aligned.

7. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 4, characterized in that, The step of calculating the local feature similarity, the boundary gradient consistency, and the residual spatial offset based on the reference local feature region and the local feature region to be aligned includes: In the non-reference modal feature map, the candidate offset range is determined with the local feature region to be aligned as the center; Within the candidate offset range, the regional difference values ​​between the reference local feature region and different candidate local feature regions are calculated respectively; The positional offset of the candidate local feature region with the smallest regional difference value relative to the local feature region to be aligned is determined as the candidate offset amount; The candidate offset and the residual space offset are calculated according to the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Candidate offsets of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary; The sequence number of the non-reference mode feature map; For any candidate position offset within the candidate offset range; Boundary position The corresponding candidate offset range; The baseline modal feature map at the boundary location The corresponding local feature region of the reference; For the first The non-reference modal feature map at the boundary location Offset by candidate position The corresponding candidate local feature region; The regional difference value between the reference local feature region and the candidate local feature region; The boundary location within the neighborhood of the lesion boundary The corresponding number Residual spatial offset of each non-reference modal feature map; The modulus of the candidate offset is given.

8. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, The step of generating boundary neighborhood alignment weights corresponding to each position within the lesion boundary neighborhood based on the local feature similarity, the boundary gradient consistency, and the residual spatial offset includes: Obtain the local feature similarity, the boundary gradient consistency, and the residual spatial offset corresponding to the boundary position within the neighborhood of the lesion boundary; The alignment confidence value and the boundary neighborhood alignment weight are calculated according to the following formula: in, The boundary location within the neighborhood of the lesion boundary The corresponding number Alignment confidence values ​​of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Local feature similarity of a non-benchmark modality feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary gradient consistency of non-reference modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Residual spatial offset of each non-reference modal feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary neighborhood alignment weights of non-benchmark modal feature maps; The boundary location within the neighborhood of the lesion boundary The corresponding number Alignment confidence values ​​of non-reference modal feature maps; The summation index of the non-reference modal feature maps participating in the local weighted fusion; This is the sum of the alignment confidence values ​​corresponding to the feature maps of each non-reference modality participating in the local weighted fusion; This is a stable parameter used to prevent the denominator from being zero.

9. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, The step of using the boundary neighborhood alignment weight to locally weight and fuse different modal feature maps within the lesion boundary neighborhood to obtain a boundary enhancement fused feature map includes: Based on the boundary neighborhood alignment weights, determine the local fusion weights corresponding to different modal feature maps within the lesion boundary neighborhood; Based on the local fusion weights, the different modal feature maps within the neighborhood of the lesion boundary are weighted. The boundary enhancement fusion feature map at the boundary location is obtained according to the following formula. Boundary enhancement fusion features at the location: in, Enhance the fusion feature map at the boundary location Enhanced fusion features at the boundary; The boundary location within the neighborhood of the lesion boundary; The baseline modal feature map at the boundary location Features of the location; The sequence number of the non-reference mode feature map; The boundary location within the neighborhood of the lesion boundary The corresponding number Boundary neighborhood alignment weights of non-benchmark modal feature maps; For the first The non-reference modal feature map at the boundary location via candidate offset Features at the corresponding position after correction; The boundary location within the neighborhood of the lesion boundary The corresponding number Candidate offsets of non-reference modal feature maps; The fused features are the non-reference modal feature maps participating in local weighted fusion after being weighted by the boundary neighborhood alignment weights; The normalization term is the corresponding to the baseline modal feature map and the non-baseline modal feature map participating in the local weighted fusion.

10. The method for fine segmentation of medical image lesions based on multimodal feature alignment according to claim 1, characterized in that, The step of correcting the lesion boundaries in the initial segmentation result based on the boundary enhancement fusion feature map to obtain a refined segmentation result of the target lesion includes: A boundary correction probability map is generated based on the boundary enhancement fusion feature map, and an initial segmentation probability map is generated based on the initial segmentation result; The location of the boundary to be corrected in the initial segmentation result is determined based on the boundary correction probability map. The lesion boundaries in the initial segmentation results are corrected according to the following formula: in, Boundary position The corresponding corrected segmentation probability; The boundary location within the neighborhood of the lesion boundary; The initial segmentation probability map at the boundary location The initial segmentation probability at the location; The boundary correction probability map is located at the boundary position. The boundary correction probability at the location; the corrected segmentation probability is used to obtain the fine segmentation result of the target lesion.

Images