A prostate image target segmentation and region representation method based on large model text prior and cross attention feature fusion

Abstract

This invention discloses a method for prostate image target segmentation and region representation based on large-model text prior and cross-attention feature fusion. The method includes: acquiring and preprocessing three-dimensional medical images of the prostate, generating offline semantic description text for the regions; constructing a bi-branch feature extraction network to obtain a three-dimensional visual feature map, a three-dimensional visual feature vector, and a text feature vector; constraining the visual features to be consistent with the text semantic space based on alignment loss; performing cross-attention fusion between the text feature vector and the spatial visual feature sequence obtained by expanding the three-dimensional visual feature map to obtain a three-dimensional multimodal fusion feature, which is then used for target segmentation and region representation, outputting a three-dimensional pixel-level prediction mask and region representation results. This invention helps improve the accuracy, stability, and interpretability of segmentation and representation.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a method for prostate image target segmentation and region representation based on the fusion of large model text prior and cross-attention features. Background Technology

[0002] Accurate segmentation and stable representation of target regions in prostate imaging are crucial for quantitative analysis, region annotation, consistency assessment, 3D visualization, and subsequent image data analysis in medical images. Most existing prostate image analysis methods rely on a single visual modality, typically directly inputting ultrasound or MRI images into convolutional neural networks for feature extraction. While this can accomplish region segmentation or representation tasks to some extent, it fails to fully utilize the global semantic information contained in medical text, resulting in limited model performance in scenarios with blurred boundaries, complex textures, or significant sample variations.

[0003] In recent years, visual language models have demonstrated strong cross-modal semantic modeling capabilities in medical image understanding tasks. Related implementations typically attempt to input both images and text into a large model to obtain textual descriptions, visual question-answering results, or semantic inference results corresponding to the image content. However, these models usually have a large parameter scale, and directly applying them to pixel-level prediction tasks of 3D medical images can easily lead to problems such as high training costs, large memory consumption, deployment difficulties, and insufficiently fine prediction boundaries.

[0004] Current methods for prostate imaging target segmentation and region representation have the following main drawbacks: (1) Single-modal networks lack advanced semantic understanding: Traditional single-visual modal networks mainly rely on image grayscale, texture and local spatial structure for feature extraction. It is difficult to utilize global semantic information that can be expressed by text, such as boundary morphology changes, regional texture differences, local intensity distribution features, etc., which leads to insufficient stability of segmentation and representation results in complex scenes.

[0005] (2) Large language models are difficult to perform dense prediction tasks directly: Although visual language models have strong image and text semantic modeling capabilities, when they are directly applied to the pixel-level target region prediction of three-dimensional medical images, they usually bring high training costs and deployment burdens. Moreover, due to the limitations of the model structure, they have low generalization and rough edges when generating pixel-level masks.

[0006] (3) The cross-modal feature fusion method is relatively crude: In the existing methods, visual features and text features are often fused by simple splicing or addition, which makes it difficult to dynamically guide the response of three-dimensional visual features in spatial position according to the semantics of the text, resulting in more feature redundancy and weak targeting after fusion.

[0007] (4) Lack of explicit alignment constraints between visual and text features: In existing multimodal methods, the feature spaces of visual and text branches usually lack explicit semantic consistency constraints, resulting in a semantic gap between the two types of features, which affects the subsequent fusion effect and the overall network performance.

[0008] Therefore, how to effectively introduce the text prior generated by the large model into the prostate 3D image processing network without significantly increasing the computational complexity of online training and deployment, and improve the accuracy and stability of target region segmentation and region representation results, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes a method for prostate image target segmentation and region representation based on the fusion of large model text prior and cross-attention features, thereby resolving the issues existing in the prior art.

[0010] To achieve the above objectives, this invention provides a method for prostate image target segmentation and region representation based on the fusion of large model text priors and cross-attention features, comprising: Three-dimensional medical imaging data of the prostate are acquired and preprocessed, and semantic description text of the region is generated offline based on the preprocessed data. A dual-branch feature extraction network is constructed to extract three-dimensional visual feature maps from the three-dimensional medical image data, and to obtain three-dimensional visual feature vectors by mapping the three-dimensional visual feature maps; text feature vectors are also extracted from the semantic description text of the region based on the dual-branch feature extraction network. During training, the parameters of the text feature extraction branch are locked to block the gradient flow during backpropagation; the alignment loss is calculated based on the 3D visual feature vector and the text feature vector; during the backpropagation of the alignment loss, the parameters of the text feature extraction branch are locked, and the gradient of the alignment loss is backpropagated to the 3D visual feature extraction branch to constrain the alignment of the 3D visual feature vector and the text feature vector. The text feature vector is fused with the spatial visual feature sequence obtained by unfolding the 3D visual feature map through cross-attention to obtain a 3D multimodal fusion feature. The 3D multimodal fusion feature is then used for both target segmentation and region representation to obtain 3D pixel-level prediction mask and region representation results.

[0011] Optionally, the process of generating region semantic description text includes: After resampling and standard normalization of the three-dimensional medical image, two-dimensional slices containing target region information are extracted. Adaptive windowing is performed on the two-dimensional slices to map them to the standard grayscale range. The processed two-dimensional slices and medical prompts are input into a pre-trained large-scale visual language model to generate regional semantic description text about the boundary features, texture distribution, intensity changes and spatial relationships of the target region, and construct the corresponding static text mapping file.

[0012] Optionally, the dual-branch feature extraction network includes a 3D visual feature extraction branch and a text feature extraction branch.

[0013] Optionally, the three-dimensional visual feature extraction branch uses an nnU-Net three-dimensional encoder to extract multi-scale features of the three-dimensional medical image data through continuous convolution and downsampling operations, and outputs a three-dimensional visual feature map; and maps the three-dimensional visual feature map into a three-dimensional visual feature vector through a global average pooling layer and a linear projection layer. The text feature extraction branch uses a pre-trained BERT model to segment and encode the semantic description text of the region, and outputs the text feature vector.

[0014] Optionally, the alignment loss is calculated using the following formula: ; in, For alignment loss, For visual feature vectors, This is the text feature vector.

[0015] Optionally, the text feature vector is fused with the spatial visual feature sequence obtained by unfolding the 3D visual feature map through cross-attention, including: The text feature vector is linearly mapped to a query vector, and the spatial visual feature sequence obtained by expanding the three-dimensional visual feature map is linearly mapped to a key vector and a value vector. The dot product of the query vector and the key vector is calculated and divided by the square root of the dimension. After Softmax normalization, it is multiplied by the value vector to obtain the cross-attention output feature. The cross-attention output feature is fused with the three-dimensional visual feature map to obtain the three-dimensional multimodal fusion feature.

[0016] Optionally, the three-dimensional multimodal fusion features are used for target segmentation, including: The three-dimensional multimodal fusion features are input into the target segmentation branch, which uses the nnU-Net three-dimensional decoder to restore the image resolution through upsampling and deconvolution operations, outputs the three-dimensional pixel-level prediction mask, and calculates the segmentation loss.

[0017] Optionally, the three-dimensional multimodal fusion features are used for region representation, including: The three-dimensional multimodal fusion features are input into the region representation branch, which outputs the region representation results through a global average pooling layer and a multilayer perceptron, and calculates the region representation loss.

[0018] Optionally, the gradients of the segmentation loss and the region representation loss flow through the target segmentation branch, the region representation branch, and the cross-attention fusion module and are then fed back to the 3D visual feature extraction branch.

[0019] Compared with the prior art, the present invention has the following advantages and technical effects: (1) Breaking through the bottleneck of single-modal semantics helps to improve the accuracy and stability of target region segmentation results and region representation results: Compared to traditional methods that rely solely on a single visual modality for feature extraction, this invention introduces a region semantic description text generated by a pre-trained visual language model as a semantic prior, and guides the response of visual features in spatial location through a cross-attention mechanism. Therefore, it can make fuller use of global semantic information in the image, and improve the accuracy and stability of target region segmentation and region representation results.

[0020] (2) Low computational power consumption, reducing the computational resource requirements for training and deployment: To address the pain points of existing large models being extremely memory-intensive and difficult to fine-tune end-to-end when directly applied to 3D medical images, this invention adopts a decoupled architecture of "offline text prior generation + online lightweight network fusion". This design fully utilizes the powerful image understanding capabilities of VLM while completely avoiding the need for online updates of a large number of parameters, significantly reducing the dependence on computing resources. This helps to reduce the computing resource requirements during online training and deployment, and improves the deployability of the method.

[0021] (3) It helps to enhance the semantic consistency of features and the robustness of the model: This invention innovatively incorporates cross-modal feature alignment loss into the network. This mechanism constrains the 3D visual encoder to align with the semantic space corresponding to the semantic description text of the region early in feature extraction, which helps to enhance the semantic consistency between visual features and text priors and improve the expressive power of fused features.

[0022] (4) It possesses high spatial interpretability and result usability: Existing deep learning models are mostly "black boxes," with relatively limited interpretability of their output. The multi-task architecture of this invention not only outputs the region representation of the target area but also simultaneously outputs a 3D target segmentation mask, and intuitively displays the spatial location corresponding to the text description through cross-attention weights. This image-text collaborative reasoning process helps improve the interpretability and visualization of the output, providing support for subsequent image post-processing, region localization, and data analysis. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating the technical solution of an embodiment of the present invention; Figure 2 This is a schematic diagram of the multi-task network architecture and gradient backhaul according to an embodiment of the present invention; Figure 3 This is a diagram of the cross-attention feature fusion structure according to an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0026] Example 1 like Figure 1 As shown, this embodiment provides a method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion. In the figure, rectangles represent data processing and network modules, parallelograms represent data tensors and feature vectors, and dashed ellipses represent loss functions. Solid arrows indicate the data flow direction during forward propagation, while annotated dashed arrows indicate the gradient flow direction during loss calculation and backpropagation. Among these, alignment loss (… ) is mainly used to constrain the semantic features output by the visual encoder, segmentation loss ( ) and region representation loss ( This is then applied to the downstream prediction branch corresponding to the fused features, achieving joint optimization of the overall network. The specific steps of the method include: Step 1: Data preprocessing and offline generation of prior semantic description text for the region.

[0027] The overall structure of this embodiment includes an offline text prior generation module, a 3D visual feature extraction branch, a text feature extraction branch, a feature dimension transformation and alignment module (feature alignment module), a cross-attention feature fusion module, a target segmentation branch, a region representation branch, and a loss constraint module. The offline text prior generation module generates and saves sample-level region semantic description text before training or inference, and does not participate in online backpropagation. The 3D visual feature extraction branch, text feature extraction branch, cross-attention fusion module, target segmentation branch, and region representation branch constitute an online multi-task network, and achieve collaborative optimization through constraints from various loss functions.

[0028] Acquiring three-dimensional medical imaging data of the prostate (i.e.) Figure 1 After processing the original 3D images, the 3D image preprocessing unit performs resampling, cropping, spatial orientation unification, and standard normalization on the input images to ensure consistency in voxel spacing, grayscale distribution, and input size across different cases. To obtain the input format required for the Visual Language Model (VLM), two-dimensional slices containing target region information or key anatomical region information are extracted from the preprocessed 3D data (i.e.,...). Figure 1 The 2D slices are then adaptively windowed to map to the standard 8-bit grayscale range. The processed 2D slices, along with specific medical prompts, are input into a pre-trained large-scale visual language model, LLaVA-Med, to generate semantic descriptions of the target region's boundary features, texture distribution, intensity variations, and spatial relationships. Figure 1 (Medical text descriptions in the text). The text mapping storage unit is indexed by case number or slice number to construct a static text mapping file containing descriptions corresponding to all samples, which can be called by subsequent text feature extraction branches.

[0029] The second step is to construct a dual-branch feature extraction network.

[0030] like Figure 2 As shown, during the network training phase, the three-dimensional medical image tensor and the corresponding regional semantic description text of the same sample are read synchronously.

[0031] The dual-branch feature extraction network includes: 3D visual feature extraction branch: Employing the 3D encoder of nnU-Net (i.e. Figure 2The nnU-Net 3D encoder serves as the image encoder. This branch consists of an input convolutional layer, multiple 3D encoding stages, and a feature projection layer. Each encoding stage includes a 3D convolutional layer, a normalization layer, a non-linear activation layer, and a downsampling layer, used to progressively extract multi-scale features from shallow edge textures to deep 3D spatial semantics. The 3D feature map output by the encoder is further processed by global average pooling and fully connected linear layers to obtain a 768-dimensional 3D visual feature vector (Image Embedding). ).

[0032] The 3D visual feature map output by the 3D visual encoder can be denoted as: It has two subsequent processing paths: In the first path, for Global average pooling is performed in the spatial dimension, and then mapped through a fully connected linear layer to obtain a 3D visual feature vector. Used with text feature vectors Calculate the alignment loss; in the second path, Expanding in the spatial dimension yields a sequence of spatial visual features. It is used to generate key vectors and value vectors in the cross-attention fusion module.

[0033] Text Feature Extraction Branch: The pre-trained language model BERT is used as the text feature extractor (TextEncoder). This branch includes a text segmentation unit, word embedding and position embedding units, a Transformer encoding layer, and a linear projection layer. The text segmentation unit converts the semantic description text of regions into a token sequence. The BERT encoder extracts contextual semantic features, and the linear projection layer maps the [CLS] position output or average pooled text semantic features into a 768-dimensional text feature vector (Text Embedding). During training, the parameters of this branch are locked and do not participate in gradient updates, thus blocking the gradient flow during backpropagation (i.e., Figure 2 The gradient truncation shown makes it serve only as a semantic scale for unidirectional guidance of visual feature alignment.

[0034] To ensure that the two types of features can be aligned for loss calculation and cross-attention fusion, the feature dimension transformation and alignment module transforms the 3D visual feature vector. and text feature vectors The images are uniformly mapped to the same dimension, and the image features and text features of the same case are kept in a one-to-one correspondence during batch training.

[0035] The third step is to construct a feature alignment module based on alignment learning.

[0036] To enable the visual encoder to adaptively extract features that conform to the semantics of medical text, an alignment loss function is introduced at the output of the feature extraction module. The constraint is achieved by calculating the cosine similarity between the 3D visual feature vector obtained through global average pooling and linear projection and the text feature vector in the embedding space. The calculation formula is as follows: ; in, It is a three-dimensional visual feature vector. For text feature vectors, Representing vectors Norm, i.e. During backpropagation, the gradient of this loss function is used only to update the parameters of the 3D visual encoder, thereby achieving alignment across modal feature spaces.

[0037] First, analyze the three-dimensional visual feature vectors respectively. and text feature vectors conduct Normalize the data and then calculate the cosine similarity between the two samples. The more consistent the visual features of the same sample are with the text prior in the semantic direction, the closer the cosine similarity is to 1, and the smaller the alignment loss. This is achieved by minimizing... Without updating the BERT parameters, the network enables the 3D visual encoder to gradually extract visual representations consistent with the semantics of medical text, such as "boundary, texture, intensity variation, and spatial relationship," thereby reducing the semantic gap between image features and text features.

[0038] Step 4: Construct the cross-attention feature fusion module (Fusion Module).

[0039] like Figure 3 As shown, the text feature vector will be used as a semantic reference. With 3D visual feature map The spatial visual feature sequence obtained by unfolding The data is fed into a cross-attention network for deep interaction. Specifically, the text feature vectors are... Mapped to query vector (Query, ), spatial visual feature sequence Mapped to a key vector (Key, ) and value vector (Value, By calculating the scaled dot product attention and then normalizing it using Softmax, spatial attention weights guided by text semantics are generated. Their calculation form can be expressed as: ; in, As the feature dimension. Based on the spatial attention weights, the spatial visual feature sequence Visual features at different spatial locations are weighted and modulated, and the weighted and modulated spatial visual feature sequence is restored to a three-dimensional feature map; subsequently, it is compared with the three-dimensional visual feature map. The fusion process yields three-dimensional multimodal fusion features. This process enhances the spatial region response corresponding to the semantic description text of the region, improving the relevance and effectiveness of the fusion features.

[0040] Step 5: Construct a multi-task downstream prediction network and calculate the overall loss function.

[0041] The multimodal fusion features obtained in step four are simultaneously input into two parallel downstream branches: Target segmentation branch: Employing the nnU-Net 3D decoder, image resolution is restored through progressive upsampling, deconvolution, or transposed convolution operations. Skip connections at the corresponding scale from the 3D vision encoder are then used to supplement boundary and texture details. Finally, a 3D pixel-level prediction mask of the target region is output after a 1×1×1 convolutional layer and a Softmax activation function. Segmentation loss is also calculated. ).

[0042] The segmentation loss can be a weighted combination of Dice loss and cross-entropy loss, expressed as: ; in, For the first The probability that an individual element is predicted to be the target region. For the first The labeled value corresponding to the individual element. To prevent constants with zero denominators or abnormal logarithmic calculations, and These are the weight coefficients for Dice loss and cross-entropy loss, respectively.

[0043] Region Representation Branch: The 3D multimodal fusion features are input to the region representation branch, which includes a global average pooling layer, a fully connected layer, a nonlinear activation layer, and a multilayer perceptron (MLP). It outputs the region representation result and calculates the region representation loss. ).

[0044] In this embodiment, the region representation result is the attribute confidence score of the target region, which represents the probability that the region belongs to a preset category. The region representation loss uses cross-entropy loss, expressed as: ; in, The confidence score output by the region characterization branch represents the predicted probability that the target region belongs to a preset category or satisfies a preset attribute. This corresponds to the actual label, and its value can be either 0 or 1. A constant to prevent abnormal logarithmic calculations.

[0045] Finally, a composite network total loss function incorporating multi-task prediction and modality alignment is constructed: ; in, , , To adjust the hyperparameters for different task weights, assign them during the initial training phase. Larger weights are used to prioritize the alignment of image and text features, and these weights are gradually increased as the network converges. , To improve the accuracy of specific segmentation and region representation.

[0046] During the reverse propagation process, and The gradient will flow through the downstream decoder, fusion module and back to the visual encoder, while The gradient is used only to update the visual encoder, enabling the visual encoder to update its parameters under the constraints of segmentation, region representation, and text semantic alignment. Thus, joint training of the visual encoder, fusion module, and downstream task branches is achieved without updating the text encoder parameters.

[0047] This invention addresses the problems of excessive memory consumption and difficulty in performing intensive pixel-level predictions (such as 3D segmentation) when directly processing 3D medical images using large Visual Language Models (VLMs). It proposes a multimodal architecture that integrates offline text prior extraction with an online 3D network, innovatively employing a decoupling strategy. In the offline stage, VLM is used to extract high-level semantic descriptions of 2D slices; these descriptions are then used as prior knowledge input to the text encoder during online training. This architecture achieves the introduction and utilization of semantic information from large models into a lightweight 3D medical segmentation network without significantly increasing the computational complexity and hardware resource requirements of the main network.

[0048] To address the issues of feature redundancy and lack of spatial directionality resulting from simple concatenation or addition of traditional multimodal features, this invention proposes a multimodal feature fusion network based on a cross-attention mechanism, designing a text-guided cross-attention fusion module. This module uses text feature vectors as query vectors for dynamic retrieval within a three-dimensional visual feature space (Key and Value). By adaptively allocating attention weights, the network enhances its response to local regions in the image that match the text description, achieving deep fusion of cross-modal features.

[0049] This invention addresses the issues of semantic differences and insufficient cross-modal consistency between visual and textual features by introducing a cross-modal feature alignment loss function to achieve deep feature-level integration. Alignment constraints based on cosine similarity are introduced at the output of the 3D visual encoder. During backpropagation, the locked text feature vector is used as a semantic reference to guide and update the parameters of the visual encoder in one direction, so that the visual features and the text prior maintain semantic consistency in the embedding space, thereby enhancing the ability of visual features to express high-level semantic information and improving the subsequent fusion and prediction effects.

[0050] This invention addresses the lack of spatial interpretability in single-region representation networks by constructing a multi-task composite loss function for the collaborative optimization of "target segmentation + region representation". This multi-task composite loss function is a combined loss function integrating region representation, segmentation, and multimodal alignment. This composite optimization framework improves the accuracy of target region boundary prediction and the stability of region representation results, ensures stable backpropagation and collaborative convergence of multi-task gradients under a unified network architecture, and enhances the reliability and interpretability of the model in auxiliary image feature quantization.

[0051] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion, characterized in that, Includes the following steps: Three-dimensional medical imaging data of the prostate are acquired and preprocessed, and semantic description text of the region is generated offline based on the preprocessed data. A dual-branch feature extraction network is constructed to extract three-dimensional visual feature maps from the three-dimensional medical image data, and the three-dimensional visual feature maps are mapped to three-dimensional visual feature vectors; text feature vectors are extracted from the semantic description text of the region based on the dual-branch feature extraction network. During training, the parameters of the text feature extraction branch are locked to block the gradient flow during backpropagation; the alignment loss is calculated based on the 3D visual feature vector and the text feature vector; during the backpropagation of the alignment loss, the parameters of the text feature extraction branch are locked, and the gradient of the alignment loss is backpropagated to the 3D visual feature extraction branch to constrain the 3D visual feature vector to align with the text feature vector. The text feature vector is fused with the spatial visual feature sequence obtained by expanding the three-dimensional visual feature map through cross-attention to obtain a three-dimensional multimodal fusion feature that preserves the spatial structure. The three-dimensional multimodal fusion feature is then used for both target segmentation and region representation to obtain three-dimensional pixel-level prediction mask and region representation results.

2. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 1, characterized in that, The process of generating region semantic description text includes: After resampling and standard normalization of the three-dimensional medical image, two-dimensional slices containing target region information are extracted. Adaptive windowing is performed on the two-dimensional slices to map them to the standard grayscale range. The processed two-dimensional slices and medical prompts are input into a pre-trained large-scale visual language model to generate regional semantic description text about the boundary features, texture distribution, intensity changes and spatial relationships of the target region, and construct the corresponding static text mapping file.

3. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 1, characterized in that, The dual-branch feature extraction network includes a 3D visual feature extraction branch and a text feature extraction branch.

4. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 3, is characterized in that, The three-dimensional visual feature extraction branch uses the nnU-Net three-dimensional encoder to extract multi-scale features of the three-dimensional medical image data through continuous convolution and downsampling operations, and outputs a three-dimensional visual feature map. The text feature extraction branch uses a pre-trained BERT model to segment and encode the semantic description text of the region, and outputs the text feature vector.

5. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 1, characterized in that, The formula for calculating the alignment loss is: ； in, For alignment loss, For visual feature vectors, This is the text feature vector.

6. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 1, characterized in that, The text feature vector is fused with the spatial visual feature sequence obtained by unfolding the 3D visual feature map through cross-attention, including: The text feature vector is linearly mapped to a query vector, and the spatial visual feature sequence obtained by expanding the three-dimensional visual feature map is linearly mapped to a key vector and a value vector. The dot product of the query vector and the key vector is calculated and divided by the square root of the dimension. After Softmax normalization, it is multiplied by the value vector to obtain the cross-attention output feature. The cross-attention output feature is fused with the three-dimensional visual feature map to obtain the three-dimensional multimodal fusion feature.

7. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 1, characterized in that, Using the three-dimensional multimodal fusion features for target segmentation includes: The three-dimensional multimodal fusion features are input into the target segmentation branch, which uses the nnU-Net three-dimensional decoder to restore the image resolution through upsampling and deconvolution operations, outputs the three-dimensional pixel-level prediction mask, and calculates the segmentation loss.

8. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 7, characterized in that, Using the three-dimensional multimodal fusion features for region representation includes: The three-dimensional multimodal fusion features are input into the region representation branch, which outputs the region representation results through a global average pooling layer and a multilayer perceptron, and calculates the region representation loss.

9. The method for prostate image target segmentation and region representation based on large model text prior and cross-attention feature fusion as described in claim 8, characterized in that, The gradients of the segmentation loss and the region representation loss flow through the target segmentation branch, the region representation branch, the cross-attention fusion module, and back to the 3D visual feature extraction branch.

Images