A prostate intervention guidance method, system, storage medium and electronic device based on PSMA PET, MRI and TRUS multimodal fusion

Abstract

The present invention proposes a prostate intervention guidance method, system, storage medium and electronic device based on PSMA PET, MRI and TRUS multimodal fusion, including: obtaining preoperative MRI images and PSMA PET / CT images, and marking prostate lesions; registering PSMA PET images to the coordinate system corresponding to MRI; registering TRUS images obtained during surgery to the coordinate system corresponding to MRI images to obtain MRI-PSMA PET-TRUS fusion images; based on the fused images, mapping the annotation results of prostate lesions to TRUS images to guide prostate interventional surgery. The present invention statically fuses the traditional preoperative MRI and PSMA PET, and then fuses them with the intraoperative TRUS in real time, combines the multimodal image features, increases the reliability of the image lesion area, plays a positive role in improving the accuracy of interventional surgery, and reduces unnecessary damage.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a prostate intervention guidance method, system, storage medium and electronic device based on PSMA PET, MRI and TRUS multi-modal fusion. Background Art

[0002] Medical image registration and fusion is a commonly used technology in medical image analysis. It converts the coordinates of one set of image sequences into another set of image sequences so that the corresponding positions match and obtain a registered and fused image sequence.

[0003] At present, the precise diagnosis and treatment of prostate diseases faces the great challenge of multimodal medical imaging fusion. Different imaging methods such as MRI, CT and PSMA PET have their own advantages. Among them, MRI (Magnetic Resonance Imaging, multi-parameter nuclear magnetic resonance) is considered to be the standard traditional imaging method for prostate cancer diagnosis at this stage due to its high sensitivity and specificity; PSMA (prostate specific membrane antigen) is expressed much more transmembrane in prostate epithelial cells than in other tumor cells and normal tissues, so it has been developed as a good target for molecular imaging of prostate cancer; at the same time, PSMA PET / CT has gradually become a new clinical diagnostic imaging basis for prostate cancer.

[0004] However, a single modality cannot fully reflect the anatomical structure and functional status of the prostate. The current limitation of interventional surgery guided by fusion of MRI and TRUS is that the sensitivity of MRI to prostate cancer is lower than that of PSMA PET, while interventional surgery guided by fusion of PSMA PET and TRUS cannot utilize the high-resolution display of the prostate anatomical structure by MRI.

[0005] Therefore, how to accurately align and fuse these preoperative static images with intraoperative real-time ultrasound TRUS images is a complex technical problem.

[0006] In addition, the deformation and displacement of the prostate during surgery also make real-time registration difficult. If the precise fusion of multimodal images cannot be achieved, the accuracy of puncture biopsy and the precision of local treatment will be seriously affected.

[0007] Therefore, how to overcome the heterogeneity between medical images of different modalities, achieve real-time and accurate registration of preoperative MRI / PET and intraoperative TRUS, and organically integrate multimodal image information to guide clinical operations is a key technical problem that needs to be solved urgently. Summary of the invention

[0008] The purpose of the present invention is to solve the problem of real-time and accurate registration of preoperative MRI / PET and intraoperative TRUS, and to propose a prostate intervention guidance method, system, storage medium and electronic device based on multimodal fusion of PSMA PET, MRI and TRUS.

[0009] The technical solution of the present invention is:

[0010] The present invention provides a prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion, comprising:

[0011] S1. Obtain preoperative MRI images and PSMA PET / CT images, and mark prostate lesions on MRI or PSMA PET images;

[0012] S2, using the MRI image as the main coordinate system, and registering the PSMA PET image to the coordinate system corresponding to the MRI image through a registration method based on a combination of image features and organ contours, to obtain an MRI-PSMA PET fusion image;

[0013] S3, obtaining intraoperative TRUS image sequences, extracting the prostate contour through a deep image segmentation network, and registering the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain an MRI-PSMA PET-TRUS fusion image;

[0014] S4. Based on the MRI-PSMA PET-TRUS fusion image, the annotation results of the prostate lesions in S1 are mapped to the TRUS image to guide prostate interventional surgery.

[0015] Furthermore, in S2, the registration method based on the combination of image features and organ contours includes:

[0016] The pre-trained deep image segmentation network is used to obtain the prostate contours on MRI images and CT images respectively;

[0017] The optimal transfer matrix from CT image to MRI image is obtained by minimizing the formula;

[0018] Based on the consistency of imaging coordinates of the PSMA PET image and the CT image, the CT image and the PSMA PET image are mapped to the coordinate system corresponding to the MRI image through the optimal transfer matrix.

[0019] Furthermore, the method of obtaining the optimal transfer matrix from the CT image to the MRI image by minimizing the formula includes:

[0020] Based on the feature similarity parameters of the MRI image and the registered CT image, as well as the coincidence parameters of the MRI prostate contour and the registered CT prostate contour, an optimization objective function is established to solve the optimal transfer matrix from the CT image to the MRI image. ;

[0021]

[0022] in: is the transfer matrix parameter, is a CT image, For MRI images, is the transfer matrix from CT image to MRI image; and It is the prostate contour on MRI and CT images; is the mutual information parameter between images, which is used to measure the feature similarity between MRI images and registered CT images. is the Dice similarity parameter, which is used to measure the overlap of the prostate contours between the MRI image and the registered CT image. is the ratio between the mutual information parameter and the Dice similarity parameter.

[0023] Furthermore, in the MRI-PSMA PET fusion image, the PSMA PET image is superimposed and displayed on the MRI image in a pseudo-color manner.

[0024] Furthermore, in S3, the point cloud registration includes:

[0025] Acquiring a TRUS image sequence acquired by scanning from the base to the tip of the prostate with an ultrasound probe, wherein the ultrasound probe is equipped with an electromagnetic sensor to extract the electromagnetic spatial coordinates of each point in the image;

[0026] Segment the TRUS image to extract the prostate contour point set;

[0027] Segment the MRI image to extract the prostate contour point set;

[0028] By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved and the electromagnetic coordinate system of the TRUS image is transformed into the MRI image coordinate system.

[0029] Furthermore, by minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the following formula is used to solve the optimal rotation and translation matrix:

[0030]

[0031] in: and Represent the rotation matrix and translation vector in space respectively, Indicates the number of the prostate contour point in the TRUS image. Represents the total number of prostate contour points in TRUS images, , represents the set of prostate contour points in TRUS images, , Contains TRUS electromagnetic space coordinates; Indicates the number of the prostate contour points in the MRI image. Represents the total number of prostate contour points in the MRI image, , represents the set of prostate contour points in MRI images, , Contains the MRI space coordinates.

[0032] Furthermore, in the MRI-PSMA PET-TRUS fusion image, the TRUS image provides real-time anatomical data information, the MRI image provides anatomical structure and lesion information, and the PSMA PET image provides prostate cancer targeting information.

[0033] Furthermore, the prostate interventional surgery includes prostate puncture biopsy and prostate local lesion treatment, and the biopsy needle or treatment needle is guided to the lesion area according to the lesion information in the fused image.

[0034] Furthermore, the deep image segmentation network adopts a UNet structure and is trained with manually annotated prostate contour data to segment the prostate contour in MRI images, CT images and TRUS images.

[0035] A prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion, based on the described method, comprising:

[0036] The preoperative registration and fusion module is used to obtain preoperative MRI images, PSMA PET images and CT images, and to mark prostate lesions on MRI or PSMA PET images. The MRI image is used as the main coordinate system, and the PSMA PET image is registered to the coordinate system corresponding to the MRI image through a registration method based on image features and organ contours to obtain an MRI-PSMA PET fusion image.

[0037] The intraoperative registration and fusion module is used to obtain the intraoperative TRUS image sequence, extract the prostate contour through the deep image segmentation network, and register the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain the MRI-PSMA PET-TRUS fusion image;

[0038] The intraoperative mapping guidance module is used to map the annotation results of prostate lesions onto TRUS images through MRI-PSMA PET-TRUS fusion images to guide prostate interventional surgery.

[0039] A computer-readable storage medium stores a computer program, which, when executed, implements the prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion.

[0040] An electronic device comprises: one or more processors; the processors are used to store one or more programs; when the one or more programs are executed by the one or more processors, the prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion is implemented.

[0041] Beneficial effects of the present invention:

[0042] The present invention statically fuses the traditional preoperative MRI and PSMA PET, and then fuses them with the intraoperative TRUS (Transrectal Ultrasound) in real time, combining multimodal image features to increase the reliability of the image lesion area, which has a positive effect on improving the accuracy of interventional surgeries, such as prostate cancer puncture biopsy and focal treatment of prostate cancer, and reduces unnecessary damage.

[0043] The present invention provides a prostate intervention guidance method, system, storage medium and electronic device based on PSMA PET, MRI and TRUS multimodal fusion. The method first obtains MRI images and PSMA PET / CT images, extracts the prostate contour through a deep learning segmentation network, and then performs MRI and CT / PET image registration based on mutual information similarity and contour coincidence. Then, an intraoperative TRUS image sequence is obtained, and the prostate contour is also extracted. The TRUS image is registered to the coordinate system corresponding to the MRI image through a point cloud registration algorithm to achieve real-time registration and fusion of MRI-PSMA PET-TRUS. Finally, a puncture biopsy or local treatment of the prostate lesion is performed under the guidance of the fused multimodal image.

[0044] The present invention solves the problem of registration and fusion between medical images of different modalities, especially the accurate registration of preoperative MRI, PET / CT and intraoperative real-time TRUS images. Through the organic combination of multimodal image information, the advantages of various imaging methods can be fully utilized to provide more comprehensive imaging information for the precise positioning and targeted treatment of prostate lesions, which helps to improve the accuracy and safety of diagnosis and treatment.

[0045] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0047] Figure 1 A flow chart according to an embodiment of the present invention is shown.

[0048] Figure 2 A schematic diagram showing the registration of PSMA PET and MRI to obtain an MRI-PSMA PET fusion image according to an embodiment of the present invention.

[0049] Figure 3 A schematic diagram showing the registration of a TRUS image with an MRI to obtain an MRI-PSMA PET-TRUS fusion image according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0050] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0051] Example 1

[0052] Figure 1 A flow chart according to an embodiment of the present invention is shown.

[0053] like Figure 1 As shown, the present invention provides a prostate intervention guidance method based on PSMA PET, MRI and TRUS multi-modality fusion, comprising:

[0054] S1. Obtain preoperative MRI images and PSMA PET / CT images, and annotate prostate lesions on MRI or PSMA PET images. The relevant terms are explained as follows:

[0055] PSMA (prostate specific membrane antigen) is a prostate-specific membrane antigen. The transmembrane expression of PSMA in prostate epithelial cells is much higher than that in other tumor cells and normal tissues. Therefore, it has been developed as a good target for molecular imaging of prostate cancer.

[0056] PET (Positron Emission Tomography) positron emission tomography;

[0057] CT (Computed Tomography) computed tomography;

[0058] PET / CT, positron emission tomography, integrates PET and CT on one device. One imaging can simultaneously obtain CT anatomical images and PET functional metabolic images, and the two images have the same coordinate system.

[0059] PSMA PET / CT uses PET / CT to observe changes in PSMA to locate, schedule, and quantify prostate cancer.

[0060] S2, using the MRI image as the main coordinate system, and registering the PSMA PET image to the coordinate system corresponding to the MRI image through a registration method based on a combination of image features and organ contours, to obtain an MRI-PSMA PET fusion image;

[0061] On the one hand, since PSMA PET and CT are imaged in the same coordinate system, MRI is used as the main image and the main data image in the subsequent images. Images of other modalities are mapped to the coordinate space of MRI, and the conversion matrix from CT to MRI is obtained. This conversion matrix is ​​applied to the PSMA PET image, and the registration of MRI and PSMA PET can be completed (CT only plays the role of registration).

[0062] On the other hand, since both MRI and CT contain the same anatomical structure, the present invention proposes a registration algorithm based on the combination of image features and organ contours by using the segmentation results of the prostate contours on MRI20 and CT and combining the similarity of image structures across modalities, including:

[0063] Using pre-trained deep image segmentation networks, such as UNet, the prostate contours on MRI images and CT images are obtained respectively;

[0064] Based on the feature similarity parameters of the MRI image and the registered CT image, as well as the coincidence parameters of the MRI prostate contour and the registered CT prostate contour, an optimization objective function is established to solve the optimal transfer matrix from the CT image to the MRI image. ;

[0065]

[0066] in: is the transfer matrix parameter, is a CT image, For MRI images, is the transfer matrix from CT image to MRI image; and It is the prostate contour on MRI and CT images; is the mutual information parameter between images, which is used to measure the feature similarity between MRI images and registered CT images. is the Dice similarity parameter, which is used to measure the overlap of the prostate contours between the MRI image and the registered CT image. is the ratio between the mutual information parameter and the Dice similarity parameter.

[0067] Based on the consistency of the imaging coordinates of the PSMA PET image and the CT image, the CT image and the PSMA PET image are mapped to the coordinate system corresponding to the MRI image through the optimal transfer matrix, the PSMA PET is displayed in pseudo color, and then fused with the MRI, as shown in FIG. Figure 2 , which is a schematic diagram of obtaining an MRI-PSMA PET fusion image by registering PSMA PET with MRI; wherein a represents an MRI image, b represents the registered PSMA PET image, and c represents the MRI-PSMA PET fusion image;

[0068] S3. Obtain intraoperative TRUS image sequences, extract the prostate contour through a deep image segmentation network, and register the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain the MRI-PSMA PET-TRUS fusion image, which specifically includes:

[0069] Acquiring a TRUS image sequence acquired by scanning from the base to the tip of the prostate with an ultrasound probe, wherein the ultrasound probe is equipped with an electromagnetic sensor to extract the electromagnetic spatial coordinates of each point in the image;

[0070] Segment TRUS images to extract prostate contour point sets , ,in Contains TRUS electromagnetic space coordinates; Indicates the number of the prostate contour point in the TRUS image. It represents the total number of prostate contour points in TRUS images;

[0071] Segment MRI images to extract prostate contour point sets , ,in Contains MRI space coordinates; Indicates the number of the prostate contour points in the MRI image. Represents the total number of prostate contour points in MRI images;

[0072] By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the following formula is used to solve the optimal rotation and translation matrix so that the prostate contour on TRUS matches the prostate contour on MRI as much as possible:

[0073]

[0074] in: and Respectively represent the rotation matrix and translation vector in space;

[0075] By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved. ;

[0076] By optimal rotation and translation matrix By mapping the electromagnetic coordinate system where TRUS is located to the MRI coordinate system, a fusion image of MRI, PSMA PET and TRUS can be formed, such as Figure 3 shown.

[0077] Specifically, in the MRI-PSMA PET-TRUS fusion image, the TRUS image provides real-time anatomical data information, the MRI image provides anatomical structure and lesion information, and the PSMA PET image provides prostate cancer targeting information.

[0078] Based on this, the lesions can be marked on MRI or PSMA PET images, confirmed again after image fusion, and then mapped onto ultrasound images to improve the accuracy and efficiency of interventional surgery guidance (puncture biopsy, focal treatment, etc.).

[0079] Specifically, the prostate interventional surgery includes prostate puncture biopsy and prostate local lesion treatment, and the biopsy needle or treatment needle is guided to the lesion area according to the lesion information in the fused image.

[0080] In this embodiment, the multimodal image features are combined to increase the reliability of the image lesion area, which has a positive effect on improving the accuracy of interventional surgeries, such as puncture biopsy of prostate cancer and focal treatment of prostate cancer, and can effectively reduce unnecessary damage.

[0081] Specifically, the deep image segmentation network adopts a UNet structure and is trained with manually annotated prostate contour data to segment the prostate contour in MRI images, CT images, and TRUS images.

[0082] Example 2

[0083] The present invention provides a prostate intervention guidance system based on PSMA PET, MRI and TRUS multi-modal fusion, based on the described method, comprising:

[0084] The preoperative registration and fusion module is used to obtain preoperative MRI images, PSMA PET images and CT images, and to mark prostate lesions on MRI or PSMA PET images. The MRI image is used as the main coordinate system, and the PSMA PET image is registered to the coordinate system corresponding to the MRI image through a registration method based on image features and organ contours to obtain an MRI-PSMA PET fusion image.

[0085] The intraoperative registration and fusion module is used to obtain the intraoperative TRUS image sequence, extract the prostate contour through the deep image segmentation network, and register the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain the MRI-PSMA PET-TRUS fusion image;

[0086] The intraoperative mapping guidance module is used to map the annotation results of prostate lesions onto TRUS images through MRI-PSMA PET-TRUS fusion images to guide prostate interventional surgery.

[0087] Example 3

[0088] A computer-readable storage medium stores a computer program, which, when executed, implements the prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion.

[0089] Example 4

[0090] An electronic device comprises: one or more processors; the processors are used to store one or more programs; when the one or more programs are executed by the one or more processors, the prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion is implemented.

[0091] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion, characterized in that: include: The preoperative registration and fusion module is used to obtain preoperative MRI images, PSMA PET images and CT images, and to mark prostate lesions on MRI or PSMA PET images. The MRI image is used as the main coordinate system, and the PSMA PET image is registered to the coordinate system corresponding to the MRI image through a registration method based on image features and organ contours to obtain an MRI-PSMA PET fusion image. The intraoperative registration and fusion module is used to obtain the intraoperative TRUS image sequence, extract the prostate contour through the deep image segmentation network, and register the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain the MRI-PSMAPET-TRUS fusion image; The intraoperative mapping guidance module is used to map the annotation results of prostate lesions onto TRUS images through MRI-PSMA PET-TRUS fusion images to guide prostate interventional surgery; The registration method based on the combination of image features and organ contours includes: The pre-trained deep image segmentation network is used to obtain the prostate contours on MRI images and CT images respectively; The optimal transfer matrix from CT image to MRI image is obtained by minimizing the formula; Based on the consistency of imaging coordinates of the PSMA PET image and the CT image, the CT image and the PSMA PET image are mapped to the coordinate system corresponding to the MRI image through the optimal transfer matrix; The method of obtaining the optimal transfer matrix from the CT image to the MRI image by minimizing the formula includes: Based on the feature similarity parameters of the MRI image and the registered CT image, as well as the coincidence parameters of the MRI prostate contour and the registered CT prostate contour, an optimization objective function is established to solve the optimal transfer matrix from the CT image to the MRI image. ; ； in: is the transfer matrix parameter, is a CT image, For MRI images, is the transfer matrix from CT image to MRI image; and It is the prostate contour on MRI and CT images; is the mutual information parameter between images, which is used to measure the feature similarity between MRI images and registered CT images. is the Dice similarity parameter, which is used to measure the overlap of the prostate contours between the MRI image and the registered CT image. is the ratio between the mutual information parameter and the Dice similarity parameter.

2. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 1, characterized in that: In the MRI-PSMA PET fusion image, the PSMA PET image is superimposed and displayed on the MRI image in a pseudo-color manner.

3. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 1, characterized in that The point cloud registration comprises: Acquiring a TRUS image sequence acquired by scanning from the base to the tip of the prostate with an ultrasound probe, wherein the ultrasound probe is equipped with an electromagnetic sensor to extract the electromagnetic spatial coordinates of each point in the image; Segment the TRUS image to extract the prostate contour point set; Segment the MRI image to extract the prostate contour point set; By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved and the electromagnetic coordinate system of the TRUS image is transformed into the MRI image coordinate system.

4. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 3, characterized in that By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved using the following formula: ； in: and Represent the rotation matrix and translation vector in space respectively, Indicates the number of the prostate contour point in the TRUS image. Represents the total number of prostate contour points in TRUS images, , represents the set of prostate contour points in TRUS images, , Contains electromagnetic space coordinates of TRUS images; Indicates the number of the prostate contour points in the MRI image. , represents the set of prostate contour points in MRI images, , Contains the MRI image space coordinates, Represents the total number of prostate contour points in the MRI image.

5. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 1, characterized in that: In the MRI-PSMA PET-TRUS fusion image, the TRUS image provides real-time anatomical data information, the MRI image provides anatomical structure and lesion information, and the PSMA PET image provides prostate cancer targeting information.

6. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 1, characterized in that: The prostate interventional surgery includes prostate puncture biopsy and prostate local lesion treatment. According to the lesion information in the fused image, the biopsy needle or the treatment needle is guided to the lesion area.

7. The prostate intervention guidance system based on PSMA PET, MRI and TRUS multimodal fusion according to claim 1, characterized in that: The deep image segmentation network adopts a UNet structure and is trained by manually annotated prostate contour data to segment the prostate contour in MRI images, CT images and TRUS images.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, a prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion is implemented, including: S1. Obtain preoperative MRI images and PSMA PET / CT images, and mark prostate lesions on MRI or PSMA PET images; S2, using the MRI image as the main coordinate system, and registering the PSMA PET image to the coordinate system corresponding to the MRI image through a registration method based on a combination of image features and organ contours, to obtain an MRI-PSMA PET fusion image; S3, obtaining intraoperative TRUS image sequences, extracting the prostate contour through a deep image segmentation network, and registering the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain an MRI-PSMA PET-TRUS fusion image; S4, based on MRI-PSMA PET-TRUS fusion images, the annotation results of prostate lesions in S1 are mapped onto TRUS images to guide prostate interventional surgery; The pre-trained deep image segmentation network is used to obtain the prostate contours on MRI images and CT images respectively; The optimal transfer matrix from CT image to MRI image is obtained by minimizing the formula; Based on the consistency of imaging coordinates of the PSMA PET image and the CT image, the CT image and the PSMA PET image are mapped to the coordinate system corresponding to the MRI image through the optimal transfer matrix; The method of obtaining the optimal transfer matrix from the CT image to the MRI image by minimizing the formula includes: Based on the feature similarity parameters of the MRI image and the registered CT image, as well as the coincidence parameters of the MRI prostate contour and the registered CT prostate contour, an optimization objective function is established to solve the optimal transfer matrix from the CT image to the MRI image. ; ； in: is the transfer matrix parameter, is a CT image, For MRI images, is the transfer matrix from CT image to MRI image; and It is the prostate contour on MRI and CT images; is the mutual information parameter between images, which is used to measure the feature similarity between MRI images and registered CT images. is the Dice similarity parameter, which is used to measure the overlap of the prostate contours between the MRI image and the registered CT image. is the ratio between the mutual information parameter and the Dice similarity parameter.

9. The computer-readable storage medium according to claim 8, wherein: In the MRI-PSMA PET fusion image, the PSMA PET image is superimposed and displayed on the MRI image in a pseudo-color manner.

10. The computer-readable storage medium according to claim 8, characterized in that In S3, point cloud registration includes: Acquiring a TRUS image sequence acquired by scanning from the base to the tip of the prostate with an ultrasound probe, wherein the ultrasound probe is equipped with an electromagnetic sensor to extract the electromagnetic spatial coordinates of each point in the image; Segment the TRUS image to extract the prostate contour point set; Segment the MRI image to extract the prostate contour point set; By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved and the electromagnetic coordinate system of the TRUS image is transformed into the MRI image coordinate system.

11. The computer-readable storage medium according to claim 10, wherein: By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved using the following formula: ； in: and Represent the rotation matrix and translation vector in space respectively, Indicates the number of the prostate contour point in the TRUS image. Represents the total number of prostate contour points in TRUS images, , represents the set of prostate contour points in TRUS images, , Contains electromagnetic space coordinates of TRUS images; Indicates the number of the prostate contour points in the MRI image. , represents the set of prostate contour points in MRI images, , Contains the MRI image space coordinates, Represents the total number of prostate contour points in the MRI image.

12. The computer-readable storage medium according to claim 8, wherein: In the MRI-PSMA PET-TRUS fusion image, the TRUS image provides real-time anatomical data information, the MRI image provides anatomical structure and lesion information, and the PSMA PET image provides prostate cancer targeting information.

13. The computer-readable storage medium according to claim 8, wherein: The prostate interventional surgery includes prostate puncture biopsy and prostate local lesion treatment. According to the lesion information in the fused image, the biopsy needle or the treatment needle is guided to the lesion area.

14. The computer-readable storage medium according to claim 8, wherein: The deep image segmentation network adopts a UNet structure and is trained by manually annotated prostate contour data to segment the prostate contour in MRI images, CT images and TRUS images.

15. An electronic device, characterized in that: include: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, a prostate intervention guidance method based on PSMA PET, MRI and TRUS multimodal fusion is implemented, including: S1. Obtain preoperative MRI images and PSMA PET / CT images, and mark prostate lesions on MRI or PSMA PET images; S2, using the MRI image as the main coordinate system, and registering the PSMA PET image to the coordinate system corresponding to the MRI image through a registration method based on a combination of image features and organ contours, to obtain an MRI-PSMA PET fusion image; S3, obtaining intraoperative TRUS image sequences, extracting the prostate contour through a deep image segmentation network, and registering the TRUS image to the coordinate system corresponding to the MRI image through the point cloud registration method to obtain an MRI-PSMA PET-TRUS fusion image; S4, based on MRI-PSMA PET-TRUS fusion images, the annotation results of prostate lesions in S1 are mapped onto TRUS images to guide prostate interventional surgery; The pre-trained deep image segmentation network is used to obtain the prostate contours on MRI images and CT images respectively; The optimal transfer matrix from CT image to MRI image is obtained by minimizing the formula; Based on the consistency of imaging coordinates of the PSMA PET image and the CT image, the CT image and the PSMA PET image are mapped to the coordinate system corresponding to the MRI image through the optimal transfer matrix; The method of obtaining the optimal transfer matrix from the CT image to the MRI image by minimizing the formula includes: Based on the feature similarity parameters of the MRI image and the registered CT image, as well as the coincidence parameters of the MRI prostate contour and the registered CT prostate contour, an optimization objective function is established to solve the optimal transfer matrix from the CT image to the MRI image. ; ； in: is the transfer matrix parameter, is a CT image, For MRI images, is the transfer matrix from CT image to MRI image; and It is the prostate contour on MRI and CT images; is the mutual information parameter between images, which is used to measure the feature similarity between MRI images and registered CT images. is the Dice similarity parameter, which is used to measure the overlap of the prostate contours between the MRI image and the registered CT image. is the ratio between the mutual information parameter and the Dice similarity parameter.

16. The electronic device according to claim 15, characterized in that: In the MRI-PSMA PET fusion image, the PSMA PET image is superimposed and displayed on the MRI image in a pseudo-color manner.

17. The electronic device according to claim 15, characterized in that In S3, point cloud registration includes: Acquiring a TRUS image sequence acquired by scanning from the base to the tip of the prostate with an ultrasound probe, wherein the ultrasound probe is equipped with an electromagnetic sensor to extract the electromagnetic spatial coordinates of each point in the image; Segment the TRUS image to extract the prostate contour point set; Segment the MRI image to extract the prostate contour point set; By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved and the electromagnetic coordinate system of the TRUS image is transformed into the MRI image coordinate system.

18. The electronic device according to claim 17, characterized in that: By minimizing the distance between the prostate contour point sets of the MRI image and the TRUS image, the optimal rotation and translation matrix is ​​solved using the following formula: ； in: and Represent the rotation matrix and translation vector in space respectively, Indicates the number of the prostate contour point in the TRUS image. Represents the total number of prostate contour points in TRUS images, , represents the set of prostate contour points in TRUS images, , Contains electromagnetic space coordinates of TRUS images; Indicates the number of the prostate contour points in the MRI image. , represents the set of prostate contour points in MRI images, , Contains the MRI image space coordinates, Represents the total number of prostate contour points in the MRI image.

19. The electronic device according to claim 15, characterized in that: In the MRI-PSMA PET-TRUS fusion image, the TRUS image provides real-time anatomical data information, the MRI image provides anatomical structure and lesion information, and the PSMA PET image provides prostate cancer targeting information.

20. The electronic device according to claim 15, characterized in that: The prostate interventional surgery includes prostate puncture biopsy and prostate local lesion treatment. According to the lesion information in the fused image, the biopsy needle or the treatment needle is guided to the lesion area.

21. The electronic device according to claim 15, characterized in that: The deep image segmentation network adopts a UNet structure and is trained by manually annotated prostate contour data to segment the prostate contour in MRI images, CT images and TRUS images.

Images