Intelligent anesthetization system for transperineal prostate puncturing based on multimodal medical images

Abstract

SOLUTION: The invention provides an intelligent anesthetization system for transperineal prostate puncturing based on multimodal medical images comprising: a coordinate system defining module; a MRI sequence image mark; a 3D reconstruction module; an ultrasonic data extraction module; a positioning module; and an anesthetic planning module. The system transforms images in a preoperative nuclear magnetic imaging coordinate system as well as a target area of anesthetization, into a real time electromagnetic coordinate system tthrough positioning of the multi modal medical images, thereby, realizing comprehension fusion of the target in the nuclear magnetic resonance image and an ultrasonic image.EFFECT: The invention can be used for medical teaching or surgical guidance and can help to achieve precise positioning of anesthetized area for transperineal prostate biopsy.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent anesthesia system for ultrasonic transperineal prostate biopsy needle based on multimodal medical images. [Background technology]

[0002] With the development of science and technology and the popularization of health education, men around the world are increasingly concerned about prostate disease. Prostate cancer is also the second most common cancer among men in the world, and has a serious impact on men's healthy lives. Doctors usually use prostate-specific antigen (PSA) testing to effectively screen for prostate cancer at an early stage, and arrange for multi-parameter magnetic resonance testing and biopsy puncture for patients with abnormal PSA, so as to formulate a treatment plan in a timely manner.

[0003] Therefore, how to complete biopsy puncture efficiently and improve the positive rate of biopsy results is a goal that the medical community is constantly pursuing. Prostate biopsy puncture tests are mainly divided into two categories:

[0004] The first is the transrectal puncture method, which has been used for many years, which has low requirements for anesthesia and is easy to operate, but has the disadvantage that rectal bacteria can easily enter the bloodstream, which can cause not only infection but also fever and even sepsis. According to statistics, the infection rate of this method reaches 5-7%.

[0005] The second category is the transperineal puncture method, which has been used frequently in recent years. This method is relatively safe, and according to statistics, the infection rate is reduced to 0.075% compared with transrectal puncture, and it can generally be cured in a short period of time. However, this method has high requirements for anesthesia, such as not only accurately identifying the anesthesia area, but also selecting an anesthesia needle of a corresponding length to achieve accurate anesthesia. If the anesthesia effect is poor, the patient will feel very painful during the puncture process and may not be able to perform the operation well. For these reasons, the large-scale promotion of this safe and effective surgical procedure has faced a certain resistance.

[0006] Multiparameter magnetic resonance imaging (mpMRI) has an excellent developing effect on soft tissues, so it is used as an image that must be obtained before prostate puncture. It can relatively accurately identify the area suspected to be a lesion on the nuclear magnetic image, and can also accurately identify the nerve anesthesia area around the prostate. Ultrasound imaging, which has high real-time accuracy, is usually used as the first choice device for puncture, but due to its low image resolution, it is relatively difficult to directly identify the exact anesthesia target area on the ultrasound image.

[0007] Therefore, there is an urgent need for a system that accurately projects the magnetic resonance anesthesia field onto real-time ultrasound images. Summary of the Invention

[0008] The objective of the present invention is to provide an intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images, which is easy to operate and uses ultrasound images to fuse preoperative multi-parameter magnetic resonance images to accurately fuse the anesthesia area and the perineum area into the high-speed real-time ultrasound images, and can recommend intelligent anesthesia tools to realize accurate preoperative anesthesia planning, effectively reducing the difficulty of transperineal puncture and reducing the infection rate of patients. [Means for solving the problem]

[0009] The technical solution of the present invention is: In the present invention, an intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images is provided, which includes a coordinate system definition module, an MRI sequence image mark and 3D reconstruction module, an ultrasound data extraction module, a registration module and an anesthesia planning module.

[0010] The coordinate system definition module defines an electromagnetic coordinate system CSe based on an ultrasound imaging plane formed by an electromagnetic field transmitter, defines a coordinate system Csm of a nuclear magnetic device based on a nuclear magnetic device that captures MRI sequence images, defines a coordinate system CSs based on an electromagnetic sensor bound to the ultrasound probe, and obtains a transformation matrix TMse of the electromagnetic sensor at an arbitrary position within CSs based on the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe.

[0011] The MRI sequence image marking and 3D reconstruction module receives a human prostate MRI sequence image, marks an anesthesia area MRI-VMa and an outer contour area MRI-VMp of the prostate, and reconstructs a 3D anesthesia area and a 3D outer contour area of ​​the prostate based on the position information of the MRI sequence image.

[0012] The ultrasound data extraction module receives human prostate ultrasound data, extracts a set of images covering the bottom to the tip of the prostate from the ultrasound data, and obtains the outer contour region US-VMp of the prostate in the electromagnetic coordinate system Cse in the ultrasound image based on the spatial position information of each frame of the ultrasound image.

[0013] The registration module uses an ICP point cloud registration algorithm to register the MRI-VMp volume data of the outer contour region of the prostate in the coordinate system Csm of the nuclear magnetic device and the US-VMp volume data of the outer contour region of the prostate in the electromagnetic coordinate system Cse of the ultrasound image, thereby obtaining a spatial transformation matrix TMme between the two.

[0014] The anesthesia planning module receives the current ultrasound image acquired by the ultrasound probe, obtains the anesthesia area EMTS-Vma projected in the electromagnetic coordinate system based on the spatial transformation matrix TMme, and fuses it with the current ultrasound image to obtain the anesthesia area in the ultrasound image.

[0015] In addition, the marking specifically involves using a pre-trained prostate segmentation deep learning model to segment the MRI sequence image and marking the segmented anesthesia area VMa and the prostate periphery zone contour area VMp.

[0016] Specifically, the establishment of a prostate segmentation deep learning model involves: Extract transverse and sagittal images from prostate MRI sequence images. Marking and segmenting the anesthesia area and the prostate gland contour area on the transverse and sagittal images; The marked and segmented prostate MRI images are then used as samples to train an artificial neural network model, and a prostate segmentation deep learning model is obtained.

[0017] In addition, the spatial transformation matrix TMse is specifically expressed as follows: JPEG2025013277000002.jpg12170Here, R is a 3x3 matrix, T is a 3x1 vector, and R and T denote the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe.

[0018] In addition, the acquisition of the outer contour area of ​​the prostate, US-VMp, is specifically performed as follows: A set of images covering the prostate from the base to the tip is extracted from the ultrasound data and the number of frames is recorded as N. An image including the prostate organ is extracted, and the outer contour region of the prostate is segmented to obtain the outer contour region US-VMp of the prostate in the ultrasound image, and the coordinates PCe of each point of the outer contour in the electromagnetic coordinate system Cse are obtained using the following formula: JPEG2025013277000003.jpg8170Here, i denotes the frame number, (u, v) denotes the pixel coordinates of the contour point in the ultrasound 2D image, TMsei denotes the transformation matrix displaying the electromagnetic sensor corresponding to the i-th frame image at an arbitrary position of CSs, and TMts denotes the transformation matrix of the relative position between the vertex of the ultrasound imaging plane and the electromagnetic sensor.

[0019] In addition, the alignment is specifically as follows: JPEG2025013277000004.jpg15170 The 3D point cloud corresponding to the contour region MRI-VMp is used as the 3D point cloud of the second group, The two groups of points are processed separately, The ICP point cloud alignment algorithm is used to align the processed 3D point clouds, and the spatial transformation matrix TMme is obtained from the nuclear magnetic device coordinate system CSm to the electromagnetic coordinate system Cse.

[0020] The anesthesia planning module also includes the following steps: Transforming the anesthesia region MRI-VMa in the coordinate system Csm of the nuclear magnetic device into the electromagnetic coordinate system of the current ultrasound image based on the spatial transformation matrix TMme to obtain a projected anesthesia region EMTS-Vma; Obtaining a slice Sa including an anesthesia region contour by obtaining a slice generated by a tangent between the current ultrasound image plane position and the anesthesia region EMTS-Vma volume data; A fusion image of the current ultrasound image and the anesthetized region is obtained by fusing and superimposing the current ultrasound image and the slice Sa including the contour of the anesthetized region, and a step of displaying the anesthetized region on the ultrasound image is performed.

[0021] The system further includes a puncture planning module, the puncture planning module comprising the following steps: Extracting perineum point cloud data in the electromagnetic coordinate system Cse of the ultrasound data and fitting a plane position PP of the perineum as the starting plane position of the puncture needle entering the body, the normal vector of said plane coinciding with the direction of the ultrasound probe entering the human; The steps of calculating the distance from the center point of the anesthesia area EMTS-Vma projected under the current ultrasound image to the puncture starting plane PP and recommending an anesthesia needle of an appropriate length based on the distance from the anesthesia area to the puncture starting plane are performed.

[0022] In addition, adapting the perineum plane position PP Extracting the ultrasound data sequence of the perineum and obtaining a series of coordinate points PPi(x, y, z, 1) on the ultrasound imaging plane, where i denotes the coordinate point number, and x, y, and z denote the coordinate values ​​of the ultrasound imaging plane, respectively; obtaining the true coordinates PPsi of each point in the coordinate system CSs of the electromagnetic sensor by using the following formula: PPsi = TMse × TMts × PPi Here, TMts represents a transformation matrix of the vertices of the ultrasound imaging plane and the relative positions of the electromagnetic sensor; Based on the true coordinates PPsi of each point, the method includes fitting the perineum plane position PP as the puncture starting surface using a discrete point based svd method. ax+by+cz+d=0

[0023] In addition, the following formula is used to calculate the distance from the center point of the projected anesthesia area EMTS-Vma to the puncture start plane PP: d=|ax m +by m +cz m +d| / (a 2 +b 2 +c 2 ) Here, (x m , y m , z m ) denotes the center point of the anesthetized region in the electromagnetic coordinate system CSe of the ultrasound imaging plane, and a, b, c, and d are the plane parameters of PP. Effect of the Invention

[0024] The system of the present invention converts the image and the anesthesia target area in the preoperative nuclear magnetic coordinate system into a real-time electromagnetic coordinate system through the alignment of multimodal medical images, thereby realizing comprehensive fusion of the anesthesia target in the nuclear magnetic image and the ultrasound image, which is used in medical education or surgical guidance and helps to realize accurate positioning of the anesthesia area for transperineal prostate biopsy.

[0025] The present invention realizes intelligent recommendation of anesthesia needle models by calculating and analyzing the distance between the target area and the starting position of the needle entry, making the anesthesia procedure proceed more smoothly and efficiently, and playing an effective role in popularizing the transperineal prostate biopsy procedure.

[0026] Other features and advantages of the present invention are described in detail in the specific embodiments section below. [Brief description of the drawings]

[0027] The above and other objects, features and advantages of the present invention will become more apparent from a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, in which like reference numerals generally represent like parts. [Figure 1] 1 is a framework flow chart for implementing the system of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing the marked surrounding prostate and anesthetized areas in one embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing the perifronatal and anesthetized regions and a three-dimensional reconstruction in one embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram illustrating different coordinate systems during anesthesia navigation in one embodiment of the present invention. [Diagram 5] FIG. 2 is a schematic diagram showing the starting surface of puncture in one embodiment of the present invention. [Figure 6]1 is a schematic diagram showing registration of prostate peripheral point clouds in ultrasound images and nuclear magnetic images in one embodiment of the present invention; [Figure 7] FIG. 1 is a schematic diagram illustrating fusion-based anesthesia field navigation in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Preferred embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, in which: Although preferred embodiments of the present invention are illustrated in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. EXAMPLES

[0029] The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images includes a coordinate system definition module, an MRI sequence image mark and 3D reconstruction module, an ultrasound data extraction module, a registration module and an anesthesia planning module.

[0030] The coordinate system definition module defines an electromagnetic coordinate system CSe based on the ultrasound imaging plane formed by the electromagnetic field transmitter, defines a coordinate system Csm of the nuclear magnetic device based on the nuclear magnetic device that captures the MRI sequence image, defines a coordinate system CSs based on the electromagnetic sensor bound to the ultrasound probe, and obtains a transformation matrix TMse of the electromagnetic sensor at any position in CSs based on the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe. As shown in Figure 4, MRI is a preoperative image, and it is explained that in order to realize effective surgical navigation, it is necessary to calculate the position of the probe in the US coordinate system through registration.

[0031] Specifically, the transformation matrix TMse is as follows: JPEG2025013277000005.jpg12170Here, R is a 3x3 matrix, T is a 3x1 vector, and R and T denote the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe.

[0032] The MRI sequence image marking and 3D reconstruction module receives a human prostate MRI sequence image, marks an anesthesia area MRI-VMa and an outer contour area MRI-VMp of the prostate, and reconstructs a 3D anesthesia area and a 3D outer contour area of ​​the prostate based on the position information of the MRI sequence image.

[0033] The ultrasound data extraction module receives ultrasound data of the human prostate, extracts a set of images covering the bottom to the tip of the prostate from the ultrasound data, and obtains an outer contour region US-VMp of the prostate in the electromagnetic coordinate system Cse in the ultrasound image based on the spatial position information of each frame of the ultrasound image.

[0034] The registration module uses an ICP point cloud registration algorithm to register the MRI-VMp volume data of the outer contour region of the prostate in the coordinate system Csm of the nuclear magnetic device and the US-VMp volume data of the outer contour region of the prostate in the electromagnetic coordinate system Cse of the ultrasound image, thereby obtaining a spatial transformation matrix TMme between the two.

[0035] The anesthesia planning module receives the current ultrasound image acquired by the ultrasound probe, obtains the anesthesia area EMTS-Vma projected in the electromagnetic coordinate system based on the spatial transformation matrix TMme, and fuses it with the current ultrasound image to obtain the anesthesia area in the ultrasound image.

[0036] In this embodiment, the image and the anesthesia target area in the preoperative nuclear magnetic coordinate system are converted into a real-time electromagnetic coordinate system through the alignment of multimodal medical images, thereby realizing comprehensive fusion of the anesthesia target in the nuclear magnetic image and the ultrasound image, which is used for medical education or surgical guidance and helps to realize accurate positioning of the anesthesia area for transperineal prostate biopsy. EXAMPLES

[0037] In the present invention, the step of marking the MRI sequence image and the 3D reconstruction module is specifically to use the pre-trained prostate segmentation deep learning model to segment the MRI sequence image and mark the segmented anesthesia area VMa and the prostate peri-zone contour area VMp. As shown in FIG. 2, according to the imported MRI image, the image including the prostate peri-zone area and the anesthesia area is marked, and the peri-zone contour area and the anesthesia area are obtained. As shown in FIG. 3, after correctly marking all the images in FIG. 2, the corresponding nuclear magnetic peri-zone contour area and the anesthetic area can be completed as a 3D reconstruction, where the central area is the peri-prostate area, and the two sides are the anesthesia areas around the prostate.

[0038] Here, the establishment of the prostate segmentation deep learning model specifically involves: Extract transverse and sagittal images from prostate MRI sequence images. Marking and segmenting the anesthesia area and the prostate gland contour area on the transverse and sagittal images; The marked and segmented prostate MRI images are then used as samples to train an artificial neural network model, and a prostate segmentation deep learning model is obtained.

[0039] In this embodiment, the prostate is segmented and marked based on a deep learning model, which effectively improves the accuracy of segmentation and recognition. EXAMPLES

[0040] In the present invention, the ultrasound data extraction module acquires the outer contour area US-VMp of the prostate gland as follows: A set of images covering the prostate from the base to the tip is extracted from the ultrasound data and the number of frames is recorded as N. An image including the prostate organ is extracted, and the outer contour region of the prostate is segmented to obtain the outer contour region US-VMp of the prostate in the ultrasound image, and the coordinates PCe of each point of the outer contour in the electromagnetic coordinate system Cse are obtained using the following formula: JPEG2025013277000006.jpg8170

[0041] Here, i denotes the frame number, (u, v) denotes the pixel coordinates of the contour point in the ultrasound 2D image, TMsei denotes the transformation matrix displaying the electromagnetic sensor corresponding to the i-th frame image at any position of CSs, and TMts denotes the transformation matrix of the relative position between the vertex of the ultrasound imaging plane and the electromagnetic sensor.

[0042] In this embodiment, an image including the prostate organ is identified from the collected N frames of images, the outer contour region of the prostate is segmented and contoured, and the outer contour region US-VMp of the prostate in the ultrasound image is obtained, and the coordinates of each point of the outer contour in the electromagnetic coordinate system Cse are obtained by equation transformation, as shown in the first point cloud of Figure 6, which represents the position of each point of the outer contour region of the ultrasound prostate in the Cse coordinate system. EXAMPLES

[0043] JPEG2025013277000007.jpg15170 was used as the 3D point cloud of the first group, and the 3D point cloud corresponding to the outer contour region of the 3D prostate MRI-VMp was used as the 3D point cloud of the second group. The two groups of point clouds are processed separately, which includes the steps of triangulating and smoothing the two groups of point clouds, restoring them to a point cloud, down-sampling to reduce the number of 3D point clouds, and obtaining the two groups of down-sampled 3D point clouds. The ICP point cloud alignment algorithm is used to align the processed 3D point clouds and obtain the spatial transformation matrix TMme from the nuclear magnetic device coordinate system CSm to the electromagnetic coordinate system Cse.

[0044] In this embodiment, by using the ICP algorithm to complete the alignment of the two groups of point clouds, a transformation matrix TMme from the nuclear magnetic coordinate system to the electromagnetic coordinate system can be obtained, and thus the prostate point cloud in the nuclear magnetic field is transformed through TMme to obtain the point cloud shown in the schematic diagram after alignment in Figure 6, which can be seen to be basically consistent with the position and orientation of the point cloud corresponding to the ultrasound. EXAMPLES

[0045] In the present invention, the anesthesia planning module comprises the following steps: Obtaining a projected anesthesia region EMTS-Vma by transforming the anesthesia region MRI-VMa in the coordinate system Csm of the nuclear magnetic device into the electromagnetic coordinate system of the current ultrasound image based on the spatial transformation matrix TMme; obtaining a slice Sa including an anesthesia region contour by obtaining a slice generated by a tangent between the current ultrasound image plane position and the anesthesia region EMTS-Vma volume data; A step of obtaining a fusion image of the current ultrasound image and the anesthetized region by fusing and superimposing the current ultrasound image and the slice Sa including the contour of the anesthetized region, and displaying the anesthetized region on the ultrasound image is performed.

[0046] In this embodiment, as shown in FIG. 7, the anesthesia area is also transformed into the coordinate system where the ultrasound is located based on the transformation matrix, completing the fusion of the anesthesia area and the ultrasound 2D image, and realizing the fusion of the ultrasound image and the nuclear magnetic image by projecting the anesthesia area marked by nuclear magnetism onto the ultrasound image. The fusion result is used for medical education or surgical guidance, and helps to realize accurate positioning of the anesthesia area for transperineal prostate biopsy. EXAMPLES

[0047] The system further includes a puncture planning module, the puncture planning module comprising the following steps: A step of extracting perineum point cloud data in the electromagnetic coordinate system Cse from the ultrasound data, and obtaining a series of coordinate points PPi (x, y, z, 1) on the ultrasound imaging plane, where i represents the coordinate point number, and x, y, and z represent the coordinate values ​​of the ultrasound imaging plane, respectively, and obtaining the true coordinates PPsi of each point in the electromagnetic sensor coordinate system CSs using the following formula: PPsi = TMse × TMts × PPi Here, TMts represents a transformation matrix of the vertices of the ultrasound imaging plane and the relative positions of the electromagnetic sensor; According to the true coordinates PPsi of each point, a plane position PP of the perineum is fitted as a puncture starting plane using a discrete point-based svd method, the normal vector of this plane coincides with the direction of the ultrasound probe entering the human; ax+by+cz+d=0. Calculate the distance from the center point of the projected anesthesia area EMTS-Vma to the puncture start plane PP in the current ultrasound image; d=|ax m +by m +cz m +d| / (a 2 +b 2 +c 2 ) Here, (x m , y m , z m ) denotes the center point of the anesthetized region in the electromagnetic coordinate system CSe of the ultrasound imaging plane, and a, b, c, and d are the plane parameters of PP.

[0048] Depending on the distance from the anesthetized area to the starting point of the puncture, an appropriate length of anesthesia needle is recommended.

[0049] In this embodiment, as shown in Fig. 7, the distance d from the anesthesia area in the ultrasound coordinate system to the adapted puncture starting plane PP is calculated, and different lengths of anesthesia needles are recommended according to different distances. Fig. 7 shows that when the distance reaches a certain value, it is recommended to use a longer anesthesia needle to achieve the intelligent anesthesia solution planning of transperineal prostate biopsy puncture. This helps to make the anesthesia work smoother and more efficient, and effectively popularize the transperineal prostate biopsy procedure.

[0050] The method of applying the system of the present invention is as follows. S1: Define an electromagnetic coordinate system CSe based on an ultrasound imaging plane formed by an electromagnetic field transmitter, define a coordinate system Csm of the nuclear magnetic device based on a nuclear magnetic device that captures MRI sequence images, define a coordinate system CSs based on an electromagnetic sensor bound to an ultrasound probe, and obtain a transformation matrix TMse of the electromagnetic sensor at an arbitrary position within CSs based on the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe.

[0051] S2: Receive a human prostate MRI sequence image, and use the pre-trained prostate segmentation deep learning model to segment the MRI sequence image, and mark the segmented anesthesia area VMa and the prostate periphery contour area VMp. Based on the position information of the MRI sequence image, reconstruct the 3D anesthesia area and the 3D outer contour area of ​​the prostate.

[0052] S3: Receive ultrasound data of a human prostate, extract a set of images from the ultrasound data covering the base to the tip of the prostate, and record the number of frames as N; The image including the prostate organ is extracted, the outer contour region of the prostate is segmented, the outer contour region US-VMp of the prostate in the ultrasound image is obtained, and the coordinates PCe of each point of the outer contour in the electromagnetic coordinate system Cse are recorded.

[0053] The 3D point cloud corresponding to the outer contour region MRI-VMp of JPEG2025013277000008.jpg14170 is used as the 3D point cloud of the second group, The two groups of points are processed separately, The processed 3D point clouds are aligned using the ICP point cloud alignment algorithm to obtain the spatial transformation matrix TMme from the nuclear magnetic device coordinate system CSm to the electromagnetic coordinate system Cse.

[0054] S5: Receive the current ultrasound image acquired by the ultrasound probe, and convert the anesthesia area MRI-VMa in the coordinate system Csm of the nuclear magnetic device into the electromagnetic coordinate system in the current ultrasound image based on the spatial transformation matrix TMme to obtain a projected anesthesia area EMTS-Vma; A slice generated by a tangent between the plane position of the current ultrasound image and the EMTS-Vma volume data of the anesthetized region is obtained, and a slice Sa including the contour of the anesthetized region is obtained. The current ultrasound image and the slice Sa including the contour of the anesthetized region are fused and superimposed to obtain a fused image of the current ultrasound image and the anesthetized region, and the anesthetized region is displayed on the ultrasound image.

[0055] S6: Extract the perineum point cloud data PPi in the electromagnetic coordinate system Cse from the ultrasound data, and obtain the true coordinates PPsi of each point in the electromagnetic sensor coordinate system CSs using the following formula: PPsi = TMse × TMts × PPi Here, TMts represents the transformation matrix of the vertices of the ultrasound imaging plane and the relative positions of the electromagnetic sensors, Based on the true coordinates PPsi of each point, the svd method based on discrete points is used to fit the perineum plane position PP as the puncture starting surface; The distance d from the center point of the anesthesia area EMTS-Vma projected under the current ultrasound image to the puncture starting plane PP is calculated, and an anesthesia needle with an appropriate length is recommended based on the distance from the anesthesia area to the puncture starting plane.

[0056] The application of the system of the present invention realizes the comprehensive fusion of the anesthesia target in the nuclear magnetic image and the ultrasound image, which is used in medical education or surgical guidance and helps to realize the accurate positioning of the anesthesia area for transperineal prostate biopsy. By calculating and analyzing the distance between the target area and the starting position of the needle, it realizes the intelligent recommendation of the anesthesia needle model, which makes the anesthesia work proceed more smoothly and efficiently, and plays an effective role in the popularization of the transperineal prostate biopsy procedure.

[0057] Although various embodiments of the present invention have been described above, the above description is illustrative, not exhaustive, and is not limited to the disclosed embodiments. It will be apparent to those skilled in the art that many modifications and variations can be made without departing from the scope and spirit of the described embodiments.

Claims

1. 1. An intelligent anesthesia system for transperineal prostate needle based on multimodal medical images, comprising: a coordinate system definition module that defines an electromagnetic coordinate system CSe based on an ultrasound imaging surface formed by an electromagnetic field transmitter, defines a coordinate system Csm of the nuclear magnetic device based on a nuclear magnetic device that captures MRI sequence images, defines a coordinate system CSs based on an electromagnetic sensor bound to an ultrasound probe, and obtains a transformation matrix TMse of the electromagnetic sensor at an arbitrary position in CSs based on the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe; an MRI sequence image marking and 3D reconstruction module, which receives a human prostate MRI sequence image, marks an anesthesia area MRI-VMa and an outer contour area MRI-VMp of the prostate, and reconstructs a 3D anesthesia area and a 3D outer contour area of ​​the prostate according to position information of the MRI sequence image; an ultrasound data extraction module for receiving ultrasound data of a human prostate, extracting a set of images covering the prostate from the bottom to the tip from the ultrasound data, and obtaining an outer contour region US-VMp of the prostate in an electromagnetic coordinate system Cse in the ultrasound image based on spatial position information of each frame of the ultrasound image; A registration module for registering the MRI-VMp volume data of the outer contour region of the prostate in the coordinate system Csm of the nuclear magnetic device and the US-VMp volume data of the outer contour region of the prostate in the electromagnetic coordinate system Cse of the ultrasound image using an ICP point cloud registration algorithm to obtain a spatial transformation matrix TMme between the two. an anesthesia planning module, which receives a current ultrasound image acquired by an ultrasound probe, obtains an anesthesia area EMTS-Vma projected in an electromagnetic coordinate system according to a spatial transformation matrix TMme, and fuses the current ultrasound image with the current ultrasound image to obtain an anesthesia area in the ultrasound image; Equipped with An intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images characterized by:

2. The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as described in claim 1, characterized in that the marking specifically involves using a pre-trained prostate segmentation deep learning model to segment the MRI sequence image and marking the segmented anesthesia area VMa and the prostate perimeter zone contour area VMp.

3. Specifically, the establishment of a prostate segmentation deep learning model involves: Extracting transverse and sagittal plane images of the prostate MRI sequence image; Marking and segmenting the anesthesia area and the prostate gland contour area on the transverse and sagittal images; The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as claimed in claim 2, characterized in that the marks and segmented prostate MRI images are input as samples into an artificial neural network model for training, and a prostate segmentation deep learning model is obtained.

4. Specifically, the spatial transformation matrix TMse is The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images according to claim 1, characterized in that R is a 3x3 matrix, T is a 3x1 vector, and R and T represent the position and orientation of the electromagnetic sensor in the electromagnetic coordinate system CSe.

5. Specifically, the outer contour area US-VMp of the prostate is acquired by: Extract a set of images from the ultrasound data covering the base to the tip of the prostate and record the number of frames as N; The image including the prostate organ is extracted, and the outer contour region of the prostate is segmented to obtain the outer contour region US-VMp of the prostate in the ultrasound image. The coordinates PCe of each point of the outer contour in the electromagnetic coordinate system Cse are obtained using the following formula: The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as described in claim 1, characterized in that i denotes a frame number, (u, v) denotes the pixel coordinates of the contour point in the ultrasound 2D image, TMsei denotes a transformation matrix representing the electromagnetic sensor corresponding to the i-th frame image at any position of CSs, and TMts denotes a transformation matrix of the relative position between the vertex of the ultrasound imaging plane and the electromagnetic sensor.

6. Specifically, the alignment is using the 3D point cloud corresponding to the contour region MRI-VMp as the 3D point cloud of the second group; The two groups of points are processed separately, The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as described in claim 1, characterized in that the processed three-dimensional point cloud is aligned using an ICP point cloud alignment algorithm to obtain a spatial transformation matrix TMme from the nuclear magnetic device coordinate system CSm to the electromagnetic coordinate system Cse.

7. The anesthesia planning module consists of the following steps: Obtaining a projected anesthesia region EMTS-Vma by transforming the anesthesia region MRI-Vma in the coordinate system Csm of the nuclear magnetic device into the electromagnetic coordinate system in the current ultrasound image based on a spatial transformation matrix TMme; A step of obtaining a slice Sa including an anesthesia region contour by acquiring a slice generated by a tangent line between the plane position of the current ultrasound image and the anesthesia region EMTS-Vma volume data; The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as described in claim 1, characterized in that the system performs a step of obtaining a fusion image of the current ultrasound image and the anesthesia area by fusing and superimposing the current ultrasound image and a slice Sa including the anesthesia area contour, and displaying the anesthesia area on the ultrasound image.

8. The system further includes a puncture planning module, the puncture planning module comprising the following steps: Extracting perineum point cloud data in the electromagnetic coordinate system Cse of the ultrasound data and fitting the perineum plane position PP as the starting plane position of the puncture needle entering the body, the normal vector of said plane coinciding with the direction of the ultrasound probe entering the human; The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as described in claim 1, characterized in that it performs a step of calculating the distance from the center point of the anesthesia area EMTS-Vma projected under the current ultrasound image to the puncture start plane PP, and recommending an anesthesia needle of an appropriate length based on the distance from the anesthesia area to the puncture start plane.

9. The step of adapting the perineal plane position PP comprises: Extracting the ultrasound data sequence of the perineum and obtaining a series of coordinate points PPi(x, y, z, 1) on the ultrasound imaging plane, where i represents the coordinate point number, and x, y, z represent the coordinate values ​​of the ultrasound imaging plane, respectively, and obtaining the true coordinates PPsi of each point in the electromagnetic sensor coordinate system CSs using the following formula: PPsi=TMse×TMts×PPi Here, the TMts includes steps of displaying a transformation matrix of the vertices of the ultrasound imaging plane and the relative positions of the electromagnetic sensor; Based on the true coordinates of each point PPsi, fitting the perineal plane position PP as the puncture starting surface using a discrete point based svd method. ax+by+cz+d=0 9. An intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images as claimed in claim 8.

10. To calculate the distance from the center point of the projected anesthesia region EMTS-Vma to the puncture start plane PP, the following formula is used: d=|ax ｍ +by ｍ +cz ｍ +d| / (a ２ +b ２ +c ２ ) Here, (x ｍ , y ｍ , z ｍ 10. The intelligent anesthesia system for transperineal prostate puncture based on multimodal medical images according to claim 9, characterized in that a, b, c, and d indicate the center point of the anesthesia area in the electromagnetic coordinate system CSe of the ultrasound imaging plane, and a, b, c, and d are plane parameters of PP.

Images