Method, system and computer readable medium for medical image motion detection and correction
The system uses deep learning to segment and track organ movement within medical images, addressing the issue of blurred images from patient movement, enhancing diagnostic accuracy and efficiency by reconstructing images based on motion curves.
Patent Information
- Application Number
- JP2024533246
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-06
- Filing Date
- 2022-12-05
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-12-05
AI Technical Summary
Current medical imaging technologies, particularly in nuclear medicine, suffer from blurred and inaccurate images due to patient movement during scans, which complicates diagnosis and requires additional monitoring devices that are resource-intensive and prone to errors.
A system and method for motion detection and correction of medical images that utilize deep learning to segment and track organ movement within medical images, reconstructing images based on motion curves without additional monitoring equipment, using techniques like binary segmentation, ellipsoid fitting, and motion compensation.
Enables accurate and efficient correction of medical images by accounting for organ movement, improving diagnostic clarity without the need for separate monitoring devices, thus optimizing image quality and reducing resource consumption.
Smart Images

Figure 2024541684000001_ABST
Abstract
Description
[Technical field]
[0001] FIELD OF THE DISCLOSURE The present disclosure relates to medical imaging techniques, and more particularly to methods, systems and computer readable media for medical image motion detection and correction. [Background technology]
[0002] Nuclear medicine, which has the characteristic of enabling functional evaluation, is indispensable in the field of precision medicine. However, nuclear medicine often requires long examination times and the images are blurred and inaccurate due to the movement of the patient's body or organs during the scan, leading to errors in doctors' judgments.
[0003] For example, myocardial perfusion imaging (MPI) in nuclear medicine is the most commonly used examination item, but is also the most susceptible to the influence of patient movement. As shown in Figure 1, due to the influence of both the contraction and pulsation behavior of the patient's heart itself (Figure 101 is a schematic diagram showing the cross-sectional shape in the dimensions of the short axis, vertical axis, and horizontal axis of the heart in a cardiac cycle, from left to right) and the up and down movement of human breathing (Figure 102 is a schematic diagram showing the cross-sectional shape in the dimensions of the short axis, vertical axis, and horizontal axis of the heart in one respiratory cycle stage, from left to right), many of the obtained MPI images of the heart (Figure 103) are blurred, making diagnosis difficult.
[0004] In the current technology, the solution to the above problem is to install a separate monitoring device to monitor the heart rate and respiratory displacement of the human body, and correct the medical image (e.g., MPI image) based on the signal measured by the monitoring device. However, this correction method not only consumes a large amount of detection resources, but also has the possibility of poor installation of the monitoring device, tracking error, or failure to be integrated with the scanning device (e.g., single photon emission computerized tomography (SPECT) devices on the market usually do not have a respiratory monitoring device as an option). Summary of the Invention [Problem to be solved by the invention]
[0005] Therefore, a technology for detecting and correcting movement in medical images while taking into account the movement of human organs and lesions without installing a separate monitoring device has become an extremely important problem to be solved in this field. [Means for solving the problem]
[0006] In order to solve the above problems, the present disclosure provides a system for detection and correction of movement in medical images, comprising: a management platform for providing a user interface for issuing a command to perform an optimization process on a medical image including list mode data related to a target organ; and an optimization device for performing optimization processing of the medical image based on the command, wherein the optimization processing by the optimization device includes a step of dividing the list mode data corresponding to the medical image into frames having a fixed time dimension and imaging each frame as a frame image, a step of labeling a volume of interest (VOI) including the target organ in each frame image, a step of calculating a motion curve of the target organ based on each volume of interest in each frame image, a step of reconstructing the medical image as an optimized medical image based on the motion curve, and a step of displaying the optimized medical image on the user interface.
[0007] In at least one embodiment of the disclosed system, the optimization device includes a deep learning module, and the step of labeling a region of interest in each frame image by the optimization device includes a step of identifying a binary segmentation region including a target organ in each frame image by the deep learning module, a step of blurring each binary segmentation region to generate a soft mask by the deep learning module, a step of applying each soft mask to each frame image by the deep learning module, a step of fitting an initial ellipsoid sphere to the target organ in each frame image based on each soft mask by the deep learning module, and a step of expanding each initial ellipsoid sphere outward a predetermined distance based on its radius by the deep learning module to generate an ellipsoid sphere representing each region of interest.
[0008] In at least one embodiment of the disclosed system, the step of the optimization device calculating a motion curve of the target organ based on each region of interest of each frame image includes the steps of: dividing each region of interest into a first sub-region of interest and a second sub-region of interest; extracting the three-dimensional coordinates of a first center of mass (COM) of each first sub-region of interest and a second center of mass of each second sub-region of interest as description values of each frame image; reducing the dimensionality of each description value of each frame image by principal component analysis, and using the maximum feature of each description value after dimension reduction as a movement / rotation signal of the target organ; and grouping and filtering each frame image based on each movement / rotation signal to calculate the motion curve.
[0009] In at least one embodiment of the disclosed system, the target organ is a heart, and the division of each region of interest into a first sub-region of interest and a second sub-region of interest is performed in a short axis direction along the long axis of the heart.
[0010] In at least one embodiment of the disclosed system, the motion curve is drawn with respect to any one of the craniocaudal, left-right, and ventral-dorsal axes of the human body, and the optimization device reconstructs the medical image as an optimized medical image based on the motion curve, including a step of selecting a reference object from each frame image, and a step of performing motion compensation for each frame image with reference to the motion curve, and the motion compensation for each frame image includes a step of adjusting all pixels other than the reference object included in each frame image with respect to the reference object in any one of the craniocaudal, left-right, and ventral-dorsal axes based on their three-dimensional coordinates, a step of repeating the adjustment of each frame image until the correlation coefficient between the integration of each frame image after the adjustment and the reference object reaches a maximum value, and a step of integrating each of the motion-compensated frame images to reconstruct the optimized medical image. In at least one specific embodiment of the disclosed system, the adjustment includes, but is not limited to, rotation, displacement, scaling, deformation, or any combination thereof. In some specific embodiments of the disclosed system, the adjustment is translation, rotation, or a combination thereof.
[0011] In at least one embodiment of the disclosed system, after calculating the motion curve of the target organ based on each center of gravity, in a situation where each region of interest of each frame image does not accurately correspond to the motion curve, the optimization process performed by the optimization device further includes a step of gating each frame image, in which frame images having similar time and / or positional relationships defined by the cyclic stages of breathing and / or heartbeat of the human body are integrated into a predetermined number of gated set images.
[0012] In at least one embodiment of the disclosed system, the motion curve is drawn relative to any one of the cranio-caudal, left-right, and ventral-dorsal axes of the human body, and the optimization device reconstructs the medical image as an optimized medical image based on the motion curve, comprising the steps of selecting a reference object from each gating set image and performing motion compensation for each gating set image using the motion curve as a reference, and the motion compensation for each gating set image comprises the steps of adjusting all pixels other than the reference object included in each gating set image relative to the reference object based on their three-dimensional coordinates in any one of the cranio-caudal, left-right, and ventral-dorsal axes, repeatedly adjusting each gating set image until the correlation coefficient between the integration of each gating set image after the adjustment and the reference object reaches a maximum value, and integrating each of the motion-compensated gating set images to reconstruct the optimized medical image.
[0013] In at least one embodiment of the disclosed system, the fixed time dimension is in units of 100 ms to 500 ms.
[0014] In at least one embodiment of the disclosed system, the system further comprises a scanning device for imaging the target organ to obtain medical images, the scanning device being one of a single photon emission computed tomography (STEM), a positron emission tomography (PET) device, a magnetic resonance imaging (MRI) device, and a computer tomography (CT) device, a picture archiving and communication system (PACS) for archiving the medical images and the optimized medical images, and a room report computer for retrieving and displaying the medical images and the optimized medical images.
[0015] The present disclosure further provides a method for motion detection and correction in medical images, comprising the steps of acquiring a medical image including list mode data relating to a target organ, dividing the list mode data corresponding to the medical image into frames having a fixed time dimension and imaging each frame as a frame image, labeling a region of interest including the target organ in each frame image, calculating a motion curve of the target organ based on each region of interest in each frame image, and reconstructing the medical image as an optimized medical image based on the motion curve.
[0016] In at least one embodiment of the disclosed method, the step of labeling the region of interest in each frame image includes the steps of: identifying a binary segmentation region including a target organ in each frame image by a deep learning module; blurring each binary segmentation region to generate a soft mask by the deep learning module; applying each soft mask to each frame image by the deep learning module; fitting an initial ellipsoid sphere to the target organ in each frame image based on each soft mask by the deep learning module; and expanding each initial ellipsoid sphere outward a predetermined distance based on its radius by the deep learning module to generate an ellipsoid sphere representing each region of interest.
[0017] In at least one embodiment of the disclosed method, the step of calculating the motion curve of the target organ based on each region of interest of each frame image includes the steps of: dividing each region of interest into a first sub-region of interest and a second sub-region of interest; extracting the three-dimensional coordinates of the first center of gravity of each first sub-region of interest and the second center of gravity of each second sub-region of interest as description values of each frame image; performing dimensionality reduction of each description value of each frame image by principal component analysis, and taking the maximum feature of each description value after dimensionality reduction as the movement / rotation signal of the target organ; and grouping and filtering each frame image based on each movement / rotation signal to calculate the motion curve.
[0018] In at least one embodiment of the method according to the present disclosure, the target organ is a heart, and the division of each region of interest into a first sub-region of interest and a second sub-region of interest is performed in the direction of the short axis along the long axis of the heart.
[0019] In at least one embodiment of the method according to the present disclosure, the motion curve is drawn relative to any one of the cranio-caudal, left-right, and ventral-dorsal axes of the human body, and the step of reconstructing the medical image as an optimized medical image based on the motion curve includes a step of selecting a reference object from each frame image, and a step of performing motion compensation for each frame image using the motion curve as a reference, and the motion compensation for each frame image includes a step of adjusting all pixels other than the reference object included in each frame image relative to the reference object in any one of the cranio-caudal, left-right, and ventral-dorsal axes based on their three-dimensional coordinates, a step of repeating the adjustment of each frame image until the correlation coefficient between the integration of each frame image after the adjustment and the reference object reaches a maximum value, and a step of integrating each of the motion-compensated frame images to reconstruct the medical image as an optimized medical image.
[0020] In at least one embodiment of the method disclosed herein, after calculating the motion curve of the target organ based on each center of gravity, in a situation where each region of interest of each frame image does not exactly correspond to the motion curve, the method further includes a step of gating each frame image, which includes integrating frame images having similar time and / or positional relationships defined by cycle stages of the human body's breathing and / or heartbeat into a predetermined number of gated set images.
[0021] In at least one embodiment of the disclosed method, the motion curve is drawn with respect to any one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis of the human body, and the step of reconstructing the medical image as an optimized medical image based on the motion curve includes the steps of selecting a reference object from each gating set image, and performing motion compensation for each gating set image with reference to the motion curve, and the motion compensation for each gating set image includes the steps of adjusting all pixels other than the reference object included in each gating set image with respect to the reference object based on their three-dimensional coordinates in any one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis, repeatedly performing the adjustment of each gating set image until the correlation coefficient between the integration of each gating set image after the adjustment and the reference object reaches a maximum value, and integrating each of the motion-compensated gating set images to reconstruct the optimized medical image.
[0022] In at least one embodiment of the disclosed method, the medical image is taken of the target organ with a scanning device, the scanning device being one of a single photon emission computed tomography, a positron emission tomography, a magnetic resonance imaging device, and a computed tomography device.
[0023] In at least one embodiment of the method according to the present disclosure, the fixed time dimension is in units of 100 ms to 500 ms.
[0024] The present disclosure further provides a computer readable storage medium having stored thereon instructions for applying to a computer to perform at least one of the above-described methods for motion detection and correction in medical images.
[0025] As described above, the method, system, and computer-readable medium for medical image movement detection and correction disclosed herein can divide a medical image relating to a target organ into multiple frame images based on list mode data, analyze multiple centroids of a region of interest in the multiple frame images, calculate a motion curve of the target organ during a scanning period, and reconstruct and optimize the medical image based on the motion curve, thereby enabling medical image movement detection and correction to be performed taking into account the movement of human organs and lesions without installing a separate monitoring device. [Brief description of the drawings]
[0026] Specific embodiments of the present disclosure will now be described with reference to the following drawings. [Figure 1] 1 illustrates an embodiment of a myocardial perfusion imaging method performed according to current technology. [Diagram 2] 1 is a schematic diagram of a system configuration for detecting and correcting movement in a medical image according to the present disclosure. [Diagram 3] 1 is a schematic diagram of an embodiment of a medical image motion detection and correction system according to the present disclosure; [Figure 4] 1 is a flow chart of a method for medical image motion detection and correction according to the present disclosure. [Diagram 5] 1 is a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 6A] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 6B] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 6C] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure, where COM represents the center of gravity. [Figure 7] 1 illustrates a partial embodiment of a medical image motion detection and correction method according to the present disclosure, where COM1 and COM2 represent center of gravity 1 and center of gravity 2, respectively. [Figure 8] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 9] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 10] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 11] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 12] 1 is a schematic diagram of an embodiment of a method for motion detection and correction of medical images according to the present disclosure; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0027] Hereinafter, a description will be given based on a specific embodiment so that a person skilled in the art can easily understand other advantages and effects of the present application from the contents described in the present specification. The structure, proportion, size, etc. shown in the drawings attached to this disclosure are to be adapted to the description of the specification so that a person skilled in the art can understand and read them, and do not limit the limited conditions that can be implemented in this disclosure. Therefore, any modification, change or adjustment shall be within the scope of the technical content described in this disclosure unless it affects the effects that can be produced and the purpose that can be achieved in this disclosure.
[0028] FIG. 2 can be viewed as a schematic diagram of a system configuration for performing motion detection and correction of medical images in the present disclosure.
[0029] In at least one embodiment, the management platform 201 of the present disclosure is for integrating processing processes for medical images, including receiving and sending medical images, providing user search for medical images, and performing optimization processing of medical images based on user requests, etc. In some embodiments, the management platform 201 can be displayed via a user interface, such as any suitable web page, application page, human machine interface, etc., without any particular limitation in this specification.
[0030] In at least one embodiment, the optimization device 202 according to the present disclosure performs background services of corresponding medical image optimization processing (including motion detection and correction) based on commands submitted by a user on the management platform 201. In some embodiments, the optimization device 202 according to the present disclosure may be implemented in any suitable physical computer system, cloud system, etc., or may be implemented in a computer system integrated with the management platform 201, without any particular limitation in the present disclosure.
[0031] In at least one embodiment, the scanning device 203 of the present disclosure may be any detection device capable of capturing medical images, including, but not limited to, single photon emission computed tomography, positron emission tomography, magnetic resonance imaging, computed tomography, etc., that captures medical images of the area of interest to the patient (e.g., including, but not limited to, the heart, lungs, coronary arteries, liver, stomach, etc.). In some embodiments, the medical images captured by the scanning device 203 of the present disclosure include corresponding list mode data, making them useful for non-real-time (e.g., post-mortem) retrospective analysis and correction of the captured medical images.
[0032] In at least one embodiment, the medical image storage and transmission system 204 of the present disclosure may be a storage system currently employed in any hospital for storing medical images acquired by the scanning device 203 and / or optimized medical images optimized by the optimization device 202.
[0033] In at least one embodiment, the examination room report computer 205 of the present disclosure may be any terminal device used by a physician in a clinic that provides the physician with retrieval or display of medical images and / or optimized medical images stored in the medical image storage and transmission system 204.
[0034] In at least one embodiment of the present disclosure, the management platform 201, the optimization device 202, the scanning device 203, the medical image storage and transmission system 204, and the clinic report computer 205 shown in FIG. 2 are configured to communicate with each other using standard Digital Imaging and Communications in Medicine (DICOM), thereby providing a high degree of scalability of the present disclosure. It should be noted that the present disclosure is not limited to the above-mentioned components. As an example, any number of the above-mentioned components may be integrated into the same device depending on the work requirements, and a single management platform 201 and / or optimization device 202 may be designed to support optimization of medical images of multiple scanning devices 203, and there is no particular limitation in the present disclosure.
[0035] FIG. 3 is a schematic diagram of a specific embodiment of performing motion detection and correction of medical images in the present disclosure.
[0036] Specifically, in at least one embodiment, the present disclosure may perform motion detection and correction of medical images of the heart taken using a single photon emission computed tomography (SPECT). For example, a scanning device 203 (e.g., a single photon emission computed tomography) according to the present disclosure may include a 19 pinhole collimator and a CdZnTe (CZT) gamma camera equipped with 19 CdZT sensors (e.g., CZT elements including 32×32 pixels) that scans the heart from a right oblique front view angle to a left oblique rear view angle and acquires an associated SPECT image (i.e., medical image). The imaging process of the SPECT image by the scanning device 203 includes the steps of scanning the heart by setting dual energy windows that are set asymmetrically (e.g., -14% to 23%) and symmetrically (e.g., -9% to 9%), respectively, storing list mode and / or frame mode data corresponding to the SPECT image according to the scan result, transmitting the list mode and / or frame mode data to a workstation built into the scanning device 203 in a standard medical digital imaging and communication format, and resampling and displaying the list mode and / or frame mode data along the short axis, vertical long axis, and horizontal long axis of the heart. However, the description of the specifications and devices applicable to the scanning device 203 of the present disclosure and the method of acquiring medical images is merely an example and does not limit the contents of the present disclosure.
[0037] In the schematic flow chart shown in FIG. 3, a medical image obtained after gamma photons 301 emitted from a patient 300 (in myocardial blood flow imaging) during imaging are imaged by the scanning device 203 (the imaging step) is indicated by reference numeral 302. In this case, the medical image 302 can be directly stored in the medical image storage and transmission system 204 so that the examination room report computer 205 can search for it. However, under a situation where the medical image 302 may be blurred due to the movement of the patient 300 during scanning, the management platform 203 can instruct the optimization device 202 to perform movement detection and correction, including processing procedures such as time dimension segmentation 303, motion center of gravity analysis 304, and deformation model correction 305, in real time on the medical image 302 received from the scanning device 203, to obtain an optimized medical image 306. Furthermore, if the obtained medical image 302 contains list mode data, when it is later discovered that the medical image 302 is blurred, the medical image 302 can be accessed by the management platform 201 in the medical image storage and transmission system 204, and retrospective analysis and correction can be performed by the optimization device 202.
[0038] FIG. 4 shows a process flow for performing motion detection and correction of medical images according to the present disclosure (for example, the optimization device 202 performs the processing steps 303 to 305 in FIG. 3 above), and an embodiment of each step can be understood through FIGS. 5 to 12 and the following description.
[0039] In at least one embodiment of the present disclosure, in step S10, a medical image for which movement detection and correction is to be performed (e.g., a set of stress state images and rest state images of a patient's heart obtained in one scan) can be selected through a user interface provided by the management platform 201. In this case, the medical images may be acquired in real time by the management platform 201 when the patient is scanned by the scanning device 203, and the management platform 201 may access the medical image storage and transmission system 204 as necessary.
[0040] In at least one embodiment of the present disclosure, in step S20, taking into account the displacement status (i.e., events) of the target organ (e.g., the heart) captured by the medical image at different respiratory and / or cardiac cycle stages (or the patient's own movement) during the imaging period, the optimization device 202 performs the time dimension segmentation 303 shown in FIG. 3 above, and divides the events included in the list mode data corresponding to the medical image (e.g., the events captured using the above-mentioned dual energy window) into frames having a fixed time dimension (e.g., in units of 500 milliseconds), and back-projects the above frames onto an object plane (e.g., the surface of the 19 CZT sensors of the above-mentioned scanning device 203 that is parallel to the corresponding image plane and intersects at the common focus of the multi-pinhole system) to image them. The results are shown in the imaging results of each frame (hereinafter also referred to as frame images) in FIG. 5. In some embodiments, each frame can be divided in the fixed time dimension in units of any value between 100 milliseconds and 500 milliseconds. However, depending on the computational capabilities or work requirements of the optimizer 202, the frames may be divided in other suitable fixed time dimensions, and is not specifically limited in this disclosure.
[0041] In at least one embodiment of the present disclosure, in step S30, as shown in FIG. 5, the optimization device 202 obtains displacement information such as the position, rotation angle, and / or axis (e.g., long axis and short axis of the heart) of the target organ in each frame image by labeling a region of interest 501 including the target organ (e.g., the heart) in each frame image. In some embodiments, the region of interest 501 including the target organ is spherical. However, the region of interest 501 may be labeled with other suitable shapes according to the shape of the target organ or the operation requirements, and there is no particular limitation in the present disclosure.
[0042] In some embodiments according to the present disclosure, an example of labeling a region of interest for the myocardial region of the heart in a frame image is given. The labeling of the myocardial region may be performed by a doctor through a specific application (e.g., an automated software built in the optimization device 202), and the actual operation includes a step of labeling a 3D elliptical sphere passing through the center of the myocardium on the frame image (e.g., the one acquired in the above step S20), a step of acquiring the numerical values of 12 spherical parameters from the 3D elliptical sphere, such as the three-dimensional coordinate including the center of the sphere, the radius in three axes (i.e., the axial directions of the short axis, vertical major axis, horizontal major axis, etc. of the heart), the rotation angle of the three axes, and the length ratio and angle of the bottom surface of the heart, and a step of fitting the 3D elliptical sphere labeled by the doctor to form smooth inner and outer surfaces toward the inside and outside, respectively, by the specific application based on the spherical parameters. The fitted frame image, which is a completed 3D elliptical sphere, is a frame image including a region of interest in which the myocardial region is defined.
[0043] In some embodiments according to the present disclosure, the labeling of the region of interest 501 in step S30 can also be realized by a deep learning module (e.g., a deep learning module built into the optimization device 202). In this embodiment, the deep learning module includes main structures such as a convolution layer, a deconvolution layer, a leaky regularized linear activation layer, a residual connection, and a beating connection, and more than one thousand sets of frame images in which the myocardial region is labeled by a doctor (e.g., labeling by the specific application) are used as training data for the deep learning module. The deep learning module, which has been trained and can accurately predict the myocardial region of the frame image when the frame image before labeling is input, can be used for labeling the region of interest 501 in step S30.
[0044] 6A to 6C are schematic diagrams of steps when applying a deep learning module to labeling of a region of interest 501. In at least one embodiment according to the present disclosure, the trained deep learning module can first receive the frame image (a split screen of the heart in dimensions such as its short axis, vertical long axis, and horizontal long axis shown in FIG. 6A) generated in step S20. In some embodiments according to the present disclosure, the deep learning module identifies a region 601 in which the heart is present in the frame image (a regular ellipse (with a rotation angle of zero) of at least 5 cm from the heart division region selected by the solid line region 601 as in FIG. 6A) and identifies a binary segmentation region 603 (a region indicated by 603 in FIG. 6B) including a myocardial region. Next, the deep learning module blurs the binary segmentation region 603 to generate a soft mask, and further applies the soft mask to the initial frame image to exclude image activity in regions other than the central muscle region of the frame image. Finally, the deep learning module fits an ellipsoidal sphere to the myocardial region 602 on the frame image based on the soft mask (the ellipsoidal sphere transverse line corresponding to the solid line indicated by 602 in FIG. 6B), and expands the ellipsoidal sphere radius outward by 2 cm to generate a new ellipsoidal sphere representing the region of interest 604 (the region corresponding to the solid line indicated by 604 in FIG. 6C).
[0045] In at least one embodiment according to the present disclosure, in step S40, the optimization device 202 performs the motion center of gravity analysis 304 shown in FIG. 3 above on the target organ in the region of interest labeled in step S30. In some embodiments according to the present disclosure, first, the state of displacement and rotation of the target organ (e.g., the heart) in each frame image is accurately observed, and the region of interest in each frame image is divided into two. The method of dividing the region of interest can be observed by referring to the difference between FIG. 6C and FIG. 7. For example, the region of interest 604 of each frame image labeled in FIG. 6C is divided into two along the long axis of the heart in this step S40, and sub-regions of interest 701 and 702 related to the heart shown in FIG. 7 are formed. Since the left side of FIG. 7 is a cross-sectional view in the short axis dimension of the heart, the observed sub-regions of interest 701 and 702 are expressed overlapping each other. Next, the respective centers of gravity 703, 704 are calculated based on the two sub-regions of interest 701, 702 of the target organ obtained, which assists in tracking the movement of the target organ in each frame image, and allows the motion curve of the target organ during imaging to be obtained.
[0046] The advantage of splitting the region of interest 604 shown in Fig. 7C into two sub-regions of interest 701, 702 resulting in two centroids 703, 704 for motion centroid analysis, compared to a region of interest 604 with only one centroid 605 shown in Fig. 6C, is shown in Fig. 8. 801-803 show how the target organ (e.g. the heart) is translated (801), translated and rotated (802), and rotated (803) (i.e., changing from a solid pattern to a dashed pattern), respectively, while 804, 805 show the effect of 801-803 observed with a single centroid 605 and with two centroids 703, 704, respectively. When a translation 801 occurs in the target organ, it can be clearly tracked using either the single center of gravity 605 or the two centers of gravity 703, 704 (for example, a change in movement from center of gravity 605 to 605', or a change in movement from centers of gravity 703, 704 to 703', 704'), but when a translation rotation 802 or rotation 803 occurs in the target organ, the tracking effect observed using only the single center of gravity 605 is clearly inferior to the tracking effect observed using the two centers of gravity 703, 704. In particular, when only rotation 803 occurs in the target organ, there is no significant change in center of gravity 605 before and after the rotation (coinciding with 605'), and the rotation of the target organ cannot be tracked.
[0047] However, the method of analyzing the center of gravity of motion according to the present disclosure is not limited to the above description. For example, those skilled in the art may divide the region of interest 604 into sub-regions of interest 701, 702 along the short axis of the heart (long axis direction), and may divide the region of interest 604 into more than two regions of interest as necessary (e.g., according to the shape characteristics of the target organ) to observe the displacement and rotation of the target organ at more than two centers of gravity.
[0048] Immediately after step S40, after the region of interest 604 is divided into sub-regions of interest 701, 702, three-dimensional coordinates (e.g., "(x1, y1, z1), (x2, y2, z2)") corresponding to the centroids 703, 704 in each frame image are calculated as description values of each frame image. In some embodiments according to the present disclosure, the description values of each frame image can be reduced in dimension using a principle component analysis (PCA) method, and the maximum feature can be obtained from the description values after dimension reduction as the movement / rotation signal of the target organ in each frame image. In some embodiments according to the present disclosure, by filtering each frame image based on the movement / rotation signal of the target organ, excessive high-frequency noise signals can be filtered, so that the motion curve of the target organ during shooting is calculated.
[0049] In at least one embodiment according to the present disclosure, the motion curve of the target organ during imaging is shown in Fig. 9. Fig. 9 shows detection of the motion curve of the target organ (heart) with respect to the left and right (X axis, the schematic diagram in the upper layer of Fig. 9), the ventral and dorsal (Y axis, the schematic diagram in the middle layer of Fig. 9), and the head and tail (Z axis, the schematic diagram in the lower layer of Fig. 9) of the human body during imaging, where 901 is a curve (shown by a dotted line) showing the relative position change of the center of gravity 703 or 704 in each frame image on the X axis, Y axis, and Z axis, 902 is a curve (shown by a short dashed line) showing the rotation signal of the center of gravity 703 or 704 on the X axis, Y axis, and Z axis calculated based on the curve 901, and 903 is a curve (shown by a solid line) showing a more accurate position change of the center of gravity 703 or 704 obtained by grouping and accumulating each frame image.
[0050] In at least one embodiment of the present disclosure, in step S50, in order to ensure the effectiveness of the correction of the medical image in the subsequent steps, the optimization device 202 may first determine whether the movement amount of the target organ is excessive according to the motion curve of the target organ, and if it is excessive (for example, if the amplitude of the motion curve of the target organ exceeds 50 mm), the radiologist may request the patient to perform a rescan and repeat steps S10 to S40 to obtain a new motion curve of the target organ. If rescan is not required, the next step can be performed to correct the deformation model of the medical image.
[0051] The above steps S10 to S50 complete the procedure of motion detection for the medical image. Depending on the result of the motion detection, the optimization device 202 can continue to perform the deformation model correction 305 shown in FIG. 3 on the medical image, as described in the following steps S60 to S80.
[0052] Since the motion curve is the average data of the positions of the regions of interest (based on the center of gravity) in all frame images, the optimization device 202 first checks in step S60 whether the positions of the regions of interest in each frame image accurately correspond to the motion curve, and if the regions of interest in each frame image do not accurately correspond to the motion curve, gating in step S70 can be performed. If they correspond accurately, optimized reconstruction of medical images in step S80 can be directly performed for each frame image. In some embodiments according to the present disclosure, regardless of whether the regions of interest in each frame image accurately correspond to the motion curve, gating in step S70 can be directly performed for each frame image, which further contributes to understanding the movement of the target organ and shortens the time required for optimizing reconstruction of medical images in step S80.
[0053] In at least one embodiment of the present disclosure, the gating described in step S70 forms a gating set image by integrating frame images with similar time and / or positional relationships based on the motion curve of the target organ (e.g., the heart) calculated in step S40.
[0054] FIG. 10 shows an embodiment in which each frame image of a medical image related to the heart in the present disclosure is organized into eight gating sets. The eight gating sets in the upper row are gating set images observed on the vertical long axis plane of the heart, and the eight gating sets in the lower row are gating set images observed on the short axis plane of the heart. Furthermore, considering that the motion curve may correspond to the regular motion of the breathing and / or heartbeat of the human body, and in the absence of vigorous motion of the patient during the scan, these eight gating set images can also correspond to each cycle phase of the breathing and / or heartbeat, such as the breathing cycle phase of inspiration / expiration from left to right shown in FIG. 10. Since each gating set image has been observed to have a repetitive displacement relationship with respect to the gating set image at the end of inspiration phase (the gating set image at the leftmost end) on any of the craniocaudal axis, left-right axis, and / or ventral-dorsal axis of the human body, this step S70 is also called respiratory gating or cardiac gating.
[0055] In some embodiments of the present disclosure, Fig. 10 shows an embodiment in which all frame images of a medical image related to the heart are combined into eight gating sets according to the stages of the human body's respiratory cycle (i.e., each gating set includes 12.5% of the total number of frame images). However, according to the needs of the correction and optimization effect of the medical image and the operational needs, the number of gating sets can be increased or decreased, one gated set image can be additionally formed by further considering the cardiac cycle of the human body, or one gating set image can be formed by comprehensively considering the stages of the respiratory cycle and cardiac cycle, and there is no particular restriction in this specification.
[0056] In at least one embodiment according to the present disclosure, in step S80, the reconstruction optimization of the medical image is performed based on each accurately positioned frame image (i.e., each frame image of the medical image not processed in step S70) or the gating set image gated in step S70. The execution of the reconstruction optimization of the medical image is mainly based on the reference object in each frame image or gating set image (for example, one frame image or gating set image that is determined to be in the end stage of inspiration, in the isovolumic contraction phase of the cardiac cycle, or to satisfy the above two), and performs motion compensation of the remaining frame images or gating set images. The motion curves of the human body with respect to the craniocaudal axis, the left-right axis, and / or the ventral-dorsal axis are used as references for adjusting the frame images or gating set images during the motion compensation (rotation, displacement, scaling, deformation, and other adjustment work). In addition, the correlation coefficient between the reconstructed and optimized medical image and the reference object is used to observe the completeness of the reconstruction optimization of this medical image.
[0057] For example, when the optimization device 202 performs reconstruction and optimization of a medical image related to the heart in consideration of the respiratory cycle stage of the human body, it selects one frame image or gating set image in the end-inhalation stage from the frame images or gating set images corresponding to the medical image as a reference object, and adjusts all pixels included in the remaining frame images or gating set images relative to the reference object based on their three-dimensional coordinates in the craniocaudal axis, left-right axis, and / or ventral-dorsal axis of the human body with reference to the motion curve (i.e., adjustment work such as the above-mentioned rotation, displacement, and scaling), integrates each adjusted frame image or gating set image to reconstruct it as an optimized medical image, and repeats the reconstruction optimization of the medical image until the optimized medical image reaches a state that is most similar to the reference object (for example, the correlation coefficient reaches a maximum value or the root mean square error reaches a minimum value). When the motion compensation of the medical image is completed, the motion compensation effect is obtained from the left side view (without reconstruction optimization) to the right side view (completed reconstruction optimization) shown in FIG. 11.
[0058] In some embodiments of the present disclosure, the motion compensation performed in step S80 is performed by incorporating a motion compensation program into a maximum a posteriori expectation maximization (MAPEM) algorithm, which can be expressed in the form of the following mathematical formula:
number
number
number
number
number
number
number
number
number
number
[0059] 12 is a schematic diagram showing the effect after the optimization device 202 performs the above steps S10-S80 on the medical image. For example, in at least one embodiment of the present disclosure, the optimization device 202 analyzes the medical image related to the target organ (e.g., the heart) into the form of frame images (e.g., the frame image corresponding to a certain respiratory cycle stage shown in FIG. 1201 and / or the frame image corresponding to a certain cardiac cycle stage shown in FIG. 1202), and then adjusts each frame image (or the gated set image) with respect to the craniocaudal axis, the left-right axis, and / or the ventral-dorsal axis of the human body in consideration of the motion curve of the target organ (e.g., the frame image adjusted to correspond to a certain respiratory cycle stage shown in FIG. 1203 and / or the frame image adjusted to correspond to a certain cardiac cycle stage shown in FIG. 1204) to obtain a reconstructed optimized medical image 1205. Here, the optimization device 202 can directly display the reconstructed optimized medical image 1205 on the user interface of the management platform 201, and can also store the reconstructed optimized medical image 1205 in the medical image storage and transmission system 204 by the management platform 201, so that the doctor can search for it via the examination room report computer 205.
[0060] The present disclosure further provides a computer-readable medium applicable to a computer or computing device having a processor and / or memory having stored thereon commands that cause the computer or computing device to execute the medical image movement detection and correction method in accordance with the commands via the processor (e.g., CPU, GPU, etc.) and / or memory.
[0061] As described above, the method, system, and computer-readable medium for medical image movement detection and correction disclosed herein can divide a medical image relating to a target organ into multiple frame images based on list mode data, analyze multiple centroids of an area of interest in the multiple frame images, calculate a motion curve of the target organ during scanning, and then perform reconstruction and optimization of the medical image based on the motion curve, thereby enabling medical image movement detection and correction to be performed taking into account the movement of human organs or lesions without the need to additionally install a separate monitoring device.
[0062] The above-mentioned embodiments are merely illustrative of the effects of the present disclosure and are not intended to limit the scope of the present disclosure, and those skilled in the art may make various modifications and changes to these embodiments without departing from the spirit of the present invention. Therefore, the scope of protection of the rights of the present disclosure is as set forth in the claims below. [Explanation of symbols]
[0063] 101~103 Example 201 Management Platform 202 Optimization Device 203 Scanning Device 204 Medical Image Storage and Transmission System 205 Examination room report computer 300 patients 301 gamma photon 302 Medical Imaging 303 Temporal Dimension Segmentation 304 Motion center of gravity analysis 305 Deformed Model Correction 306 Optimized Medical Imaging 501 Areas of Interest 601 area 602 Myocardial Region 603 Binary Segmentation Regions 604 Area of Interest 605, 605' center of gravity 701, 702 Sub-area of interest 703, 703' center of gravity 704, 704' center of gravity 801, 802, 803 Status 804, 805 Effects 901, 902, 903 curves 1201, 1202, 1203, 1204 Illustrations 1205 Optimized Medical Imaging S10~S80 Step
Claims
1. a management platform providing a user interface for issuing commands to perform an optimization process on medical images corresponding to the list-mode data of a target organ; an optimization device that performs the optimization process on the medical image based on the command, The optimization process includes: segmenting list-mode data corresponding to the medical image into a plurality of frames having a fixed time dimension and imaging each frame as a plurality of frame images; labeling a region of interest in each of the frame images to include the target organ; calculating a motion curve of the target organ based on the region of interest of each of the frame images; reconstructing the medical image based on the motion curve; displaying the reconstructed medical image on the user interface; A system for medical image motion detection and correction, comprising:
2. The optimization device includes a deep learning module, In the step of labeling the region of interest in each frame image, identifying a binary segmentation region in each of the frame images that includes the target organ by the deep learning module; blurring the binary segmentation region to generate a soft mask using the deep learning module; applying each of the soft masks to each of the frame images by the deep learning module; Fitting an initial ellipsoid to the target organ in each of the frame images based on each of the soft masks by the deep learning module; Expanding each of the initial ellipsoids outward by a preset distance according to its radius using the deep learning module to generate an ellipsoid representing the region of interest; The system of claim 1 , comprising:
3. In the step of calculating a motion curve of the target organ based on the region of interest of each of the frame images, Dividing the region of interest into a first sub-region of interest and a second sub-region of interest; extracting three-dimensional coordinates of a first centroid of the first sub-region of interest and a second centroid of the second sub-region of interest as descriptive values of the frame image, respectively; A step of performing dimensionality reduction of the description values of each of the frame images by principal component analysis, and determining the maximum feature of each of the description values after dimensionality reduction as a signal of movement and / or rotation of the target organ; grouping and filtering the frame images based on the translation and / or rotation signals to calculate the motion curve; The system of claim 1 , comprising:
4. 4. The system of claim 3, wherein the target organ is a heart, and the division of each of the regions of interest into the first and second sub-regions of interest is performed in a short axis direction along a long axis of the heart.
5. The movement curve is plotted relative to one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis of the human body; In the step of reconstructing the medical image based on the motion curve, selecting a reference object from each of the frame images; and performing motion compensation for each of the frame images by referring to the motion curve; The motion compensation includes: adjusting all pixels other than the reference object included in each of the frame images in any one of the head-tail axis, the left-right axis, and the ventral-dorsal axis based on the three-dimensional coordinates of the pixels; Repeating the adjustment process for each of the frame images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; a step of integrating the motion-compensated frame images to reconstruct the medical image; The system of claim 1 , comprising:
6. The system described in claim 3, further characterized in that after calculating the motion curve of the target organ based on each of the centers of gravity, if each of the regions of interest in each of the frame images does not accurately correspond to the motion curve, the optimization device executes the optimization process and also performs gating of each of the frame images by integrating frame images among the frame images that have similar time and / or positional relationships defined by cycle stages of breathing and / or heartbeat of the human body into a predetermined number of gating set images.
7. The motion curve is drawn with respect to any one of a cranial-caudal axis, a left-right axis, and a ventral-dorsal axis of the human body, and the medical image is reconstructed based on the motion curve, selecting a reference object from each of the gating set images; performing motion compensation on each of the gating set images using the motion curve as a reference; Including, For each of the gating set images, motion compensation may include: adjusting all pixels other than the reference object included in each of the gating set images in any one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis based on the three-dimensional coordinates of the pixels relative to the reference object; Repeating the adjustment process for each of the gating set images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; integrating the motion-compensated gating set images to reconstruct the medical image; The system of claim 6, comprising:
8. 2. The system of claim 1, wherein the fixed time dimension is in units of 100 milliseconds to 500 milliseconds.
9. a scanning device for imaging the target organ and acquiring the medical image, the scanning device being selected from any one or combination of the group consisting of single photon emission computed tomography, positron emission tomography, magnetic resonance imaging, and computed tomography; a medical image storage and transmission system for storing the medical image and the reconstructed medical image; a clinical report computer for retrieving and displaying the medical images and reconstructed medical images; The system of claim 1 further comprising:
10. acquiring a medical image corresponding to list-mode data of the target organ; segmenting list-mode data corresponding to the medical image into a plurality of frames having a fixed time dimension and imaging each of the frames as a plurality of frame images; labeling a region of interest in each of the frame images to include the target organ; calculating a motion curve of the target organ based on the region of interest of each of the frame images; reconstructing the medical image based on the motion curve; 1. A method for medical image motion detection and correction, comprising:
11. The step of labeling a region of interest in each of the frame images to include the target organ comprises: identifying a binary segmentation region including the target organ in each of the frame images using a deep learning module; blurring each of the binary segmentation regions to generate a soft mask using the deep learning module; applying each of the soft masks to each of the frame images by the deep learning module; Fitting an initial ellipsoid to the target organ in each of the frame images based on each of the soft masks by the deep learning module; Expanding each of the initial ellipsoids outward a predetermined distance based on its radius using the deep learning module to generate an ellipsoid representing the region of interest; The method of claim 10 further comprising:
12. In the step of calculating a motion curve of the target organ based on the region of interest of each of the frame images, Dividing the region of interest into a first sub-region of interest and a second sub-region of interest; extracting three-dimensional coordinates of a first centroid of the first sub-region of interest and a second centroid of the second sub-region of interest as descriptive values of a frame image, respectively; performing dimension reduction of the description values of each of the frame images by principal component analysis, and determining the maximum feature of each of the dimension-reduced description values as a signal of movement and / or rotation of the target organ; grouping and filtering the frame images based on the translation and / or rotation signals to calculate a motion curve; The method of claim 10, comprising:
13. 13. The method of claim 12, wherein the target organ is a heart, and the division of each of the regions of interest into the first and second sub-regions of interest is performed in a short axis direction along a long axis of the heart.
14. The motion curve is drawn relative to any one of a cranial-caudal axis, a left-right axis, and a ventral-dorsal axis of a human body, and the medical image is reconstructed based on the motion curve, selecting a reference object from each of the frame images; and performing motion compensation for each of the frame images by referring to the motion curve; The motion compensation includes: adjusting all pixels other than the reference object included in each of the frame images in any one of the head-tail axis, the left-right axis, and the ventral-dorsal axis based on the three-dimensional coordinates of the pixels; Repeating the adjustment process for each of the frame images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; a step of integrating the motion-compensated frame images to reconstruct the medical image; The method of claim 10, comprising:
15. After calculating the motion curve of the target organ based on the respective centers of gravity, if the respective regions of interest of the respective frame images do not accurately correspond to the motion curve, The method according to claim 12, further comprising a step of gating each of the frame images by integrating frame images having similar time and / or positional relationships defined by respiratory and / or cardiac cycle phases of the human body into a predetermined number of gating set images.
16. The motion curve is drawn with respect to any one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis of the human body, and the medical image is reconstructed based on the motion curve, selecting a reference object from each of the gating set images; and performing motion compensation for each of the gating set images using the motion curve as a reference; For each of the gating set images, motion compensation may include: adjusting all pixels other than the reference object included in each of the gating set images in one of the cranial-caudal axis, the left-right axis, and the ventral-dorsal axis based on the three-dimensional coordinates of the pixels relative to the reference object; performing an adjustment operation on each of the gating set images repeatedly until a correlation coefficient between the integration of each of the gating set images after the adjustment operation and the reference object reaches a maximum value; integrating the motion-compensated gating set images to reconstruct the medical image; 16. The method of claim 15, comprising:
17. 11. The method of claim 10, wherein the medical image is obtained by imaging the target organ with a scanning device, the scanning device being selected from any one or combination of the group consisting of single photon emission computed tomography, positron emission tomography, magnetic resonance imaging, and computed tomography.
18. 11. The method of claim 10, wherein the fixed time dimension is in units of 100 ms to 500 ms.
19. A computer-readable storage medium having stored thereon instructions for executing the method for detecting and correcting motion in medical images according to any one of claims 10 to 18 when applied to a computer.
Citation Information
Patent Citations
Radiological imaging incorporating local motion monitoring, correction, and assessment
CN102067176A
Local motion compensation based on list mode data
JP2009528139A
Method, device and program for processing nuclear medicine image
JP2011022119A
Data-driven respiratory body movement estimation method
JP2020511665A
Radiological imaging incorporating local motion monitoring, correction, and assessment
US20110081068A1
