Method, system and computer-readable medium for medical image motion detection and correction

Abstract

A method, system, and computer readable medium for medical image motion detection and correction are provided. According to the present invention, a medical image of a target organ is divided into a plurality of frame images based on list mode data, a plurality of centroids of a region of interest in the plurality of frame images are analyzed to calculate a motion curve of the target organ during a scan, and the medical image is reconstructed and optimized based on the motion curve, thereby enabling motion detection and correction of the medical image in consideration of the motion of the organ or lesion in the human body without additionally installing a separate monitoring device.

Description

[Technical Field]

[0001] The present disclosure relates to medical imaging technology, and more particularly to a method, system, and computer-readable medium for motion detection and correction in medical images. [Background technology]

[0002] Nuclear medicine, which allows for functional evaluation, is essential in the field of precision medicine. However, nuclear medicine often requires long examination times and the patient's body or organs move during the scan, resulting in blurred and inaccurate images and leading to incorrect diagnosis by the physician.

[0003] For example, myocardial perfusion imaging (MPI) in nuclear medicine is the most commonly used examination, but it is also the most susceptible to the influence of patient motion. As shown in Figure 1, due to the influence of both the contraction and pulsation behavior of the patient's heart itself (Figure 101 is a schematic diagram showing the cross-sectional shape of the heart in dimensions such as the short axis, vertical axis, and horizontal axis during a cardiac cycle, from left to right) and the up and down movement of a human's breathing (Figure 102 is a schematic diagram showing the cross-sectional shape of the heart in dimensions such as the short axis, vertical axis, and horizontal axis during one respiratory cycle, from left to right), many of the obtained MPI images of the heart (Figure 103) are blurred, making diagnosis difficult.

[0004] Current solutions to the above problems involve installing a separate monitoring device to monitor the heart rate and respiratory rate of the human body and correcting medical images (e.g., MPI images) based on the signals measured by the monitoring device. However, this correction method not only consumes a large amount of detection resources, but also faces the risk of poor installation of the monitoring device, tracking errors, or the inability to integrate with the scanning device (e.g., single photon emission computerized tomography (SPECT) devices currently on the market do not usually have optional respiratory monitoring devices). Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, a technology for detecting and correcting movement in medical images, taking into account the movement of human organs and lesions, without installing a separate monitoring device, has become an extremely important issue to be solved in this field. [Means for solving the problem]

[0006] In order to solve the above problems, the present disclosure provides a system for detecting and correcting movement in medical images, comprising: a management platform for providing a user interface for issuing a command to perform an optimization process on a medical image including list-mode data related to a target organ; and an optimization device for performing optimization processing on the medical image based on the command. The optimization processing by the optimization device includes the steps of: dividing the list-mode data corresponding to the medical image into frames having a fixed time dimension and imaging each frame as a frame image; labeling a volume of interest (VOI) including the target organ in each frame image; calculating a motion curve of the target organ based on each VOI in each frame image; reconstructing the medical image as an optimized medical image based on the motion curve; and displaying the optimized medical image on the user interface.

[0007] In at least one embodiment of the disclosed system, the optimization device includes a deep learning module, and the step of labeling regions of interest in each frame image by the optimization device includes the steps of: identifying a binary segmentation region including a target organ in each frame image by the deep learning module; blurring each binary segmentation region to generate a soft mask by the deep learning module; applying each soft mask to each frame image by the deep learning module; fitting an initial ellipsoidal sphere to the target organ in each frame image based on each soft mask by the deep learning module; and expanding each initial ellipsoidal sphere outward by a predetermined distance based on its radius by the deep learning module to generate an ellipsoidal sphere representing each region of interest.

[0008] In at least one embodiment of the disclosed system, the step of the optimization device calculating a motion curve of the target organ based on each region of interest of each frame image includes the steps of: dividing each region of interest into a first sub-region of interest and a second sub-region of interest; extracting the three-dimensional coordinates of the first center of mass (COM) of each first sub-region of interest and the second center of mass of each second sub-region of interest as descriptive values ​​of each frame image; reducing the dimensionality of each descriptive value of each frame image by principal component analysis, and using the maximum feature of each descriptive value after dimension reduction as the movement / rotation signal of the target organ; and grouping and filtering each frame image based on each movement / rotation signal to calculate the motion curve.

[0009] In at least one embodiment of the disclosed system, the target organ is a heart, and the division of each region of interest into a first sub-region of interest and a second sub-region of interest is performed in the short axis direction along the long axis of the heart.

[0010] In at least one embodiment of the system disclosed herein, the motion curve is plotted relative to one of the craniocaudal, left-right, and ventrodorsal axes of the human body. The optimization device reconstructs the medical image as an optimized medical image based on the motion curve, including selecting a reference object from each frame image and performing motion compensation for each frame image using the motion curve as a reference. The motion compensation for each frame image includes adjusting all pixels other than the reference object in each frame image relative to the reference object along one of the craniocaudal, left-right, and ventrodorsal axes based on their three-dimensional coordinates. Repeating this adjustment for each frame image until a correlation coefficient between the integration of each frame image after the adjustment and the reference object reaches a maximum value, and reconstructing the motion-compensated frame images by integrating them as an optimized medical image. In at least one specific embodiment of the present disclosure, the adjustment includes, but is not limited to, rotation, displacement, scaling, deformation, or any combination thereof. In some specific embodiments of the present disclosure, the adjustment is translation, rotation, or a combination thereof.

[0011] In at least one embodiment of the disclosed system, after calculating the motion curve of the target organ based on each center of gravity, in a situation where each region of interest in each frame image does not accurately correspond to the motion curve, the optimization process performed by the optimization device further includes a step of gating each frame image, in which frame images that have similar time and / or positional relationships defined by the cycle stages of human breathing and / or heartbeat are integrated into a predetermined number of gated set images.

[0012] In at least one embodiment of the disclosed system, the motion curve is drawn relative to any one of the craniocaudal, left-right, and ventrodorsal axes of the human body, and the step of the optimization device reconstructing the medical image as an optimized medical image based on the motion curve includes the steps of selecting a reference object from each gating set image and performing motion compensation on each gating set image using the motion curve as a reference, and the motion compensation on each gating set image includes the steps of adjusting all pixels other than the reference object included in each gating set image relative to the reference object in any one of the craniocaudal, left-right, and ventrodorsal axes based on their three-dimensional coordinates, repeatedly adjusting each gating set image until the correlation coefficient between the integration of each gating set image after the adjustment and the reference object reaches a maximum value, and integrating each motion-compensated gating set image to reconstruct as an optimized medical image.

[0013] In at least one embodiment of the disclosed system, the fixed time dimension is in the range of 100 ms to 500 ms.

[0014] In at least one embodiment of the disclosed system, the system further comprises a scanning device for imaging a target organ to obtain medical images, the scanning device being one of a single photon emission computed tomography (SPC), a positron emission tomography (PET) device, a magnetic resonance imaging (MRI) device, and a computer tomography (CT) device; a picture archiving and communication system (PACS) for archiving the medical images and the optimized medical images; and a clinic report computer for retrieving and displaying the medical images and the optimized medical images.

[0015] The present disclosure further provides a method for motion detection and correction of medical images, including the steps of acquiring a medical image including list mode data relating to a target organ, dividing the list mode data corresponding to the medical image into frames having a fixed time dimension and imaging each frame as a frame image, labeling a region of interest including the target organ in each frame image, calculating a motion curve of the target organ based on each region of interest in each frame image, and reconstructing the medical image as an optimized medical image based on the motion curve.

[0016] In at least one embodiment of the disclosed method, the step of labeling the region of interest in each frame image includes the steps of: identifying a binary segmentation region including a target organ in each frame image using a deep learning module; blurring each binary segmentation region to generate a soft mask using the deep learning module; applying each soft mask to each frame image using the deep learning module; fitting an initial ellipsoid sphere to the target organ in each frame image based on each soft mask using the deep learning module; and expanding each initial ellipsoid sphere outward a predetermined distance based on its radius using the deep learning module to generate an ellipsoid sphere representing each region of interest.

[0017] In at least one embodiment of the disclosed method, the step of calculating the motion curve of the target organ based on each region of interest of each frame image includes the steps of: dividing each region of interest into a first sub-region of interest and a second sub-region of interest; extracting the three-dimensional coordinates of the first centroid of each first sub-region of interest and the second centroid of each second sub-region of interest as descriptive values ​​of each frame image; reducing the dimension of each descriptive value of each frame image by principal component analysis, and using the maximum feature of each descriptive value after dimension reduction as the movement / rotation signal of the target organ; and grouping and filtering each frame image based on each movement / rotation signal to calculate the motion curve.

[0018] In at least one embodiment of the method according to the present disclosure, the target organ is a heart, and the division of each region of interest into a first sub-region of interest and a second sub-region of interest is performed in the direction of the short axis along the long axis of the heart.

[0019] In at least one embodiment of the method according to the present disclosure, the motion curve is drawn relative to any one of the craniocaudal, left-right, and ventrodorsal axes of the human body, and the step of reconstructing the medical image as an optimized medical image based on the motion curve includes the steps of selecting a reference object from each frame image and performing motion compensation for each frame image using the motion curve as a reference, and the motion compensation for each frame image includes the steps of adjusting all pixels other than the reference object included in each frame image relative to the reference object in any one of the craniocaudal, left-right, and ventrodorsal axes based on their three-dimensional coordinates, repeating the adjustment of each frame image until the correlation coefficient between the integration of each frame image after the adjustment and the reference object reaches a maximum value, and integrating each motion-compensated frame image to reconstruct the optimized medical image.

[0020] In at least one embodiment of the method disclosed herein, after calculating the motion curve of the target organ based on each center of gravity, in a situation where each region of interest in each frame image does not exactly correspond to the motion curve, the method further includes a step of gating each frame image, in which frame images having similar time and / or positional relationships defined by the cyclical stages of the human body's breathing and / or heartbeat are integrated into a predetermined number of gated set images.

[0021] In at least one embodiment of the disclosed method, the motion curve is drawn relative to any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis of the human body, and the step of reconstructing the medical image as an optimized medical image based on the motion curve includes the steps of selecting a reference object from each gating set image and performing motion compensation on each gating set image using the motion curve as a reference, and the motion compensation on each gating set image includes the steps of adjusting all pixels other than the reference object included in each gating set image relative to the reference object based on their three-dimensional coordinates in any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis, repeatedly adjusting each gating set image until the correlation coefficient between the integration of each gating set image after the adjustment and the reference object reaches a maximum value, and integrating each motion-compensated gating set image to reconstruct as an optimized medical image.

[0022] In at least one embodiment of the disclosed method, the medical image is of a target organ captured by a scanning device, the scanning device being one of a single photon emission computed tomography, a positron emission tomography, a magnetic resonance imaging device, and a computed tomography device.

[0023] In at least one embodiment of the method according to the present disclosure, the fixed time dimension is in the range of 100 ms to 500 ms.

[0024] The present disclosure further provides a computer-readable storage medium having stored thereon instructions for applying to a computer to perform at least one of the above-described methods for motion detection and correction in medical images.

[0025] As described above, the method, system, and computer-readable medium for medical image motion detection and correction disclosed herein can divide a medical image relating to a target organ into multiple frame images based on list mode data, analyze multiple centroids of a region of interest within the multiple frame images, calculate a motion curve of the target organ during a scanning period, and then reconstruct and optimize the medical image based on the motion curve, thereby enabling motion detection and correction of medical images to be performed taking into account the movement of human organs or lesions without installing a separate monitoring device. [Brief explanation of the drawings]

[0026] Specific embodiments of the present disclosure will be described with reference to the following drawings. [Figure 1] 1 illustrates an embodiment of a myocardial perfusion imaging method performed according to current technology. [Figure 2] 1 is a schematic diagram of a system configuration for detecting and correcting movement in a medical image according to the present disclosure. [Figure 3] 1 is a schematic diagram of an embodiment of a system for detecting and correcting motion in medical images according to the present disclosure; [Figure 4] 1 is a flowchart of a method for medical image motion detection and correction according to the present disclosure. [Figure 5] 1 is a partial embodiment of a method for medical image motion detection and correction according to the present disclosure; [Figure 6A] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 6B] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 6C] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure, where COM represents the center of gravity. [Figure 7] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure, where COM1 and COM2 represent centroid 1 and centroid 2, respectively. [Figure 8] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 9] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 10] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 11] 1 illustrates a partial embodiment of a method for medical image motion detection and correction according to the present disclosure. [Figure 12] 1 is a schematic diagram of an embodiment of a method for motion detection and correction of medical images according to the present disclosure; DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, a description will be given based on specific embodiments so that those skilled in the art can easily understand other advantages and effects of the present application from the contents described in this specification. The structures, proportions, sizes, etc. shown in the drawings attached to this disclosure are intended to be consistent with the description in the specification so that those skilled in the art can understand and read them, and do not limit the limitations that can be implemented in this disclosure. Therefore, any modifications, changes, or adjustments that do not affect the effects that can be produced and the objectives that can be achieved in this disclosure shall fall within the scope of the technical content described in this disclosure.

[0028] FIG. 2 can be viewed as a schematic diagram of a system configuration for performing motion detection and correction of medical images in this disclosure.

[0029] In at least one embodiment, the management platform 201 of the present disclosure is for integrating processing processes for medical images, including receiving and sending medical images, providing user search for medical images, and performing optimization processing of medical images based on user requests. In some embodiments, the management platform 201 can be displayed via any suitable user interface, such as a web page, application page, human-machine interface, or the like, without any particular limitation herein.

[0030] In at least one embodiment, the optimization device 202 according to the present disclosure performs background services for corresponding medical image optimization processes (including motion detection and correction) based on commands submitted by a user on the management platform 201. In some embodiments, the optimization device 202 according to the present disclosure may be implemented in any suitable physical computer system, cloud system, etc., or may be implemented in a computer system integrated with the management platform 201, and is not particularly limited in the present disclosure.

[0031] In at least one embodiment, the scanning device 203 of the present disclosure may be any detection device capable of capturing medical images, including, but not limited to, single photon emission computed tomography, positron emission tomography, magnetic resonance imaging, computed tomography, etc., that acquires medical images of a patient's desired location (e.g., including, but not limited to, the heart, lungs, coronary arteries, liver, stomach, etc.). In some embodiments, the medical images acquired by the scanning device 203 of the present disclosure include corresponding list-mode data, which is useful for non-real-time (e.g., post-mortem) retrospective analysis and correction of the acquired medical images.

[0032] In at least one embodiment, the medical image storage and transmission system 204 of the present disclosure may be a storage system currently employed in any hospital that stores medical images acquired by the scanning device 203 and / or optimized medical images optimized by the optimization device 202.

[0033] In at least one embodiment, the examination room report computer 205 of the present disclosure may be any terminal device used by a physician in a clinic that provides the physician with access to search for or display medical images and / or optimized medical images stored in the medical image storage and transmission system 204.

[0034] In at least one embodiment of the present disclosure, the management platform 201, optimization device 202, scanning device 203, medical image archiving and transmission system 204, and clinic report computer 205 shown in FIG. 2 are configured to communicate with each other using standard Digital Imaging and Communications in Medicine (DICOM) standards, thereby providing a high degree of scalability for the present disclosure. However, the present disclosure is not limited to the above-described components. As an example, depending on operational requirements, any number of the above components may be integrated into the same device, and a single management platform 201 and / or optimization device 202 may be designed to support optimization of medical images from multiple scanning devices 203, without any particular limitations being placed on the present disclosure.

[0035] FIG. 3 is a schematic diagram of a specific embodiment of performing motion detection and correction of medical images in the present disclosure.

[0036] Specifically, in at least one embodiment, the present disclosure may perform motion detection and correction for medical images of the heart taken using a single-photon emission computed tomography (SPECT) scanner. For example, a scanning device 203 (e.g., a single-photon emission computed tomography scanner) according to the present disclosure may include a CdZnTe (CZT) gamma camera equipped with 19 pinhole collimators and 19 CdZnTe (CZT) sensors (e.g., CZT elements including 32×32 pixels) that scans the heart from a right-oblique anterior viewing angle to a left-oblique posterior viewing angle and acquires associated SPECT images (i.e., medical images). The imaging process of the SPECT image by the scanning device 203 includes the steps of scanning the heart by setting dual energy windows that are asymmetrically (e.g., −14% to 23%) and symmetrically (e.g., −9% to 9%), respectively, saving list mode and / or frame mode data corresponding to the SPECT image according to the scan results, transmitting the list mode and / or frame mode data to a workstation built into the scanning device 203 in a standard medical digital imaging and communication format, and resampling and displaying the list mode and / or frame mode data along the short axis, vertical major axis, and horizontal major axis of the heart. However, the descriptions of the specifications, devices, and medical image acquisition methods applicable to the scanning device 203 of the present disclosure are merely examples and do not limit the contents of the present disclosure.

[0037] In the flowchart diagram shown in FIG. 3 , a medical image obtained after gamma photons 301 emitted from a patient 300 (in myocardial perfusion imaging) during imaging are imaged by a scanning device 203 (the imaging step) is indicated by reference numeral 302. In this case, the medical image 302 can be directly stored in a medical image archiving and transmission system 204 so that it can be retrieved by a consultation room report computer 205. However, in a situation where the medical image 302 may be blurred due to the movement of the patient 300 during scanning, the management platform 203 can instruct the optimization device 202 to perform movement detection and correction, including processing procedures such as time dimension segmentation 303, motion center of gravity analysis 304, and deformation model correction 305, in real time on the medical image 302 received from the scanning device 203, to obtain an optimized medical image 306. Furthermore, if the obtained medical image 302 contains list mode data, when blurring of the medical image 302 is later discovered, the management platform 201 can access the medical image 302 in the medical image storage and transmission system 204, and the optimization device 202 can perform retrospective analysis and correction.

[0038] FIG. 4 shows a process flow for performing motion detection and correction of medical images according to the present disclosure (for example, the optimization device 202 performs the processing steps 303 to 305 in FIG. 3 above), and embodiments of each step can be understood through FIGS. 5 to 12 and the following description.

[0039] In at least one embodiment of the present disclosure, in step S10, a user can select medical images for which movement detection and correction are desired (e.g., a set of stress state images and rest state images of a patient's heart obtained in a single scan) through a user interface provided by the management platform 201. In this case, the medical images may be acquired in real time by the management platform 201 when the patient is scanned by the scanning device 203, or may be accessed by the management platform 201 from the medical image storage and transmission system 204 as needed.

[0040] In at least one embodiment of the present disclosure, in step S20, taking into account the displacement (i.e., events) of the target organ (e.g., the heart) captured by the medical images at different respiratory and / or cardiac cycle stages (or the patient's own movement) during the imaging period, the optimization device 202 performs time dimension segmentation 303 shown in FIG. 3 above to divide events included in the list mode data corresponding to the medical images (e.g., events captured using the aforementioned dual energy window) into frames with a fixed time dimension (e.g., in units of 500 milliseconds), and then back-projects the frames onto an object plane (e.g., the surfaces of the 19 CZT sensors of the aforementioned scanning device 203 that are parallel to the corresponding image plane and intersect at the common focus of the multi-pinhole system) to image them. The results are shown in the imaging results of each frame (hereinafter also referred to as frame images) in FIG. 5. In some embodiments, each frame can be divided in units of any value between 100 milliseconds and 500 milliseconds along the fixed time dimension. However, depending on the computational capabilities or work requirements of the optimizer 202, the frames may be divided in other suitable fixed time dimensions, and are not particularly limited in this disclosure.

[0041] In at least one embodiment of the present disclosure, in step S30, as shown in FIG. 5, the optimization device 202 labels a region of interest 501 including a target organ (e.g., a heart) in each frame image to obtain displacement information such as the position, rotation angle, and / or axis (e.g., the long and short axes of the heart) of the target organ in each frame image. In some embodiments, the region of interest 501 including the target organ is spherical. However, the region of interest 501 may be labeled with any other appropriate shape depending on the shape of the target organ or operational requirements, and this is not particularly limited in the present disclosure.

[0042] In some embodiments according to the present disclosure, an example will be given in which a region of interest is labeled for a myocardial region of the heart in a frame image. Labeling of the myocardial region may be performed by a physician using a specific application (e.g., automated software built into the optimization device 202). The actual operation includes the steps of: labeling a 3D ellipsoidal sphere passing through the center of the myocardium on the frame image (e.g., the one acquired in step S20); acquiring the values ​​of 12 spherical parameters from the 3D ellipsoidal sphere, such as the three-dimensional coordinates including the center of the sphere, the radii along three axes (i.e., the axial directions of the heart, such as the short axis, the vertical major axis, and the horizontal major axis); the rotation angles of the three axes; and the length ratio and angle of the base of the heart. Based on the spherical parameters, the specific application fits a 3D active contour model from the 3D ellipsoidal sphere labeled by the physician to form smooth inner and outer surfaces toward the inside and outside, respectively. The fitted frame image, which is the completed 3D ellipsoidal sphere, is a frame image including a region of interest in which the myocardial region is defined.

[0043] In some embodiments according to the present disclosure, the labeling of the region of interest 501 in step S30 can be realized by a deep learning module (e.g., a deep learning module built into the optimization device 202). In this embodiment, the deep learning module includes main structures such as a convolutional layer, a deconvolutional layer, a leaky normalized linear activation layer, a residual connection, and a beating connection, and more than one thousand sets of frame images in which myocardial regions have been labeled by a doctor (e.g., labeling according to the specific application) are used as training data for the deep learning module. The deep learning module, which has been trained and can accurately predict the myocardial regions of the frame images when the unlabeled frame images are input, can be used to label the region of interest 501 in step S30.

[0044] 6A to 6C are schematic diagrams illustrating the steps involved in applying a deep learning module to label a region of interest 501. In at least one embodiment according to the present disclosure, the trained deep learning module can first receive the frame image generated in step S20 (a split screen of the heart in dimensions such as its short axis, vertical major axis, and horizontal major axis, as shown in FIG. 6A ). In some embodiments according to the present disclosure, the deep learning module identifies a region 601 in which the heart is present in the frame image (a regular ellipse (with a rotation angle of zero) extending at least 5 cm from the heart segmentation region selected by the solid-line region 601, as shown in FIG. 6A ), and identifies a binary segmentation region 603 (the region indicated by 603 in FIG. 6B ) including the myocardial region. Next, the deep learning module blurs the binary segmentation region 603 to generate a soft mask, and then applies the soft mask to the initial frame image to exclude image activity in regions other than the central muscle region of the frame image. Finally, the deep learning module fits an elliptical sphere to the myocardial region 602 on the frame image based on the soft mask (the elliptical sphere cross-section corresponding to the solid line indicated by 602 in Figure 6B), expands the elliptical sphere radius outward by 2 cm, and generates a new elliptical sphere representing the region of interest 604 (the region corresponding to the solid line indicated by 604 in Figure 6C).

[0045] In at least one embodiment according to the present disclosure, in step S40, the optimization device 202 performs the motion center of gravity analysis 304 shown in FIG. 3 on the target organ within the region of interest labeled in step S30. In some embodiments according to the present disclosure, the displacement and rotation of the target organ (e.g., the heart) in each frame image are first accurately observed, and the region of interest in each frame image is divided into two. The method for dividing the region of interest can be seen by referring to the difference between FIG. 6C and FIG. 7. For example, the region of interest 604 in each frame image labeled in FIG. 6C is divided into two along the long axis of the heart in step S40, forming sub-regions of interest 701 and 702 related to the heart as shown in FIG. 7. Because the left side of FIG. 7 is a cross-sectional view of the heart in the short axis dimension, the observed sub-regions of interest 701 and 702 are depicted overlapping each other. Next, the two sub-regions of interest 701, 702 of the target organ obtained are used to calculate their respective centers of gravity 703, 704, which assist in tracking the movement of the target organ in each frame image, and allow the motion curve of the target organ during imaging to be obtained.

[0046] The advantage of dividing the region of interest 604 shown in Figure 7C into two sub-regions of interest 701, 702 to obtain two centroids 703, 704 for motion centroid analysis, as compared to the region of interest 604 shown in Figure 6C with only one centroid 605, is shown in Figure 8. 801-803 represent the target organ (e.g., the heart) translated (801), translated and rotated (802), and rotated (803) (i.e., change from solid to dashed pattern), respectively, while 804 and 805 show the effects of 801-803 observed with the single centroid 605 and the two centroids 703, 704, respectively. When a translation 801 of the target organ occurs, it can be clearly tracked using either the single center of gravity 605 or the two centers of gravity 703 and 704 (for example, a change in movement from center of gravity 605 to 605', or a change in movement from centers of gravity 703 and 704 to 703' and 704'), but when a translation rotation 802 or rotation 803 occurs in the target organ, the tracking effect observed with only the single center of gravity 605 is clearly inferior to the tracking effect observed with the two centers of gravity 703 and 704. In particular, when only rotation 803 occurs in the target organ, there is no significant change in center of gravity 605 before and after the rotation (it coincides with 605'), and the rotation of the target organ cannot be tracked.

[0047] However, the method for analyzing the center of gravity of motion according to the present disclosure is not limited to the above description. For example, those skilled in the art may divide the region of interest 604 into sub-regions of interest 701 and 702 along the short axis of the heart (long axis direction), or may divide the region of interest 604 into more than two regions of interest as needed (e.g., depending on the shape characteristics of the target organ) to observe the displacement and rotation of the target organ at more than two centers of gravity.

[0048] Immediately after step S40, after the region of interest 604 is divided into sub-regions of interest 701 and 702, three-dimensional coordinates (e.g., "(x1, y1, z1)," (x2, y2, z2)") corresponding to the centroids 703 and 704 in each frame image are calculated as description values ​​for each frame image. In some embodiments according to the present disclosure, principal component analysis (PCA) can be used to reduce the dimension of the description values ​​for each frame image, and the maximum feature can be obtained from the dimension-reduced description values ​​as the translation / rotation signal of the target organ in each frame image. In some embodiments according to the present disclosure, by filtering each frame image based on the translation / rotation signal of the target organ, excessive high-frequency noise signals can be filtered out, and the motion curve of the target organ during imaging can be calculated.

[0049] In at least one embodiment according to the present disclosure, the motion curve of the target organ during imaging is shown in Fig. 9. Fig. 9 shows the detection of the motion curve of the target organ (heart) relative to the left and right (X axis, upper schematic diagram in Fig. 9), ventrodorsal (Y axis, middle schematic diagram in Fig. 9), and craniocaudal (Z axis, lower schematic diagram in Fig. 9) of the human body during imaging, where 901 is a curve (shown by a dotted line) showing the relative positional change of the center of gravity 703 or 704 on the X axis, Y axis, and Z axis in each frame image, 902 is a curve (shown by a short dashed line) showing the rotation signal of the center of gravity 703 or 704 on the X axis, Y axis, and Z axis calculated based on curve 901, and 903 is a curve (shown by a solid line) showing a more accurate positional change of the center of gravity 703 or 704 obtained by grouping and accumulating each frame image.

[0050] In at least one embodiment of the present disclosure, in step S50, to ensure the effectiveness of the correction of the medical image in the subsequent steps, the optimization device 202 may first determine whether the movement amount of the target organ is excessive according to the motion curve of the target organ. If it is excessive (for example, if the amplitude of the motion curve of the target organ exceeds 50 mm), the radiologist may request the patient to perform a rescan and repeat steps S10 to S40 to obtain a new motion curve of the target organ. If a rescan is not required, the process may proceed to the next step to perform deformation model correction of the medical image.

[0051] The above steps S10 to S50 complete the procedure for detecting movement in the medical image. Depending on the result of the movement detection, the optimization device 202 can subsequently perform the deformation model correction 305 shown in FIG. 3 on the medical image, as described below in steps S60 to S80.

[0052] Because the aforementioned motion curve is average data of the positions of the regions of interest (based on the center of gravity) in all frame images, the optimization device 202 first checks in step S60 whether the position of the region of interest in each frame image accurately corresponds to the motion curve. If the region of interest in each frame image does not accurately correspond to the motion curve, gating can be performed in step S70. If they accurately correspond, optimized reconstruction of medical images can be directly performed in step S80 for each frame image. In some embodiments of the present disclosure, regardless of whether the region of interest in each frame image accurately corresponds to the motion curve, directly performing gating in step S70 for each frame image further contributes to understanding the movement of the target organ and shortens the time required for optimizing reconstruction of medical images in step S80.

[0053] In at least one embodiment of the present disclosure, the gating described in step S70 forms a gating set image by integrating frame images with similar time and / or positional relationships based on the motion curve of the target organ (e.g., the heart) calculated in step S40.

[0054] FIG. 10 illustrates an embodiment of the present disclosure in which each frame image of a cardiac medical image is organized into eight gating sets. The top eight gating sets are gating set images observed in the vertical long-axis plane of the heart, and the bottom eight gating sets are gating set images observed in the short-axis plane of the heart. Furthermore, considering that the motion curve may correspond to the regular motion of the human body's breathing and / or heartbeat, if the patient does not move vigorously during the scan, these eight gating set images can also correspond to each cycle phase of the breathing and / or heartbeat, such as the inspiration / expiration respiratory cycle phase from left to right shown in FIG. 10. Because each gating set image exhibits a repetitive displacement relationship with the gating set image at the end of inspiration (the leftmost gating set image) on the craniocaudal, left-right, and / or ventrodorsal axes of the human body, this step S70 is also referred to as respiratory gating or cardiac gating.

[0055] 10 schematically illustrates an embodiment in which all frame images of a cardiac medical image are combined into eight gating sets according to the stages of the human body's respiratory cycle (i.e., each gating set includes 12.5% ​​of the total number of frame images). However, depending on the needs of medical image correction and optimization effects and operational needs, the number of gating sets can be increased or decreased, an additional gated set image can be formed by further considering the human body's cardiac cycle, or a gating set image can be formed by comprehensively considering the stages of the respiratory cycle and cardiac cycle, and there are no particular limitations in this specification.

[0056] In at least one embodiment according to the present disclosure, in step S80, reconstruction optimization of the medical image is performed based on each accurately positioned frame image (i.e., each frame image of the medical image not processed in step S70) or the gating set image gated in step S70. The medical image reconstruction optimization is performed by performing motion compensation of the remaining frame images or gating set images primarily based on a reference object in each frame image or gating set image (e.g., a frame image or gating set image determined to be in the end of inspiration, in the isovolumic contraction phase of the cardiac cycle, or satisfying both of the above conditions). The motion curves of the human body relative to the craniocaudal, left-right, and / or ventrodorsal axes are used as references for adjusting the frame images or gating set images during the motion compensation (such as rotation, displacement, scaling, deformation, and other adjustments). Furthermore, the correlation coefficient between the reconstructed and optimized medical image and the reference object is used to observe the degree of completion of the medical image reconstruction optimization.

[0057] For example, when reconstructing and optimizing a medical image related to the heart, taking into account the stages of the human body's respiratory cycle, the optimization device 202 selects one frame image or gating set image corresponding to the medical image at the end of inspiration as a reference object from among the frame images or gating set images, and adjusts all pixels contained in the remaining frame images or gating set images relative to the reference object based on their three-dimensional coordinates along the craniocaudal, left-right, and / or ventrodorsal axes of the human body using the motion curve (i.e., adjustments such as rotation, displacement, and scaling described above). The adjusted frame images or gating set images are integrated to reconstruct an optimized medical image, and the optimization of the reconstruction of the medical image is iteratively performed until the optimized medical image is closest to the reference object (e.g., the correlation coefficient reaches a maximum value or the root mean square error reaches a minimum value). When motion compensation of the medical image is completed, the motion compensation effect is obtained from the left diagram (without reconstruction optimization) to the right diagram (completed reconstruction optimization) shown in Figure 11.

[0058] In some embodiments of the present disclosure, the motion compensation performed in step S80 is performed by incorporating the motion compensation program into a maximum a posteriori expectation maximization (MAPEM) algorithm, which can be expressed in the form of the following mathematical formula:

number

number

number

number

number

number

number

number

number

number

[0059] 12 is a schematic diagram illustrating an effect after the optimization device 202 performs steps S10 to S80 on a medical image. For example, in at least one embodiment according to the present disclosure, the optimization device 202 analyzes a medical image of a target organ (e.g., a heart) by parsing it into the form of frame images (e.g., a frame image corresponding to a certain respiratory cycle stage shown in FIG. 1201 and / or a frame image corresponding to a certain cardiac cycle stage shown in FIG. 1202), and then adjusts each frame image (or gated set images) relative to the craniocaudal, left-right, and / or ventrodorsal axes of the human body in consideration of the motion curve of the target organ (e.g., a frame image adjusted to correspond to a certain respiratory cycle stage shown in FIG. 1203 and / or a frame image adjusted to correspond to a certain cardiac cycle stage shown in FIG. 1204) to obtain a reconstructed optimized medical image 1205. Here, the optimization device 202 can directly display the reconstructed optimized medical image 1205 on the user interface of the management platform 201, and can also store the reconstructed optimized medical image 1205 in the medical image storage and transmission system 204 by the management platform 201, so that the doctor can search for it via the examination room report computer 205.

[0060] The present disclosure further provides a computer-readable medium applicable to a computer or computing device having a processor and / or memory, having stored thereon commands that cause the computer or computing device to execute the medical image movement detection and correction method in accordance with the commands via the processor (e.g., CPU, GPU, etc.) and / or memory.

[0061] As described above, the method, system, and computer-readable medium for detecting and correcting movement of medical images disclosed herein can divide a medical image of a target organ into multiple frame images based on list mode data, analyze multiple centroids of a region of interest in the multiple frame images, calculate a motion curve of the target organ during scanning, and then perform reconstruction and optimization of the medical image based on the motion curve, thereby enabling detection and correction of movement of medical images taking into account the movement of human organs or lesions without the need for additional monitoring devices.

[0062] The above-described embodiments merely exemplify the effects of the present disclosure and are not intended to limit the scope of the present disclosure. Those skilled in the art can make various modifications and changes to these embodiments without departing from the spirit of the present disclosure. Therefore, the scope of protection of the rights of the present disclosure is defined in the claims below. [Explanation of symbols]

[0063] 101~103 Example 201 Management Platform 202 Optimization Device 203 Scanning Device 204 Medical Image Storage and Transmission System 205 Examination Room Report Computer 300 patients 301 gamma photon 302 Medical Imaging 303 Temporal Dimension Segmentation 304 Motion center of gravity analysis 305 Deformed Model Correction 306 Optimized Medical Imaging 501 Areas of Interest 601 area 602 Myocardial Region 603 Binary Segmentation Regions 604 Area of ​​Interest 605, 605' center of gravity 701, 702 Sub-area of ​​interest 703, 703' center of gravity 704, 704' center of gravity 801, 802, 803 Status 804, 805 Effects 901, 902, 903 curves 1201, 1202, 1203, 1204 illustrations 1205 Optimized Medical Images S10~S80 steps

Claims

1. a management platform providing a user interface for issuing commands to perform an optimization process on medical images corresponding to the list-mode data of the target organ; an optimization device that performs the optimization process on the medical image based on the command, The optimization process includes: segmenting list-mode data corresponding to the medical image into a plurality of frames having a fixed time dimension and imaging each frame as a plurality of frame images; labeling a region of interest within each of the frame images to include the target organ; calculating a motion curve of the target organ based on the region of interest of each of the frame images; reconstructing the medical image based on the motion curve; displaying the reconstructed medical image on the user interface; Including, In the step of calculating a motion curve of the target organ based on the region of interest of each of the frame images, Dividing the region of interest into a first sub-region of interest and a second sub-region of interest; extracting three-dimensional coordinates of a first centroid of the first sub-region of interest and a second centroid of the second sub-region of interest as descriptive values ​​of the frame image; a step of performing dimension reduction of the description values ​​of each of the frame images by principal component analysis, and determining the maximum feature of each of the description values ​​after dimension reduction as a signal of movement and / or rotation of the target organ; calculating the motion curve by grouping and filtering the frame images based on the translation and / or rotation signals; A system for medical image motion detection and correction, comprising:

2. The optimization device includes a deep learning module; In the step of labeling the region of interest in each frame image, identifying a binary segmentation region including the target organ in each of the frame images by the deep learning module; blurring the binary segmentation region to generate a soft mask using the deep learning module; applying each of the soft masks to each of the frame images by the deep learning module; A step of fitting an initial ellipsoid to the target organ in each of the frame images based on each of the soft masks by the deep learning module; Expanding each of the initial ellipsoidal spheres outward by a predetermined distance according to its radius using the deep learning module to generate an ellipsoidal sphere representing the region of interest; The system of claim 1 , comprising:

3. 2. The system of claim 1, wherein the target organ is a heart, and the division of each region of interest into the first sub-region of interest and the second sub-region of interest is performed in a minor axis direction along a major axis of the heart.

4. The movement curve is plotted relative to one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis of the human body; In the step of reconstructing the medical image based on the motion curve, selecting a reference object from each of the frame images; and performing motion compensation for each of the frame images by referring to the motion curve; The motion compensation includes: adjusting all pixels other than the reference object included in each of the frame images in any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis based on the three-dimensional coordinates of the pixels; a step of repeatedly performing the adjustment process for each of the frame images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; a step of integrating the motion-compensated frame images to reconstruct the medical image; The system of claim 1 , comprising:

5. The system of claim 1 further characterized in that, after calculating the motion curve of the target organ based on each of the centers of gravity, if each of the regions of interest in each of the frame images does not accurately correspond to the motion curve, the optimization device executes the optimization process and also performs gating of each of the frame images by integrating frame images among the frame images that have similar time and / or positional relationships defined by cycle stages of breathing and / or heartbeat of the human body into a predetermined number of gating set images.

6. The motion curve is drawn relative to any one of a craniocaudal axis, a left-right axis, and a ventrodorsal axis of the human body, and the medical image is reconstructed based on the motion curve, selecting a reference object from each of the gating set images; motion-compensating each of the gating set images using the motion curve as a reference; Including, For each of the gating set images, motion compensation includes: adjusting all pixels other than the reference object included in each gating set image in any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis based on the three-dimensional coordinates of the pixels relative to the reference object; Repeating the adjustment process for each of the gating set images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; a step of integrating the motion-compensated gating set images to reconstruct the medical image; The system of claim 5, comprising:

7. 2. The system of claim 1, wherein the fixed time dimension is in units of 100 milliseconds to 500 milliseconds.

8. a scanning device for imaging the target organ and acquiring the medical image, the scanning device being selected from the group consisting of a single photon emission computed tomography, a positron emission tomography, a magnetic resonance imaging device, and a computed tomography device, or any combination thereof; a medical image storage and transmission system for storing the medical image and the reconstructed medical image; a consultation room report computer for retrieving and displaying the medical images and reconstructed medical images; The system of claim 1 further comprising:

9. acquiring a medical image corresponding to list-mode data of the target organ; segmenting list-mode data corresponding to the medical image into a plurality of frames having a fixed time dimension, and imaging each of the frames as a plurality of frame images; labeling a region of interest within each of the frame images to include the target organ; calculating a motion curve of the target organ based on the region of interest of each of the frame images; reconstructing the medical image based on the motion curve; Including, In the step of calculating a motion curve of the target organ based on the region of interest of each of the frame images, Dividing the region of interest into a first sub-region of interest and a second sub-region of interest; extracting three-dimensional coordinates of a first centroid of the first sub-region of interest and a second centroid of the second sub-region of interest as descriptive values ​​of the frame image; a step of performing dimension reduction of the description values ​​of each of the frame images by principal component analysis, and determining the maximum feature of each of the description values ​​after dimension reduction as a signal of movement and / or rotation of the target organ; calculating the motion curve by grouping and filtering the frame images based on the translation and / or rotation signals; 1. A method for medical image motion detection and correction, comprising:

10. The step of labeling a region of interest in each of the frame images so as to include the target organ comprises: Identifying a binary segmentation region including the target organ in each of the frame images using a deep learning module; blurring each of the binary segmentation regions with the deep learning module to generate a soft mask; applying each of the soft masks to each of the frame images by the deep learning module; fitting an initial ellipsoid to the target organ in each of the frame images based on each of the soft masks by the deep learning module; Expanding each of the initial ellipsoidal spheres outward by a predetermined distance based on its radius using the deep learning module to generate an ellipsoidal sphere representing the region of interest; 10. The method of claim 9, further comprising:

11. 10. The method of claim 9, wherein the target organ is a heart, and the division of each region of interest into the first and second sub-regions of interest is performed in a minor axis direction along the major axis of the heart.

12. The motion curve is drawn relative to any one of a craniocaudal axis, a left-right axis, and a ventrodorsal axis of a human body, and the step of reconstructing the medical image based on the motion curve includes: selecting a reference object from each of the frame images; and performing motion compensation for each of the frame images by referring to the motion curve; The motion compensation includes: adjusting all pixels other than the reference object included in each of the frame images in any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis based on the three-dimensional coordinates of the pixels; a step of repeatedly performing the adjustment process for each of the frame images until a correlation coefficient between the integration of each of the frame images after the adjustment process and the reference object reaches a maximum value; a step of integrating the motion-compensated frame images to reconstruct the medical image; 10. The method of claim 9, comprising:

13. After calculating the motion curve of the target organ based on the centers of gravity, if the regions of interest of the frame images do not accurately correspond to the motion curve, 10. The method according to claim 9, further comprising a step of gating each of the frame images by integrating frame images having similar time and / or positional relationships defined by respiratory and / or cardiac cycle phases of the human body into a predetermined number of gating set images.

14. The motion curve is drawn relative to any one of a craniocaudal axis, a left-right axis, and a ventrodorsal axis of the human body, and the medical image is reconstructed based on the motion curve, selecting a reference object from each of the gating set images; and motion-compensating each of the gating set images using the motion curve as a reference; For each of the gating set images, motion compensation includes: adjusting all pixels other than the reference object included in each of the gating set images in any one of the craniocaudal axis, the left-right axis, and the ventrodorsal axis based on the three-dimensional coordinates of the pixels relative to the reference object; Iteratively adjusting each of the gating set images until a correlation coefficient between the integration of each of the gating set images after the adjustment and the reference object reaches a maximum value; a step of integrating the motion-compensated gating set images to reconstruct the medical image; 14. The method of claim 13, comprising:

15. 10. The method of claim 9, wherein the medical image is obtained by imaging the target organ with a scanning device, the scanning device being selected from any one or combination of the group consisting of a single photon emission computed tomography, a positron emission tomography, a magnetic resonance imaging device, and a computed tomography device.

16. 10. The method of claim 9, wherein the fixed time dimension is in units of 100 ms to 500 ms.

17. A computer-readable storage medium that is applied to a computer and stores instructions for executing the method for detecting and correcting movement in medical images according to any one of claims 9 to 16.

Images