Medical image processing apparatus, medical image processing method, storage medium, and radiation therapy apparatus

Abstract

The present application relates to a medical image processing apparatus, a medical image processing method, a storage medium, and a radiation therapy apparatus. The medical image processing apparatus of an embodiment includes a first image acquisition unit, a second image acquisition unit, a first likelihood distribution calculation unit, a tracking possibility determination unit, and a tracking unit. The first image acquisition unit acquires a first image of a fluoroscopic image of a patient. The second image acquisition unit acquires a second image of the fluoroscopic image of the patient generated at a different time from the first image. The first likelihood distribution calculation unit calculates a first likelihood distribution representing a distribution of likelihoods of the first image, the likelihoods representing similarities of an object. The tracking possibility determination unit determines whether the object can be tracked based on the first likelihood distribution. The tracking unit tracks a position of the object in the second image according to the determination result.

Description

Technical Field

[0001] Embodiments of the present invention relate to a medical image processing apparatus, a medical image processing method, a storage medium, and a radiation therapy apparatus. Background Art

[0002] Radiation therapy is a treatment method that destroys a lesion by irradiating the lesion inside a patient with radiation. At this time, the radiation needs to be accurately irradiated to the position of the lesion. This is because there are cases where radiation is irradiated to normal tissue inside the patient, which may affect the normal tissue. Therefore, when performing radiation therapy, first in the treatment planning stage, Computed Tomography (CT) is performed in advance to three-dimensionally grasp the position of the lesion inside the patient. Then, based on the grasped position of the lesion, the direction of irradiating radiation and the intensity of the irradiated radiation are planned to reduce the irradiation to normal tissue. After that, in the treatment stage, the position of the patient is aligned with the position of the patient in the treatment planning stage, and the lesion is irradiated with radiation according to the irradiation direction and irradiation intensity planned in the treatment planning stage.

[0003] In the alignment of the patient in the treatment stage, the three-dimensional CT data is virtually arranged in the treatment room, and the position of the examination table is adjusted so that the position of the patient actually lying on the movable examination table in the treatment room matches the position of the three-dimensional CT data. More specifically, by comparing two images, namely an X-ray fluoroscopic image of the inside of the patient captured while the patient is lying on the examination table, and a Digitally Reconstructed Radiograph (DRR) image obtained by virtually reconstructing an X-ray fluoroscopic image from the three-dimensional CT image captured at the time of treatment planning, the offset of the patient's position between the respective images is obtained. Then, the examination table is moved based on the offset of the patient's position obtained by the image comparison, so that the positions of the lesion, bones, and the like inside the patient are aligned with those at the time of treatment planning. After that, the lesion is irradiated with radiation while comparing the newly captured X-ray fluoroscopic image with the DRR image.

[0004] When the patient's lesion is located in an organ that moves due to respiratory or cardiac activity, such as the lung or liver, it is necessary to determine the position of the lesion during irradiation. For example, the position of the lesion can be tracked by capturing an X-ray fluoroscopic image of the patient irradiated with radiation. In the case where the lesion is not clearly captured in the X-ray fluoroscopic image, the position of the lesion can be indirectly tracked by tracking a marker indwelled in the patient. As methods of irradiating radiation, there are tracking irradiation that tracks the position of the lesion for irradiation, and gating irradiation (Japanese: waiting ambush irradiation) that performs irradiation when the lesion reaches the position at the time of treatment planning. These irradiation methods are referred to as respiratory-synchronized irradiation methods.

[0005] However, when objects such as lesions or markers overlap with organs with low X-ray transmittance, such as the liver, they may not be clearly visible in X-ray fluoroscopy images. Furthermore, patterns like ribs can sometimes be difficult to distinguish from markers in X-ray fluoroscopy images. In these cases, the location of lesions, markers, or other objects may not be traceable.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2018-82767 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] The problem to be solved by the present invention is to provide a medical image processing device, a medical image processing method, a storage medium, and a radiotherapy device that can track objects inside a patient’s body with high precision based on the patient’s fluoroscopic image.

[0011] Methods for solving problems

[0012] The medical image processing apparatus of this embodiment is an apparatus for tracking the position of an object within a patient's body. The medical image processing apparatus includes a first image acquisition unit, a second image acquisition unit, a first likelihood distribution calculation unit, a tracking feasibility determination unit, and a tracking unit. The first image acquisition unit acquires multiple first images, which are perspective images of the patient taken from different directions. The second image acquisition unit acquires multiple second images, which are perspective images of the patient generated at different times than the first images and taken from different directions. The first likelihood distribution calculation unit calculates a first likelihood distribution representing the distribution of likelihoods for each of the first images acquired by the first image acquisition unit, where the likelihood represents the similarity of the objects. The tracking feasibility determination unit determines whether the object can be tracked based on the first likelihood distribution calculated by the first likelihood distribution calculation unit. The tracking unit tracks the position of the object in the second images acquired by the second image acquisition unit according to the determination result of the tracking feasibility determination unit. Attached Figure Description

[0013] Figure 1 This is a block diagram showing the general configuration of the treatment system according to the first embodiment.

[0014] Figure 2 This is a block diagram showing the general configuration of the medical image processing apparatus according to the first embodiment.

[0015] Figure 3 is a diagram showing an example of a first image of the first embodiment.

[0016] Figure 4 is a diagram showing an example of a template used in calculation of the separation degree of the first embodiment.

[0017] Figure 5 is a flowchart showing the operation of the medical image processing apparatus in the pre-treatment stage for the patient to be treated according to the first embodiment.

[0018] Figure 6 is a flowchart showing the operation of the medical image processing apparatus during the treatment of the patient according to the first embodiment.

[0019] Figure 7 is a block diagram showing a general configuration of a medical image processing apparatus according to the second embodiment.

[0020] Figure 8 is a flowchart showing the operation of the medical image processing apparatus in the pre-treatment stage for the patient to be treated according to the second embodiment.

[0021] Figure 9 is a block diagram showing a general configuration of a radiation therapy apparatus according to the third embodiment.

[0022] Figure 10 is a flowchart showing the operation of the radiation therapy apparatus during the treatment of the patient according to the third embodiment. DETAILED DESCRIPTION

[0023] Hereinafter, the medical image processing apparatus, the medical image processing method, the storage medium, and the radiation therapy apparatus according to the embodiments will be described with reference to the accompanying drawings.

[0024] (First Embodiment)

[0025] Figure 1 is a block diagram showing a general configuration of a treatment system according to the first embodiment. The treatment system 1 includes, for example, a treatment table 10, a couch control unit 11, two radiation sources 20 (radiation source 20-1 and radiation source 20-2), two radiation detectors 30 (radiation detector 30-1 and radiation detector 30-2), a therapeutic beam irradiation gate 40, an irradiation control device 50, and a medical image processing apparatus 100.

[0026] In addition, in Figure 1The hyphen "-" and the numbers following it in the various reference numerals are used to distinguish the correspondences. More specifically, in the correspondence between radiation source 20 and radiation detector 30, it indicates that radiation source 20-1 and radiation detector 30-1 correspond to each other and form a group, and it indicates that radiation source 20-2 and radiation detector 30-2 correspond to each other and form a group. Furthermore, in the following description, when multiple identical constituent elements are represented without distinction, the hyphen "-" and the numbers following it are not shown.

[0027] Treatment table 10 is an examination table that fixes the subject (patient) P to receive radiation therapy. Examination table control unit 11 is a control unit that controls the translation and rotation mechanisms provided on the treatment table 10 to change the direction of the treatment beam B irradiated onto the patient P fixed on the treatment table 10. For example, the examination table control unit 11 controls the translation and rotation mechanisms of the treatment table 10 in three-axis directions, i.e., six-axis directions.

[0028] Radiation source 20-1 irradiates the patient P from a predetermined angle with radiation r-1 for fluoroscopy. Radiation source 20-2 irradiates the patient P from a predetermined angle different from radiation source 20-1 with radiation r-2 for fluoroscopy. Radiation r-1 and radiation r-2 are, for example, X-rays. Figure 1 The image shows X-ray imaging of patient P, fixed on treatment table 10, from two directions. Additionally, in... Figure 1 The diagram of the control unit that controls the irradiation by radiation r from radiation source 20 is omitted.

[0029] Radiation detector 30-1 detects radiation r-1 that has irradiated from radiation source 20-1 and passed through the patient P's body, generating an X-ray fluoroscopic image of the patient P's body corresponding to the energy level of the detected radiation r-1. Radiation detector 30-2 detects radiation r-2 that has irradiated from radiation source 20-2 and passed through the patient P's body, generating an X-ray fluoroscopic image of the patient P's body corresponding to the energy level of the detected radiation r-2.

[0030] The radiation detectors 30 are arranged in a two-dimensional array to generate digital images, which represent the energy levels of the radiation r reaching each X-ray detector as digital values, serving as X-ray fluoroscopic images. The radiation detectors 30 can be, for example, flat panel detectors (FPDs), image intensifiers, or color image intensifiers. In the following description, each radiation detector 30 is assumed to be an FPD. The radiation detectors 30 (FPDs) output the generated X-ray fluoroscopic images to the medical image processing apparatus 100. Furthermore, in... Figure 1 The diagram of the control unit that controls the generation of X-ray fluoroscopic images using the radiation detector 30 is omitted.

[0031] In addition, Figure 1 The diagram shows the configuration of a treatment system 1 equipped with two imaging devices (a group of radiation sources 20 and radiation detectors 30). However, the number of imaging devices in the treatment system 1 is not limited to two. For example, the treatment system 1 may also be equipped with three or more imaging devices (three or more groups of radiation sources 20 and radiation detectors 30).

[0032] The therapeutic beam irradiation gate 40 irradiates radiation intended to destroy the lesion within patient P as the therapeutic beam B. The therapeutic beam B is, for example, X-rays, gamma rays, electron beams, proton beams, neutron beams, heavy particle beams, etc. The therapeutic beam B is delivered in a straight line from the therapeutic beam irradiation gate 40 to patient P (more specifically, to the lesion within patient P).

[0033] The irradiation control device 50 controls the irradiation of the therapeutic beam B through the therapeutic beam irradiation gate 40. The irradiation control device 50 irradiates the therapeutic beam B to the patient P from the therapeutic beam irradiation gate 40 according to an irradiation instruction signal that indicates the irradiation timing of the therapeutic beam B output from the medical image processing device 100.

[0034] exist Figure 1 The diagram shows a treatment system 1 with a fixed treatment beam irradiation gate 40, but it is not limited to this; the treatment system 1 may also have multiple treatment beam irradiation gates. For example, the treatment system 1 may also have a treatment beam irradiation gate that irradiates a treatment beam towards the patient P from a horizontal direction. Furthermore, the treatment system 1 may also be configured to irradiate a treatment beam towards the patient P from various directions by rotating a treatment beam irradiation gate around the patient P. More specifically, Figure 1 The therapeutic beam irradiation gate 40 shown can also be relative to Figure 1The treatment system 1 shown is configured to rotate 360 ​​degrees relative to the horizontal Y-axis. This configuration is called a rotating gate type treatment system. Furthermore, in the rotating gate type treatment system, the radiation source 20 and the radiation detector 30 also rotate 360 ​​degrees simultaneously relative to the same axis as the rotation axis of the treatment beam irradiation gate 40.

[0035] The medical image processing device 100 determines the irradiation timing of the therapeutic beam B to be irradiated onto the lesion of patient P by tracking the positions of objects within patient P's body, such as the lungs and liver, which move due to the patient P's breathing and heartbeat. The objects within patient P's body are metallic markers placed within the patient P's body, but are not limited to this. For example, the objects within patient P's body could also be lesions within patient P's body. In the following description, examples where the object is a marker will be explained.

[0036] The medical image processing device 100 automatically determines the irradiation timing of the therapeutic beam B to the lesion by tracking the images of markers in the X-ray fluoroscopic images of the patient P captured in real time by each radiation detector 30. At this time, the medical image processing device 100 determines whether the position of the image of the marker placed in the patient P, which is being tracked, is within the prescribed range for radiotherapy. Then, if the position of the marker image is within the prescribed range, the medical image processing device 100 outputs an irradiation instruction signal instructing the irradiation of the therapeutic beam B to the irradiation control device 50. Consequently, the irradiation control device 50 irradiates the therapeutic beam B into the therapeutic beam irradiation gate 40 according to the irradiation instruction signal output by the medical image processing device 100.

[0037] Furthermore, the medical image processing apparatus 100 performs positioning image processing to align the current position of the patient P with a position predetermined during the planning stage before radiotherapy, such as the treatment planning stage. The medical image processing apparatus 100 compares a DRR image, obtained by virtually reconstructing an X-ray fluoroscopy image from a three-dimensional CT image or similar image taken before radiotherapy, with the current X-ray fluoroscopy image output by each radiation detector 30, thereby automatically searching for a suitable position for the patient P to undergo radiotherapy. Then, the medical image processing apparatus 100 determines the amount of movement required to move the current position of the patient P, fixed on the treatment table 10, to a predetermined appropriate position for radiotherapy. Details related to the configuration and processing of the medical image processing apparatus 100 will then be described.

[0038] In addition, the medical image processing device 100 and the radiation detector 30 can be connected via LAN (Local Area Network) and WAN (Wide Area Network), respectively.

[0039] Next, the configuration of the medical image processing device 100 constituting the treatment system 1 will be described. Figure 2 This is a block diagram showing the general configuration of the medical image processing apparatus according to the first embodiment. Figure 2 The medical image processing device 100 shown includes a first image acquisition unit 101, a first likelihood distribution calculation unit 102, a tracking feasibility determination unit 103, a tracking unit 104, a second image acquisition unit 105, and a second likelihood distribution calculation unit 106.

[0040] The first image acquisition unit 101 acquires multiple first images, which are fluoroscopic images of the patient P and are taken from different directions. The first images may be X-ray fluoroscopic images taken by the radiation detector 30 in the aforementioned location of the patient P, or DRR images obtained by virtually reconstructing X-ray fluoroscopic images based on three-dimensional CT images or the like taken before radiation therapy.

[0041] The first likelihood distribution calculation unit 102 calculates a first likelihood distribution, which represents the distribution of the likelihood of each of the plurality of first images 60 acquired by the first image acquisition unit 101. The likelihood represents the similarity of markers (objects).

[0042] Figure 3 This is a diagram illustrating an example of a first image according to the first embodiment. The first image 60 includes an image of a marker 62 placed inside the patient P and an image of a lesion 63 inside the patient P. The first likelihood distribution calculation unit 102 calculates not only the likelihood distribution for the entire first image 60, but also the likelihood distribution for the tracking region 61. The tracking region 61 is the area where the position of the marker 62 is tracked, encompassing the area where the marker 62 moves due to the patient P's breathing and heartbeat. Thus, by limiting the object for calculating the first likelihood distribution to a portion of the first image 60 (the tracking region 61), the processing load on the first likelihood distribution calculation unit 102 can be reduced.

[0043] For example, the tracking region 61 is determined based on the position of the marker 62 specified in the CT image during treatment planning. Furthermore, the tracking region 61 is determined taking into account a margin of error that may occur during actual treatment. For example, the tracking region 61 could also be a three-dimensional region centered on the position of the marker 62 on the CT image, with an additional margin, projected onto the first image. Additionally, the tracking region 61 can be determined based on a margin determined considering the patient's condition immediately preceding treatment.

[0044] In addition, Figure 3In the above, the marker 62 is configured as a rod-shaped marker, but the shape of the marker 62 is not limited thereto. The marker 62 may also be, for example, a spherical marker.

[0045] Next, a calculation method of the likelihood performed by the first likelihood distribution calculation unit 102 will be described. The first likelihood distribution calculation unit 102 previously holds an image pattern obtained when the marker 62 is projected as a template image, and calculates the similarity between the first image 60 and the template image, thereby calculating the likelihood.

[0046] In a case where the projected shape of the marker 62 can be predicted in advance, such as when the marker 62 is spherical, an image depicting the projection can be used as the template image. Furthermore, in a case where the projection of the marker 62 changes depending on the posture of the marker 62 when indwelling in the body of the patient P, a plurality of template images depicting projections corresponding to the postures of the marker 62 may be generated, and one of the plurality of template images may be selected for use.

[0047] The similarity, which is a numerical value of the likelihood, is represented by the pixel value and the spatial correlation value between the template image and the first image 60. For example, the similarity may be the reciprocal of the difference in pixel values at each corresponding pixel position. Further, the similarity may also be a cross-correlation value at the corresponding pixel position.

[0048] Furthermore, the first likelihood distribution calculation unit 102 may generate a discriminator by a machine learning method using template images including the marker 62 as learning data, and calculate the likelihood using the generated discriminator. For example, a plurality of template images including the marker 62 and first images 60 of positions not including the marker 62 are prepared, and learning is performed thereon by a machine learning method, thereby generating a discriminator. By using the generated discriminator, it is possible to discriminate whether the marker 62 is included or not.

[0049] Further, as the machine learning method, a support vector machine, random forest, deep learning, or the like can be used. The first likelihood distribution calculation unit 102 calculates a real value calculated using these machine learning methods as the likelihood. As the real value, for example, a distance to the discrimination boundary, a ratio of discriminators that determine that the target is included, an output value of Softmax, or the like can be used.

[0050] Furthermore, the first likelihood distribution calculation unit 102 may calculate the likelihood using a degree of separation. Figure 4 is a diagram illustrating an example of a template used in calculation of the degree of separation according to the first embodiment. In Figure 4In one example of the template 70 shown, there is a first region 71 where the rod-shaped marker 62 is present, and a second region 72 outside of the first region 71. That is, the second region 72 is the region where the rod-shaped marker 62 is not present.

[0051] In the calculation of separation, when there is a region in the first image 60 that is identical to the template 70, the histogram of pixel values ​​contained in the first image 60 is classified into a histogram of pixel values ​​belonging to the first region 71 and a histogram of pixel values ​​belonging to the second region 72, respectively. This is because, when the image of the marker 62 captured in the first image 60 overlaps with the first region 71, the frequency of pixel values ​​of darker pixels in the histogram of pixel values ​​belonging to the first region 71 increases, and the frequency of pixel values ​​of brighter pixels in the histogram of pixel values ​​belonging to the second region 72 increases.

[0052] The first likelihood distribution calculation unit 102 uses the Fisher criterion to quantify the separability of the histogram of pixel values ​​as described above. Then, the first likelihood distribution calculation unit 102 calculates the ratio of the average discrete pixel value (within-group discreteness) of pixels belonging to each region to the discrete pixel value (between-group discreteness) between each region, and calculates this ratio as the separation degree. Thus, the first likelihood distribution calculation unit 102 calculates the likelihood degree using the separation degree calculated with the template 70. Using the calculated likelihood degree, the first likelihood distribution calculation unit 102 calculates a first likelihood distribution representing the distribution of the likelihood degrees of each of the plurality of first images 60 acquired by the first image acquisition unit 101.

[0053] The tracking feasibility determination unit 103 determines whether the marker 62 can be tracked based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. Specifically, the tracking feasibility determination unit 103 calculates the tracking difficulty in each shooting direction of the first image 60 based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. Furthermore, if the tracking difficulty is higher than a predetermined threshold, the tracking feasibility determination unit 103 determines that the marker 62 cannot be tracked.

[0054] The tracking feasibility determination unit 103 calculates the tracking difficulty based on the statistics of the first likelihood distribution. The statistics can be, for example, the mean or discrete value of the first likelihood distribution, but are not limited to these. For instance, the statistics could be the maximum value of the first likelihood distribution, or a value calculated using a linear formula that unifies these values. Furthermore, the tracking feasibility determination unit 103 can calculate the tracking difficulty by narrowing down the statistics of the first likelihood distribution around the location of the marker 62, after obtaining the position of the marker 62 corresponding to the first likelihood distribution. The tracking feasibility determination unit 103 outputs the tracking feasibility determination result for the marker 62 in each shooting direction to the tracking unit 104 and the second image acquisition unit 105.

[0055] The tracking unit 104 tracks the position of the marker 62 in the second image acquired by the second image acquisition unit 105 based on the determination result of the tracking feasibility determination unit 103. The details of the tracking process using the tracking unit 104 will then be described.

[0056] The second image acquisition unit 105 acquires multiple second images, which are perspective images of patient P generated at different times than the first image and taken from different directions. The aforementioned first image is a perspective image generated based on a three-dimensional image of patient P taken during treatment planning, or a perspective image taken from the same shooting direction as the second image immediately before treatment. The second image is a perspective image taken by the radiation detector 30 during treatment of patient P. The second images are repeatedly taken at predetermined time intervals to track marker 62.

[0057] The first image should ideally be taken from the same direction as the second image. For example, if the second image is taken from a different direction than when the patient P was positioned, the first image may not be a perspective image taken when the patient P was positioned, but rather a DRR image obtained by simulating a perspective image taken from the same direction as the second image. Furthermore, the first image may be the second image taken during the previous treatment, or it may be an image taken immediately before treatment from the same direction as the second image. Additionally, the first image may be a dynamic image of more than one respiratory cycle.

[0058] Furthermore, before treating patient P with irradiation using the therapeutic beam B, the second image acquisition unit 105 determines the shooting direction of the second image used for tracking the position of the marker 62 in the tracking unit 104 based on the determination result of the tracking feasibility determination unit 103. For example, if the determination result of the first image in the shooting direction of radiation detector 30-1 is "trackable" and the determination result of the first image in the shooting direction of radiation detector 30-2 is "trackable", the second image acquisition unit 105 acquires both the second image captured by radiation detector 30-1 and the second image captured by radiation detector 30-2. Conversely, if the determination result of the first image in the shooting direction of radiation detector 30-1 is "trackable" and the determination result of the first image in the shooting direction of radiation detector 30-2 is "untrackable", the second image acquisition unit 105 acquires only the second image captured by radiation detector 30-1. Furthermore, if the determination result of the first image in the shooting direction of radiation detector 30-1 is "untraceable" and the determination result of the first image in the shooting direction of radiation detector 30-2 is "traceable", the second image acquisition unit 105 acquires only the second image captured by radiation detector 30-2.

[0059] Thus, by not acquiring a second image in the shooting direction that is determined to be untrackable marker 62, the second image acquisition unit 105 can suppress radiation to patient P and improve the success rate of tracking marker 62.

[0060] The second likelihood distribution calculation unit 106 calculates a second likelihood distribution representing the distribution of the likelihood of the second image acquired by the second image acquisition unit 105. Here, the second likelihood distribution calculation unit 106 calculates not only the second likelihood distribution for the entire second image, but also the second likelihood distribution for the tracking region 61 within the second image. The calculation method for the second likelihood distribution is the same as that for the first likelihood distribution, therefore, its description is omitted.

[0061] The tracking unit 104 determines the position of the marker 62 in the second image based on the second likelihood distribution calculated by the second likelihood distribution calculation unit 106. For example, the tracking unit 104 determines the position of the marker 62 as the position where the second likelihood distribution has the maximum value.

[0062] When second images are acquired from two directions, once the position of marker 62 in one second image is determined, the position of marker 62 in the other second image is geometrically defined on an epipolar line. Therefore, the tracking unit 104 can determine the position of marker 62 as the position where the likelihood distribution on the epipolar line is the maximum value in the second likelihood distribution of the other second image. When second images are acquired from two directions, the tracking unit 104 can geometrically convert the position of marker 62 into three-dimensional coordinates for calculation.

[0063] The tracking unit 104 determines the irradiation timing of the treatment beam B based on the determined position of the marker 62. The positional relationship between the marker 62 and the lesion 63 can be determined based on the fluoroscopic image of the patient P obtained during treatment planning. Therefore, when the position of the lesion 63 in the patient P, determined based on the position of the marker 62, is within a predetermined range, the tracking unit 104 sends an irradiation instruction signal instructing the irradiation of the treatment beam B to the irradiation control device 50.

[0064] The irradiation control device 50 irradiates the therapeutic beam B from the therapeutic beam irradiation gate 40 according to the irradiation instruction signal received from the medical image processing device 100. Thus, the therapeutic beam B can be irradiated onto the lesion 63 within the patient P.

[0065] On the other hand, when the location of the lesion 63 in the patient P, determined based on the position of the marker 62, is not within a predetermined range, the tracking unit 104 does not send an irradiation instruction signal to the irradiation control device 50. This suppresses the irradiation of the therapeutic beam B to sites other than the lesion 63.

[0066] Furthermore, some or all of the functions of the constituent elements of the aforementioned medical image processing device 100 may include, for example, a hardware processor such as a CPU (Central Processing Unit) and a storage device (a storage device with a non-volatile storage medium) storing programs (software). Various functions can be implemented by the processor executing the programs. Additionally, some or all of the functions of the constituent elements of the aforementioned medical image processing device 100 may be implemented using hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), including the circuitry section. Various functions can also be implemented through the cooperation of software and hardware. Furthermore, some or all of the functions of the constituent elements of the aforementioned medical image processing device 100 may be implemented using a dedicated LSI.

[0067] Here, the program (software) can be pre-stored in a storage device (a storage device with a non-volatile storage medium) provided by the treatment system 1, such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), or flash memory. It can also be stored in a removable storage medium (non-volatile storage medium) such as a DVD or CD-ROM. This storage medium is installed on a drive device provided by the treatment system 1, thereby being installed on the storage device provided by the treatment system 1. Alternatively, the program (software) can be pre-downloaded from other computer devices via a network and installed on the storage device provided by the treatment system 1.

[0068] Figure 5 This is a flowchart illustrating the operation of the medical image processing apparatus in the stage preceding the treatment of the patient according to the first embodiment. Before the patient P is treated with irradiation using the therapeutic beam B, the medical image processing apparatus 100 performs processing based on this flowchart.

[0069] First, the first image acquisition unit 101 acquires multiple first images, which are fluoroscopic images of the patient P and are taken from different directions (S101). As described above, the first image may be an X-ray fluoroscopic image taken by the radiation detector 30 in the location of the patient P, or it may be a DRR image obtained by virtually reconstructing the X-ray fluoroscopic image based on a three-dimensional CT image or the like taken before the radiation therapy.

[0070] Next, the first likelihood distribution calculation unit 102 calculates a first likelihood distribution representing the distribution of the likelihoods of each of the plurality of first images 60 acquired by the first image acquisition unit 101, wherein the likelihood represents the similarity of markers (objects) (S102). Here, the first likelihood distribution calculation unit 102 calculates not only the overall likelihood distribution of the first images 60, but also... Figure 3 The likelihood distribution of the tracking region 61 is shown.

[0071] Next, the tracking feasibility determination unit 103 determines whether the marker 62 can be tracked based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102 (S103). For example, the tracking feasibility determination unit 103 calculates the tracking difficulty in each shooting direction of the first image 60 based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. Furthermore, if the tracking difficulty is higher than a predetermined threshold, the tracking feasibility determination unit 103 determines that the marker 62 cannot be tracked.

[0072] Subsequently, the second image acquisition unit 105 determines the shooting direction of the second image based on the determination result of the tracking feasibility determination unit 103 (S104), and ends the processing based on this flowchart. In addition, when treating the patient P with irradiation using the therapeutic beam B, the marker 62 is tracked based on the second image corresponding to the shooting direction determined in step S104.

[0073] Figure 6 This is a flowchart illustrating the operation of the medical image processing device during the treatment of a patient according to the first embodiment. During the treatment phase where the patient P is irradiated with a therapeutic beam B, the medical image processing device 100 repeatedly performs the processing based on this flowchart at predetermined time intervals.

[0074] First, the second image acquisition unit 105 acquires the second image corresponding to the shooting direction determined in step S104 (S201). The second image is an image captured by the radiation detector 30 during the treatment of the patient P. Specifically, for shooting directions where the tracking difficulty is higher than a predetermined threshold, the second image acquisition unit 105 does not acquire the second image, but only acquires the second image where the tracking difficulty is lower than the predetermined threshold.

[0075] Next, the second likelihood distribution calculation unit 106 calculates a second likelihood distribution (S202) representing the distribution of the likelihood of the second image acquired by the second image acquisition unit 105. The calculation method of the second likelihood distribution is the same as the calculation method of the first likelihood distribution.

[0076] Next, the tracking unit 104 determines the position of the marker 62 (object) in the second image based on the second likelihood distribution calculated by the second likelihood distribution calculation unit 106 (S203). For example, the tracking unit 104 determines the position of the marker 62 as the position where the second likelihood distribution has the maximum value.

[0077] Subsequently, the tracking unit 104 determines the irradiation timing of the therapeutic beam B based on the determined position of the marker 62 (object) (S204). For example, when the position of the lesion 63 in the patient P, determined based on the position of the marker 62, is within a predetermined range, the tracking unit 104 sends an irradiation instruction signal instructing the irradiation of the therapeutic beam B to the irradiation control device 50. The irradiation control device 50 irradiates the therapeutic beam B from the therapeutic beam irradiation gate 40 according to the irradiation instruction signal received from the medical image processing device 100.

[0078] As explained above, in the first embodiment, the first image acquisition unit 101 acquires multiple first images, which are perspective images of patient P and are captured from different directions. The second image acquisition unit 105 acquires multiple second images, which are perspective images of patient P generated at different times than the first images and are captured from different directions. The first likelihood distribution calculation unit 102 calculates a first likelihood distribution representing the distribution of the likelihood of each of the multiple first images acquired by the first image acquisition unit 101, where the likelihood represents the similarity of objects. The tracking feasibility determination unit 103 determines whether an object can be tracked based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. The tracking unit 104 tracks the position of the object in the second image acquired by the second image acquisition unit 105 according to the determination result of the tracking feasibility determination unit 103. Thus, objects inside the patient's body can be tracked with high accuracy based on the patient's perspective images.

[0079] (Second Implementation)

[0080] In the first embodiment, the second image acquisition unit 105 is configured to determine the shooting direction of the second image based on the determination result of the tracking feasibility determination unit 103. In contrast, in the second embodiment, the tracking unit 104 is configured to determine the area, i.e., the tracking area, for tracking the position of the object based on the determination result of the tracking feasibility determination unit 103. This allows for more precise tracking of objects within the patient's body. The second embodiment will be described in detail below.

[0081] Figure 7 This is a block diagram showing the general configuration of the medical image processing apparatus according to the second embodiment. Figure 7 In China, Figure 2 The corresponding parts of each part are marked with the same reference numerals, and the descriptions are omitted.

[0082] The tracking feasibility determination unit 103 determines whether the marker 62 can be tracked based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. Specifically, the tracking feasibility determination unit 103 calculates the tracking difficulty in each shooting direction of the first image 60 based on the first likelihood distribution calculated by the first likelihood distribution calculation unit 102. The method for calculating the tracking difficulty is the same as in the first embodiment, so the description is omitted. Furthermore, if the tracking feasibility determination unit 103 determines that the marker 62 cannot be tracked if the tracking difficulty is higher than a predetermined threshold, the tracking feasibility determination unit 103 then outputs the determination result for the tracking feasibility of the marker 62 in each shooting direction to the tracking unit 104. On the other hand, unlike the first embodiment, the tracking feasibility determination unit 103 does not output the determination result to the second image acquisition unit 105.

[0083] The second image acquisition unit 105 acquires multiple second images, which are fluoroscopic images of patient P generated at different times than the first image and with different shooting directions. For example, the second image is an image captured by the radiation detector 30 during treatment of patient P. Specifically, the second image acquisition unit 105 acquires both the fluoroscopic image captured by radiation detector 30-1 and the fluoroscopic image captured by radiation detector 30-2 as the second image.

[0084] The second likelihood distribution calculation unit 106 calculates a second likelihood distribution representing the likelihood distribution of the second image acquired by the second image acquisition unit 105. Specifically, the second likelihood distribution calculation unit 106 calculates the likelihood distribution of the perspective image (second image) captured by the radiation detector 30-1 and the likelihood distribution of the perspective image (second image) captured by the radiation detector 30-2, as the second likelihood distribution. Here, the second likelihood distribution calculation unit 106 calculates not only the second likelihood distribution for the second image as a whole, but also the second likelihood distribution for the tracking region 61 in the second image. The calculation method for the second likelihood distribution is the same as the calculation method for the first likelihood distribution, so the explanation is omitted.

[0085] Based on the determination result of the tracking feasibility determination unit 103, the tracking unit 104 determines the area for tracking the position of the marker 62, namely the tracking area 61. Figure 3Specifically, for a second image taken in a shooting direction that is determined by the tracking ability determination unit 103 to have a tracking difficulty of marker 62 that is higher than a predetermined threshold, the tracking unit 104 limits the tracking area 61 to the area on the epipolar line.

[0086] For example, suppose the tracking difficulty for the imaging direction of radiation detector 30-1 is higher than a predetermined threshold, while the tracking difficulty for the imaging direction of radiation detector 30-2 is lower than the predetermined threshold. In this case, when determining the position of marker 62 from the second image captured by radiation detector 30-2, the position of marker 62 in the second image captured by radiation detector 30-1 is geometrically defined on the epipolar line. Therefore, for the second image captured by radiation detector 30-1, the tracking unit 104 defines the tracking area 61 as the area on the epipolar line.

[0087] Thus, based on the ease of tracking, the shooting direction of the easily trackable marker 62 is determined, and the marker 62 is tracked from the second image taken from the difficult-to-track shooting direction, with the tracking area 61 defined as the region on the epipolar line. Therefore, the medical image processing apparatus 200 of this embodiment can improve the tracking accuracy of the marker 62.

[0088] Figure 8 This is a flowchart illustrating the operation of the medical image processing apparatus in the stage preceding the treatment of the patient according to the second embodiment. Before the patient P is treated with irradiation using the therapeutic beam B, the medical image processing apparatus 200 performs processing based on this flowchart. Furthermore, Figure 8 Steps S301 to S303 and Figure 5 Steps S101 to S103 are the same, so the explanation is omitted.

[0089] The tracking feasibility determination unit 103 determines whether there is a tracking difficulty level higher than a predetermined threshold among the tracking difficulty levels of each shooting direction calculated in step S303 (S304). If there is no tracking difficulty level higher than the predetermined threshold (S304: No), the tracking unit 104 does not change the tracking area 61 and ends the processing based on this flowchart. On the other hand, if there is a tracking difficulty level higher than the predetermined threshold (S304: Yes), the tracking unit 104 calculates a straight line connecting the position of the marker 62 in the shooting direction with the lowest tracking difficulty to the shooting position (position of the radiation detector 30) in the shooting direction with the lowest tracking difficulty, as the epipolar line (S305). Then, the tracking unit 104 limits the tracking area 61 of the shooting direction with tracking difficulty higher than the predetermined threshold to the area on the epipolar line calculated in step S305 (S306) and ends the processing based on this flowchart.

[0090] Thus, for second images captured in shooting directions where the tracking difficulty is determined to be below a predetermined threshold, the tracking unit 104 does not change the tracking area 61. On the other hand, for second images captured in shooting directions where the tracking difficulty is determined to be above the predetermined threshold, the tracking unit 104 limits the tracking area 61 to the region on the epipolar line calculated based on the position of the marker 62 in the shooting direction with the lowest tracking difficulty. As a result, the medical image processing apparatus 200 of this embodiment can improve the tracking accuracy of the marker 62.

[0091] (Third Implementation)

[0092] In the first and second embodiments, an irradiation control device 50 is provided to control the treatment beam irradiation gate 40. In contrast, in the third embodiment, the irradiation control device 50 is not provided in the treatment system 1, and the radiation therapy device 300 has the function of controlling the treatment beam irradiation gate 40. The third embodiment will be described in detail below.

[0093] Figure 9 This is a block diagram showing the general configuration of the radiation therapy apparatus according to the third embodiment. Figure 9 In China, Figure 2 The parts corresponding to each other are marked with the same reference numerals, and the description is omitted. The radiation therapy device 300 of the third embodiment includes an irradiation control unit 301.

[0094] The tracking unit 104 determines the position of the marker 62 in the second image based on the second likelihood distribution calculated by the second likelihood distribution calculation unit 106. For example, the tracking unit 104 determines the position of the marker 62 as the position where the second likelihood distribution has the maximum value.

[0095] The tracking unit 104 determines the irradiation timing of the treatment beam B based on the determined position of the marker 62. The positional relationship between the marker 62 and the lesion 63 can be determined based on the fluoroscopic image of the patient P obtained during treatment planning. Therefore, when the position of the lesion 63 in the patient P, determined based on the position of the marker 62, is within a predetermined range, the tracking unit 104 outputs an irradiation instruction signal to the irradiation control unit 301 to instruct the irradiation of the treatment beam B.

[0096] The irradiation control unit 301 irradiates the therapeutic beam B from the therapeutic beam irradiation gate 40 according to the irradiation instruction signal output from the tracking unit 104. This allows the therapeutic beam B to be irradiated onto the lesion 63 within the patient P.

[0097] On the other hand, when the tracking unit 104 detects that the location of the lesion 63 in the patient P, determined based on the position of the marker 62, is not within a predetermined range, it does not output an irradiation instruction signal to the irradiation control unit 301. This suppresses the irradiation of the therapeutic beam B to sites other than the lesion 63.

[0098] Figure 10 This is a flowchart illustrating the operation of the radiation therapy apparatus during patient treatment according to the third embodiment. During the treatment phase where patient P is irradiated with the therapeutic beam B, the radiation therapy apparatus 300 repeatedly performs the processes based on this flowchart at predetermined time intervals. Furthermore, Figure 10 Steps S401 to S403 and Figure 6 Steps S201 to S203 are the same, so the explanation is omitted.

[0099] The irradiation control unit 301 irradiates the treatment beam B from the treatment beam irradiation gate 40 based on the position of the marker 62 (object) determined by the tracking unit 104 in step S403. For example, when the position of the lesion 63 in the patient P, determined based on the position of the marker 62, is within a predetermined range, the irradiation control unit 301 irradiates the treatment beam B from the treatment beam irradiation gate 40.

[0100] Thus, in addition to possessing all the functions of the medical image processing device 100 of the first embodiment, the radiotherapy apparatus 300 of the third embodiment also includes an irradiation control unit 301 for controlling the irradiation gate 40 of the treatment beam. Similar to the first embodiment, the radiotherapy apparatus 300 of this embodiment can also improve the tracking accuracy of the marker 62.

[0101] Furthermore, in the above embodiment, the object tracked by the tracking unit 104 is a metal marker placed inside the patient P's body, but it could also be a lesion inside the patient P's body. In this case, an image including the entire lesion or a part of the lesion can be cropped from the first image and used as a template image. If the first image is a DRR image generated from a CT image including treatment planning information, the treatment planning information includes the location of the lesion within the CT image; therefore, an image projected from the lesion can be used as a template image. If the first image is an X-ray fluoroscopy image taken by the radiation detector 30 at the patient P's location, the DRR image corresponds to the patient P's location; therefore, an X-ray fluoroscopy image of the surrounding area can be cropped using the lesion's location and used as a template image.

[0102] Furthermore, the tracking success / failure determination unit 103 can also use a notification unit (not shown) to inform the user of the tracking difficulty in each shooting direction. For example, the tracking success / failure determination unit 103 can notify the user of shooting directions with low tracking difficulty when the user plans the shooting direction. As a result, the user can estimate the success rate of the respiratory synchronization irradiation method and determine whether to limit the tracking of the marker 62 to the second image in only one shooting direction.

[0103] Furthermore, the aforementioned medical image processing apparatus 100, 200, and radiotherapy apparatus 300 can also be implemented, for example, by using a general-purpose computer device as basic hardware. That is, the first image acquisition unit 101, the first likelihood distribution calculation unit 102, the tracking feasibility determination unit 103, the tracking unit 104, the second image acquisition unit 105, the second likelihood distribution calculation unit 106, and the irradiation control unit 301 can be implemented by executing a program on the processor of the aforementioned computer device. In this case, the medical image processing apparatus 100, 200, and radiotherapy apparatus 300 can be implemented by pre-installing the aforementioned program on the computer device, or by storing the program on a storage medium such as a CD-ROM or distributing it via a network, and then appropriately installing the program on the computer device. Furthermore, B and C can be appropriately implemented using memory, hard disks, or storage media such as CD-R, CD-RW, DVD-RAM, DVD-R, etc., built into or external to the aforementioned computer device.

[0104] This concludes the description of several embodiments of the present invention. However, these embodiments are merely illustrative and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, and are also included within the scope of the invention as described in the claims and its equivalents.

[0105] Explanation of reference numerals in the attached figures

[0106] 1: Treatment System

[0107] 100: Medical image processing device

[0108] 101: First Image Acquisition Unit

[0109] 102: First Likelihood Distribution Calculation Section

[0110] 103: Tracking Feasibility Determination Department

[0111] 104: Tracking Department

[0112] 105: Second Image Acquisition Unit

[0113] 106: Second Likelihood Distribution Calculation Section

[0114] 200: Medical image processing device

[0115] 300: Radiation therapy device

[0116] 301: Irradiation Control Department

Claims

1. A medical image processing device for tracking the position of objects within a patient's body, wherein, The medical image processing device includes: The first image acquisition unit acquires multiple first images, which are fluoroscopic images of the patient and are captured from different directions. The first likelihood distribution calculation unit calculates, for each of the plurality of first images, the similarity at each pixel position of the object as a likelihood representing the similarity of the object, and calculates a first likelihood distribution representing the distribution of the likelihood in the first image; The tracking feasibility determination unit determines whether the object can be tracked for each of the multiple first images based on the first likelihood distribution; The second image acquisition unit, when determining that the object is traceable for each of the plurality of first images, acquires a plurality of second images as perspective images of the patient generated at a different time than the plurality of first images and with the same shooting direction as the plurality of first images; when determining that the object in one or more of the plurality of first images is traceable and the object in one or more of the other first images is untraceable, acquires a plurality of second images as perspective images of the patient generated at a different time than the one or more of the first images and with the same shooting direction as the one or more of the first images, but does not acquire one or more other second images as perspective images of the patient generated at a different time than the other one or more of the first images and with the same shooting direction as the other one or more of the first images. The second likelihood distribution calculation unit calculates, for each of the plurality of second images, the similarity at each pixel position of the object as a likelihood representing the similarity of the object, and calculates a second likelihood distribution representing the distribution of the likelihood in the second image; as well as The tracking unit determines the position of the object in the second image based on the second likelihood distribution, tracks the position of the object, and sends a signal to control the irradiation of radiation according to the position of the object.

2. The medical image processing device according to claim 1, wherein, It also has a notification department. The tracking feasibility determination unit uses the notification unit to prompt the user with the tracking difficulty in each shooting direction.

3. The medical image processing device according to claim 1, wherein, Before treating the patient, the second image acquisition unit determines the shooting direction of the second image used in tracking the position of the object by the tracking unit.

4. The medical image processing device according to claim 1, wherein, The tracking unit determines the area, i.e. the tracking area, for tracking the position of the object based on the tracking ability determination result of the tracking ability determination unit.

5. The medical image processing device according to claim 4, wherein, The tracking feasibility determination unit calculates a tracking difficulty level, which represents the difficulty of tracking the object in the plurality of second images. For the second image whose tracking difficulty, calculated by the tracking feasibility determination unit, is higher than a predetermined threshold, the tracking unit defines the tracking region as the region on the epipolar line.

6. The medical image processing apparatus according to claim 1, wherein, The object is a metal marker.

7. The medical image processing apparatus according to any one of claims 1 to 6, wherein, The first image is a perspective image generated based on a three-dimensional image of the patient taken during the treatment plan, or a perspective image taken from the same shooting direction as the second image immediately before treatment. The second image is a perspective image taken during the treatment of the patient.

8. A medical image processing method for tracking the position of objects within a patient's body, wherein, The medical image processing method includes the following steps: The first image acquisition process involves acquiring multiple first images, which are fluoroscopic images of the patient and are captured from different directions. The first likelihood distribution calculation step involves, for each of the multiple first images, calculating the similarity at each pixel position of the object as the likelihood of the object's similarity, and calculating a first likelihood distribution representing the distribution of the likelihood in the first image. The tracking availability determination process involves determining whether the object can be tracked for each of the multiple first images based on the first likelihood distribution. The second image acquisition step acquires multiple second images. The second image acquisition step includes the following steps: if it is determined that the object in each of the multiple first images is traceable, acquiring multiple second images as perspective images of the patient generated at a different time than the multiple first images and with the same shooting direction as the multiple first images; and if it is determined that the object in one or more of the multiple first images is traceable and the object in one or more other first images is untraceable, acquiring multiple second images as perspective images of the patient generated at a different time than the one or more first images and with the same shooting direction as the one or more first images, without acquiring one or more other second images as perspective images of the patient generated at a different time than the other one or more first images and with the same shooting direction as the other one or more first images. The second likelihood distribution calculation step involves, for each of the multiple second images, calculating the similarity at each pixel position of the object as the likelihood of the object's similarity, and calculating a second likelihood distribution representing the distribution of the likelihood in the second image; The tracking process involves determining the position of the object in the second image based on the second likelihood distribution, and tracking the position of the object; and The process of sending a signal to control the irradiation of the object based on its location.

9. The medical image processing method according to claim 8, wherein, In the tracking feasibility determination process, the notification unit prompts the user with the tracking difficulty in each shooting direction.

10. A storage medium storing a medical image processing program for tracking the location of objects within a patient's body, wherein, The medical image processing program causes the computer to perform the following procedures: The first image acquisition process involves acquiring multiple first images, which are fluoroscopic images of the patient and are captured from different directions. The first likelihood distribution calculation step involves, for each of the multiple first images, calculating the similarity at each pixel position of the object as the likelihood of the object's similarity, and calculating a first likelihood distribution representing the distribution of the likelihood in the first image. The tracking availability determination process involves determining whether the object can be tracked for each of the multiple first images based on the first likelihood distribution. The second image acquisition step acquires multiple second images. The second image acquisition step includes the following steps: if it is determined that the object in each of the multiple first images is traceable, acquiring multiple second images as perspective images of the patient generated at a different time than the multiple first images and with the same shooting direction as the multiple first images; and if it is determined that the object in one or more of the multiple first images is traceable and the object in one or more other first images is untraceable, acquiring multiple second images as perspective images of the patient generated at a different time than the one or more first images and with the same shooting direction as the one or more first images, without acquiring one or more other second images as perspective images of the patient generated at a different time than the other one or more first images and with the same shooting direction as the other one or more first images. The second likelihood distribution calculation step involves, for each of the multiple second images, calculating the similarity at each pixel position of the object as the likelihood of the object's similarity, and calculating a second likelihood distribution representing the distribution of the likelihood in the second image; The tracking process involves determining the position of the object in the second image based on the second likelihood distribution, and tracking the position of the object; and Based on the location of the object, a signal is sent to control the irradiation of the object with radiation.

11. The storage medium according to claim 10, wherein, The medical image processing program causes the computer to perform a step in the tracking feasibility determination process, in which a notification unit prompts the user with the tracking difficulty in each shooting direction.

12. A radiation therapy apparatus that irradiates the patient with radiation while tracking the position of an object within the patient's body, wherein, The radiation therapy device includes: The first image acquisition unit acquires multiple first images, which are fluoroscopic images of the patient and are captured from different directions. The first likelihood distribution calculation unit calculates, for each of the plurality of first images, the similarity at each pixel position of the object as a likelihood representing the similarity of the object, and calculates a first likelihood distribution representing the distribution of the likelihood in the first image; The tracking feasibility determination unit determines whether the object can be tracked for each of the multiple first images based on the first likelihood distribution; The second image acquisition unit acquires multiple second images when it determines that the object in each of the multiple first images is traceable. The multiple second images are perspective images of the patient generated at a different time than the multiple first images and the shooting direction is the same as the multiple first images. When it determines that the object in one or more of the multiple first images is traceable and the object in one or more other first images is untraceable, it acquires multiple second images, which are perspective images of the patient generated at a different time than the one or more first images and the shooting direction is the same as the one or more first images, but does not acquire one or more other second images, which are perspective images of the patient generated at a different time than the other one or more first images and the shooting direction is the same as the other one or more first images. The second likelihood distribution calculation unit calculates, for each of the plurality of second images, the similarity at each pixel position of the object as a likelihood representing the similarity of the object, and calculates a second likelihood distribution representing the distribution of the likelihood in the second image; The tracking unit determines the position of the object in the second image based on the second likelihood distribution and tracks the position of the object; as well as The irradiation control unit controls the irradiation of the object based on the position of the object tracked by the tracking unit.

Images