System, radiation treatment system, method executed by system, and program

Abstract

To achieve estimation of displacement or deformation of a three-dimensional volume that can be applied to segments of a wide range of types contained in the three-dimensional volume.SOLUTION: A system 102 displaces or deforms a segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment so as to correspond to estimation contents of the displacement or deformation of a three-dimensional volume 191, and generates a segment-corresponding two-dimensional simulation image (DR_sim[*]). The system 102 composites each DR_sim[*], and generates a two-dimensional simulation image (DR_sim). The same modality similarity calculator 134 calculates a similarity between the DR_sim and a two-dimensional real time image (DR_RT) generated when the three-dimensional volume 191 is X-ray imaged, which have the same modality. The system 102 determines the propriety of the estimation contents on the basis of the similarity between the DR_sim and the DR_RT.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technique for estimating (tracking) the motion (displacement or deformation) of a three-dimensional volume (e.g., a body), or to control of a therapeutic beam for radiation therapy performed on a target (e.g., a tumor) included in the three-dimensional volume based on an estimated content of the motion (displacement or deformation) of the three-dimensional volume. [Background technology]

[0002] It is desirable to estimate the motion (displacement or deformation) of a three-dimensional volume (e.g., a body). For example, when performing radiation therapy on a target (e.g., a tumor) contained in a three-dimensional volume, it is desirable to accurately estimate (track) the motion (displacement or deformation) of each segment (e.g., a site, organ, tissue, target (e.g., tumor), etc.) contained in the three-dimensional volume. When a treatment beam is irradiated onto a target contained in a three-dimensional volume, the more accurately the target's motion (displacement or deformation) is estimated, the more appropriately the treatment beam is irradiated onto the target (the appropriate radiation dose is irradiated) and the less the impact of the treatment beam on tissues other than the target is reduced. In other words, if the accuracy of the target's motion (displacement or deformation) is low, the accuracy of the treatment beam irradiation may decrease, resulting in an inappropriate radiation dose being irradiated onto the target and potentially increasing the impact of the treatment beam on tissues other than the target. Furthermore, if the movement (displacement or deformation) of a three-dimensional volume can be estimated accurately, it is expected that the content of the treatment will be appropriate not only for radiation therapy but also for general treatments performed on three-dimensional volumes. Here, there can be various causes of the movement (displacement or deformation) of the three-dimensional volume. For example, the movement of the respiratory system in the three-dimensional volume can cause the displacement or deformation (change in position or shape) of each of the segments included in the three-dimensional volume. In addition to the movement of the respiratory system, other anatomical changes such as swallowing, peristalsis, and bladder filling can also cause the displacement or deformation (change in position or shape) of each of the segments.

[0003] Traditionally, internal fiducial markers have been used to estimate (track) the motion of a 3D volume (e.g., a body). For example, multiple internal fiducial markers may be implanted near a target (e.g., a tumor) contained in the 3D volume. This provides comprehensive motion information related to the changing shape of the target.

[0004] Furthermore, techniques are being developed for estimating (tracking) the movement of a three-dimensional volume (for example, a body) without using fiducial markers as described above. For example, Patent Documents 1 and 2 disclose a technique for comparing a digitally reconstructed radiography (DRR), which is a CT image (CT volume) showing a three-dimensional volume that is deformed and projected onto a two-dimensional plane, with a real-time X-ray image obtained by X-ray imaging of the three-dimensional volume. If the comparison results are favorable (for example, the degree of similarity is high), it means that the deformed CT image (CT volume) is valid as an estimate (tracking) of the movement of the three-dimensional volume. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-142146 [Patent Document 2] Special Publication No. 2010-500151 Summary of the Invention [Problem to be solved by the invention]

[0006] First, the technique of estimating (tracking) the motion of a three-dimensional volume (e.g., a body) using internal fiducial markers is invasive to the three-dimensional volume, and in some cases, the invasiveness of the fiducial markers may cause complications to the three-dimensional volume. Furthermore, while techniques that use internal fiducial markers to estimate (track) the movement of 3D volumes can sometimes provide comprehensive information about the movement of a target (e.g., a tumor) as it changes shape, they often fall short of providing a complete picture of the target's shape deformation. In other words, there is room for improvement in the accuracy of internal fiducial marker techniques in estimating (tracking) the movement (displacement or deformation) of 3D volumes.

[0007] Next, prior art techniques for estimating (tracking) the movement of a three-dimensional volume (e.g., a body) without using internal fiducial markers tend to be limited in the types of segments that can be appropriately estimated (tracked) among the segments contained in the three-dimensional volume (e.g., parts, organs, tissues, targets (e.g., tumors), etc.). Specifically, in the prior art disclosed in Patent Documents 1 and 2, a comparison (matching, which may also be called template matching) is performed between a digitally reconstructed X-ray image (DRR) derived from a CT image (CT volume) and a real-time X-ray image (usually a digital X-ray image (Digital Radiography (DR))) generated by X-ray photography. Here, the digitally reconstructed X-ray image (DRR) and the digital X-ray image (DR) have different modalities (the imaging device used to obtain the image and the physical background that is the premise of the processing to obtain the image). When comparing (matching) images of different modalities, appropriate comparison (matching) tends to be possible when the contrast (difference in pixel values between regions of different properties in the image) in the segment is large. In fact, since the contrast of tumors and other features in the lungs is relatively large (due to the large amount of air in the lungs), it is possible to compare (match) the digitally reconstructed X-ray image (DRR) with the digital X-ray image (DR) in Patent Documents 1 and 2 in real time. On the other hand, since the contrast of tumors and other features in the pancreas and other digestive organs is relatively small, it is difficult to appropriately compare (match) the digitally reconstructed X-ray image (DRR) with the digital X-ray image (DR) in Patent Documents 1 and 2 in real time.

[0008] Furthermore, each of the segments (e.g., parts, organs, tissues, targets (e.g., tumors), etc.) included in a three-dimensional volume (e.g., a body) may move (displace or deform) differently in three-dimensional space. In other words, a two-dimensional X-ray image generated by X-raying a three-dimensional volume includes a superposition of multiple image layers that have different contrasts, move at different speeds, and deform differently. If such multiple image layers are handled undifferentiated, it can be difficult to properly estimate (track) the movement (displacement or deformation) of the segments included in the three-dimensional volume.

[0009] In light of the above, one of the objectives of the present disclosure may be to realize estimation (tracking) of the movement (displacement or deformation) of a three-dimensional volume that is non-invasive to the three-dimensional volume, applicable to a wide variety of segments contained in the three-dimensional volume, and capable of accommodating the inclusion of multiple segments in the three-dimensional volume. [Means for solving the problem]

[0010] In order to achieve at least one of the above objects, the present disclosure may have the following features, for example. One aspect of the present disclosure is a system including a two-dimensional image acquirer, a two-dimensional displacement transformer, a two-dimensional simulation image combiner, a same-modality similarity calculator, and a volume estimation validity determiner. The two-dimensional image acquirer acquires two-dimensional real-time images (DR_RT) generated by radiographing a three-dimensional volume, where the two-dimensional real-time images (DR_RT) have a predetermined modality. The two-dimensional displacement transformer displaces or transforms each of the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment of the three-dimensional volume to correspond to the estimated displacement or deformation of the three-dimensional volume, thereby generating each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment of the three-dimensional volume, where the segment-corresponding two-dimensional reference images (DR_ref[*]) and the segment-corresponding two-dimensional simulation images (DR_sim[*]) have the predetermined modality. The two-dimensional simulation image synthesizer synthesizes each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) to generate a two-dimensional simulation image (DR_sim), where the two-dimensional simulation image (DR_sim) has the predetermined modality. The same modality similarity calculator calculates the similarity between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim). The volume estimation validity determiner determines the validity of the volume estimation content based on the similarity. [Effects of the Invention]

[0011] The present disclosure calculates similarity between a two-dimensional real-time image (DR_RT) and a two-dimensional simulation image (DR_sim) of the same modality to determine the validity of estimated displacement or deformation of a three-dimensional volume. Since similarity calculation (image comparison) is performed between images of the same modality, a wide variety of segments included in the three-dimensional volume can be handled in real time.

[0012] Furthermore, the present disclosure displaces or deforms each of the segment-corresponding reference images (DR_ref[*]) for each segment to generate each of the segment-corresponding two-dimensional simulation images (DR_sim[*]). Then, the present disclosure synthesizes each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) to generate a two-dimensional simulation image (DR_sim). In this way, the present disclosure can individually handle the displacement or deformation of each of the segments included in the three-dimensional volume.

[0013] As a result, the present disclosure makes it possible to realize estimation (tracking) of the movement (displacement or deformation) of a three-dimensional volume in a non-invasive manner, capable of accommodating a wide variety of segments contained in the three-dimensional volume, and capable of accommodating the inclusion of multiple segments in the three-dimensional volume.

[0014] Methods and programs that achieve the same processing as the above system can also achieve the same effects as the above system. In the form of a program, costs can often be reduced. Programs also make it easier to make design changes to the processing. Other features that the present disclosure may have and the effects corresponding to those features will be disclosed in this specification, claims, or drawings. [Brief explanation of the drawings]

[0015] [Figure 1] 1 shows a basic functional configuration of a system (control system) according to the present disclosure. [Figure 2] The overall configuration of the radiation therapy system is shown. [Figure 3] Details of the radiation therapy room system (part 1) are shown. [Figure 4] Details of the radiation therapy room system (part 2) are shown. [Figure 5] The computer architecture of the system (control system) is shown. [Figure 6] This shows the image information group handled before radiation therapy. [Figure 7] The functional configuration that operates before radiation therapy is shown. [Figure 8] The flowchart for generating a model is shown below. [Figure 9] A flowchart before radiation therapy is shown. [Figure 10] This shows the group of image information handled during radiation therapy. [Figure 11] The functional configuration that operates during radiation therapy is shown. [Figure 12] This shows a flowchart during radiation therapy. DETAILED DESCRIPTION OF THE INVENTION

[0016] Embodiments of the present disclosure will be described in detail below with reference to the drawings. Note that the embodiments described below do not limit the disclosure according to the claims, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solutions of the present disclosure. The following description and drawings are examples for explaining the present disclosure, and appropriate omissions and simplifications have been made for clarity of explanation. The present disclosure can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present disclosure is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. Each of the systems, devices, or functional units (units) disclosed herein may be integrated into a single piece of hardware, or may be divided into multiple parts that work together to perform their functions. Several systems, devices, or functional units (units) may be integrated into a single piece of hardware. Each of the systems, devices, or functional units (units) may be realized by causing a computer to execute software (programs) (as in FIG. 5). Some of the functions of the systems, devices, or functional units (units) may be realized by hardware (e.g., hardwired logic or field programmable gate arrays (FPGAs)), and the remaining functions may be realized by executing software (programs). All of the functions of each of the systems, devices, or functional units (units) may be realized by hardware. Some or all of the steps shown in the flowcharts, etc. described in this disclosure may be realized by hardware. One or more systems, devices, or functional units (units) of the present disclosure may be realized using one or more hardware resources. For this purpose, each of the systems, devices, or functional units (units) of the present disclosure may be realized virtually. For example, a virtual computer or container technique may be used. The program of the present disclosure may be included in the general concept of software that encompasses software that cooperates with hardware resources to construct a specific information processing system (system, control system) or its operating method according to the intended use. In other words, the program of the present disclosure is not limited to a specific type or form of program. Furthermore, the program may be initially recorded in a compressed format. The same reference numbers are used in multiple drawings. In the drawings showing flowcharts, rectangular boxes indicate processing steps, and hexagonal boxes indicate conditional branching steps. In the drawings showing flowcharts, "step" is abbreviated as "S." Also, in the drawings, database is abbreviated as "DB."

[0017] 1. Functional configuration and system configuration of an embodiment of the present disclosure (FIG. 1) Fig. 1 shows the basic functional configuration of a system (control system) according to an embodiment of the present disclosure. Note that not all of the functional configurations shown in Fig. 1 are essential. Furthermore, the presence of functional configurations other than those shown in Fig. 1 is not prohibited.

[0018] The system 102 (control system) is for estimating (tracking) in real time the movement (displacement or deformation) occurring inside a three-dimensional volume 191 (for example, a body). The three-dimensional volume 191 has the following characteristics. (1) Movement (displacement or deformation) occurs in real time inside the three-dimensional volume 191. If the three-dimensional volume 191 is a body, movement (displacement or deformation) may occur at each location inside the three-dimensional volume 191 in accordance with movement of at least the respiratory organs. (2) The three-dimensional volume 191 includes a plurality of segments, which may be, for example, a region, an organ, a tissue, a target (e.g., a tumor), etc. (3) The segments included in the three-dimensional volume 191 may have different modes of movement (displacement or deformation).

[0019] The system 102 (control system) includes a two-dimensional image acquirer 121 , a two-dimensional displacement transformer 132 , a two-dimensional simulation image synthesizer 133 , a same-modality similarity calculator 134 , and a volume estimation validity determiner 136 . The two-dimensional image acquirer 121 acquires a two-dimensional real-time image (DR_RT) generated by X-ray imaging of the three-dimensional volume 191. For example, as shown in FIG. 1, the two-dimensional real-time image (DR_RT) may be generated by emitting X-rays from an X-ray source 194a, transmitting the emitted X-rays through the three-dimensional volume 191, and being detected by an X-ray image detector 195a. Similarly, the two-dimensional real-time image (DR_RT) may be generated by emitting X-rays from an X-ray source 194b, transmitting the emitted X-rays through the three-dimensional volume 191, and being detected by an X-ray image detector 195b. The two-dimensional real-time image (DR_RT) has a predetermined modality because it is generated based on X-ray imaging. Note that the two-dimensional real-time image (DR_RT) has a form in which image layers representing each of the segments (e.g., a site, an organ, a tissue, a target (e.g., a tumor), etc.) included in the three-dimensional volume 191 are overlapped.

[0020] Prior to real-time estimation (tracking) of displacement or deformation occurring within the three-dimensional volume 191, the system 102 (control system) has access to a segment-corresponding two-dimensional reference image (DR_ref[*], where * in [*] below is a segment identifier (e.g., 1, 2, 3, etc.)) for each segment included in the three-dimensional volume 191. The segment-corresponding two-dimensional reference image (DR_ref[*]) has the same modality (the above-mentioned predetermined modality) as the two-dimensional real-time image (DR_RT). In other words, the segment-corresponding two-dimensional reference image (DR_ref[*]) is derived from an image generated by X-ray photography. The system 102 (control system) uses some method (an example of the method will be described later) to estimate the displacement or deformation occurring inside the three-dimensional volume 191. The content of the estimation is indicated as "estimated content" in FIG. The two-dimensional displacement transformer 132 displaces or transforms each of the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment to generate each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment. The two-dimensional displacement transformer 132 displaces or transforms each of the segment-corresponding two-dimensional reference images (DR_ref[*]) so as to correspond to the above-mentioned "estimated content" regarding the displacement or deformation occurring inside the three-dimensional volume 191. In this way, the two-dimensional displacement transformer 132 individually displaces or transforms each of the segment-corresponding two-dimensional reference images (DR_ref[*]). Therefore, even if the manner of displacement or deformation differs for each segment of the three-dimensional volume 191, the system 102 (control system) can appropriately estimate (track) the displacement or deformation of each of the segments included in the three-dimensional volume 191. In addition, the two-dimensional displacement transformer 132 generates each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) using each of the segment-corresponding two-dimensional reference images (DR_ref[*]), so that each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) has the same modality (the above-mentioned specified modality) as the two-dimensional real-time image (DR_RT). The two-dimensional simulation image synthesizer 133 synthesizes each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment to generate a two-dimensional simulation image (DR_sim). As described above, each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) corresponds to the aforementioned "estimated content" regarding the displacement or deformation occurring inside the three-dimensional volume 191. The two-dimensional simulation image (DR_sim) is derived from each of the segment-corresponding two-dimensional simulation images (DR_sim[*]). In other words, it can be assumed that the two-dimensional image generated when the three-dimensional volume 191 is subjected to X-ray imaging in a state in which the interior of the three-dimensional volume 191 is displaced or deformed in accordance with the aforementioned "estimated content" regarding the displacement or deformation occurring inside the three-dimensional volume 191 is the two-dimensional simulation image (DR_sim). This two-dimensional simulation image (DR_sim) is derived from the segment-corresponding two-dimensional simulation image (DR_sim[*]), and the segment-corresponding two-dimensional simulation image (DR_sim[*]) has the same modality (the above-mentioned predetermined modality) as the two-dimensional real-time image (DR_RT). In other words, the two-dimensional simulation image (DR_sim) has the same modality (the above-mentioned predetermined modality) as the two-dimensional real-time image (DR_RT).

[0021] The same-modality similarity calculator 134 calculates the similarity (same-modality image similarity) between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim). As described above, the two-dimensional simulation image (DR_sim) has the same modality (the predetermined modality described above) as the two-dimensional real-time image (DR_RT). In other words, the same-modality similarity calculator 134 calculates the similarity (same-modality image similarity) between two-dimensional images having the same modality (here, the modality of the two-dimensional image generated by X-ray imaging of the three-dimensional volume 191). Therefore, the same-modality similarity calculator 134 can appropriately calculate the similarity between images in real time for a wide variety of segments, regardless of differences in the characteristics (e.g., the level of contrast between regions with different properties) of each segment (e.g., part, organ, tissue, target (e.g., tumor), etc.) included in the three-dimensional volume 191. The volume estimation validity determiner 136 determines the validity of the aforementioned "estimated content" regarding the displacement or deformation occurring inside the three-dimensional volume 191 based on the similarity (similarity between images of the same modality). If the similarity (similarity between images of the same modality) between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim) generated based on the "estimated content" is sufficiently high, it means that the "estimated content" is valid as an estimate (tracking) of the displacement or deformation occurring inside the three-dimensional volume 191. If the similarity (similarity between images of the same modality) is not sufficiently high, the system 102 (control system) may redo the estimation (tracking) of the displacement or deformation occurring inside the three-dimensional volume 191. In other words, the system 102 (control system) may redo the generation of the two-dimensional real-time image (DR_sim) and the calculation of the similarity (similarity between images of the same modality) based on a new "estimated content" of the displacement or deformation occurring inside the three-dimensional volume 191. Alternatively, the system 102 (control system) may prepare multiple "estimated content" and select from the multiple "estimated content" an "estimated content" that has a high similarity (similarity between images of the same modality).

[0022] The system 102 (control system) in the embodiment of the present disclosure has the above-described functional configuration, and therefore can have the effects shown in the above-described [Effects of the Invention].

[0023] 2. Overall configuration of a radiation therapy system including the system (control system) of the present disclosure (Fig. 2) Figure 2 shows the overall configuration 200 of a radiation therapy system to which the system 102 (control system) of the present disclosure can be applied. Note that not all of the functional configurations shown in Figure 2 are essential. Furthermore, the presence of functional configurations other than those shown in Figure 2 is not prohibited.

[0024] The radiation therapy system 101 shown in FIG. 2 is for irradiating a target (e.g., a tumor) included in a three-dimensional volume (e.g., a body) 191, such as that shown in FIG. 1, with a therapeutic beam. For example, the radiation therapy system 101 may be a particle beam therapy (PBT) system that uses a charged particle beam (e.g., a proton beam or a heavy particle beam) as a therapeutic beam. Within the three-dimensional volume 191, each of the segments (e.g., a site, an organ, a tissue, a target (e.g., a tumor), etc.) exhibits real-time movement (displacement or deformation). Therefore, in order to properly irradiate the target with the therapeutic beam, it is useful to improve the accuracy of estimating (tracking) the real-time movement (displacement or deformation) of each of the segments included in the three-dimensional volume 191. A system 102 (control system) according to an embodiment of the present disclosure can be useful for estimating (tracking) the real-time movement (displacement or deformation) of each of the segments included in the three-dimensional volume 191. Furthermore, the system 102 (control system) may output control information indicating control details for irradiating the treatment beam based on the results (estimated details) of the above-mentioned estimation (tracking) of the displacement or deformation.

[0025] 2, the radiation therapy system 101 may include an accelerator 107, a beam transport line 108, a radiation therapy room system 109, and a system 102 (control system) according to an embodiment of the present disclosure. The system 102 (control system) may provide, to the accelerator 107, the beam transport line 108, and the radiation therapy room system 109, control information for controlling various devices included in the accelerator 107, the beam transport line 108, and the radiation therapy room system 109. The system 102 (control system) may also receive information (including measurement information) obtained by the various devices included in the accelerator 107, the beam transport line 108, and the radiation therapy room system 109. Note that in FIG. 2, the accelerator 107, the beam transport line 108, the radiation therapy room system 109, and the system 102 (control system) are depicted as if they exist in separate physical spaces. However, there is some flexibility in the location of the accelerator 107, the beam transport system 108, the radiation treatment room system 109, and the system 102 (control system). For example, the system 102 (control system) may exist within the physical space in which the radiation treatment room system 109 exists. The accelerator 107 gives a desired energy to a therapeutic beam (for example, a charged particle beam), and then emits the therapeutic beam to a beam transport line . The beam transport system 108 transports the therapeutic beam incident (extracted) from the accelerator 107 to a gantry 196 (shown in FIGS. 3 and 4) in a radiation therapy room system 109. Note that if the physical distance between the accelerator 107 and a gantry 196 shown in FIGS. 3 and 4 (to be described later) is short and there is no need to provide the beam transport system 108, the radiation therapy system 101 may not have the beam transport system 108. The radiation therapy room system 109 includes various devices for irradiating a therapeutic beam onto a target (e.g., a tumor) included in a three-dimensional volume (e.g., a body) 191. The radiation therapy room system 109 also includes various devices for collecting measurement information from the three-dimensional volume 191. One example of the measurement information collected from the three-dimensional volume 191 may be a two-dimensional image (X-ray image) generated by X-ray imaging of the three-dimensional volume 191 using a pair of the X-ray source 194a and the X-ray image detector 195a or a pair of the X-ray source 194b and the X-ray image detector 195b shown in FIG. 1. Details of the radiation therapy room system 109 will be described later using FIGS. 3 and 4.

[0026] A system 102 (control system) according to an embodiment of the present disclosure may include various functional units (units) as shown in FIG. 2. A functional unit (unit) may be realized by a computer executing a program corresponding to the functional unit (unit). Alternatively, some of the functional units (units) may be realized by hardware (for example, by hardwired logic or by a field programmable gate array (FPGA)). Furthermore, some of the functions of a functional unit (unit) may be realized by a program, and other functions may be realized by hardware. Each (part or all of the functions of) a functional unit (unit) realized by a program may be realized on the system 102 (control system) only when processing is performed by the functional unit (unit). When system 102 (control system) is used for radiation therapy system 101, as shown in Figure 2, the functional units (units) that can be realized in system 102 (control system) can be classified into (1) functional units (units) that operate both before and during radiation therapy, (2) functional units (units) that operate mainly before radiation therapy, and (3) functional units (units) that operate mainly during radiation therapy. (1) As functional units (units) that operate both before and during radiation therapy, a two-dimensional image acquirer 121, a displacement / deformation setter 122, a three-dimensional displacement / deformation unit 123 (using model 129), a reconstructed image generator 124, and a proxy tissue measurement acquirer 125 may be implemented in system 102 (control system). (2) As functional units (devices) that mainly operate before radiation therapy, a model generator 141, a three-dimensional image acquirer 151, a three-dimensional segment generator 152, a different modality similarity calculator 153, a different modality difference calculator 154, a volume presetting validity determiner 155, and a segment-corresponding two-dimensional reference image generator 156 may be realized in the system 102 (control system). (3) As functional units (devices) that mainly operate during radiation therapy, a two-dimensional deformation vector field generator 131, a two-dimensional displacement transformer 132, a two-dimensional simulation image synthesizer 133, a same-modality similarity calculator 134, a same-modality difference calculator 135, a volume estimation validity determiner 136, and a treatment beam irradiation controller 137 may be realized in the system 102 (control system). The functional configuration realized by the functional units (units) realized in the system 102 (control system) and the processes realized by these functional units (units) will be described later with reference to FIGS.

[0027] 3. Radiation therapy room system included in the radiation therapy system (Figures 3 and 4) FIG. 3 shows details 300 of the radiation therapy room system 109. FIG. 4 shows details 400 of the radiation therapy room system 109. The details 300 in FIG. 3 show the radiation therapy room as viewed from the side of the gantry rotation axis 197, while the details 400 in FIG. 4 show the three-dimensional volume (e.g., a body) 191 in the radiation therapy room as viewed from the end of the gantry rotation axis 197. Note that not all of the devices and functional configurations shown in FIG. 3 or FIG. 4 are required. Furthermore, the presence of functional configurations other than the devices and functional configurations shown in FIG. 3 or FIG. 4 is not prohibited.

[0028] The radiation treatment room system 109 may include a treatment couch 193, a gantry 196, an X-ray source 194a, an X-ray source 194b, an X-ray image detector 195a, and an X-ray image detector 195b. The radiation treatment room system 109 may also include a treatment controller (not shown in FIGS. 3 and 4) that controls various devices (including the treatment couch 193, the gantry 196, the X-ray source 194a, the X-ray source 194b, the X-ray image detector 195a, and the X-ray image detector 195b) included in the radiation treatment room system 109. When the treatment controller is present, the treatment controller receives control information from the system 102 (control system) and controls various devices in the radiation treatment room system 109 based on the control information. In the radiation therapy room system 109, a three-dimensional volume 191 (e.g., the body of a patient receiving radiation therapy) is placed on a treatment couch 193. For example, the patient lies on the treatment couch 193. The three-dimensional volume 191 contains a target (e.g., a tumor) 192 to be irradiated with a treatment beam (e.g., a charged particle beam) 198.

[0029] A three-dimensional volume (here, a patient's body) 191 on a treatment couch 193 may be subjected to X-ray imaging. For this X-ray imaging, an X-ray source 194a emits X-rays, which pass through the three-dimensional volume 191 (a certain range (usually including a target (e.g., a tumor) 192)), and the transmitted X-rays are detected by an X-ray image detector 195a. Similarly, an X-ray source 194b emits X-rays, which pass through the three-dimensional volume 191 (a certain range (usually including a target (e.g., a tumor) 192)), and the transmitted X-rays are detected by an X-ray image detector 195b. In FIGS. 3 and 4 , X-ray imaging is performed on the three-dimensional volume 191 from two directions. When X-ray imaging is performed from the two directions, it is expected that the state of each coordinate in the three-dimensional space inside the three-dimensional volume 191 can be grasped to some extent. In the radiation therapy room system 109, the number of directions in which X-ray imaging can be performed on the three-dimensional volume 191 (the number of combinations of X-ray sources and X-ray image detectors) is not limited to 2. The number of directions may be 3 or more, or may be 1. Both before and during radiation therapy, a two-dimensional image (X-ray image) is generated by X-ray imaging of the three-dimensional volume 191 in the radiation therapy room system 109. Here, the X-ray imaging performed before radiation therapy and the X-ray imaging performed during radiation therapy may have different purposes for using the two-dimensional image (X-ray image) generated by X-ray imaging. Before radiation therapy, a separate three-dimensional imaging is performed on the three-dimensional volume (here, the patient's body) 191 (using another device in the radiation therapy room system 109 or another system for three-dimensional imaging). The three-dimensional imaging is, for example, computed tomography (CT) or magnetic resonance imaging (MRI) based on tomography. The three-dimensional imaging provides image information with a modality different from that provided by simpler X-ray imaging. (In this specification, X-ray photography (at least X-ray photography that is not based on tomography, for example, taking two-dimensional images based on projection (digital X-ray images (Digital Radiography (DR)))) that provides a modality different from modalities based on three-dimensional photography is simply referred to as "X-ray photography.") This allows three-dimensional images (for example, images by CT or MRI) to be acquired. Then, digital reconstructed X-ray images (Digital Reconstructed Radiography (DRR)) that are images digitally reconstructed as two-dimensional transmission images based on three-dimensional images (for example, images by CT or MRI) obtained before radiation therapy, and two-dimensional images (Digital X-ray images (Digital X-ray images)) generated by the radiation therapy room system 109 before radiation therapy are combined. An image matching process called registration is performed between the two-dimensional images (digital X-ray images (DR)) and the three-dimensional volume 191. If the registration is performed before radiation therapy, it becomes possible to displace or deform the two-dimensional image for each segment (segment-corresponding two-dimensional reference image (DR_ref[*])) associated with the two-dimensional image (digital X-ray image (DR)) that was the subject of registration in association with the "estimated content" of the internal movement (displacement or deformation) of the three-dimensional volume 191 during subsequent radiation therapy. On the other hand, during radiation therapy, 3D imaging (e.g., CT or MRI) may not be performed in real time. One reason for this is that 3D imaging takes a relatively long time. Another reason is that some types of 3D imaging (e.g., CT imaging) deliver a relatively large amount of radiation to the 3D volume 191 compared to X-ray imaging, which is something we would like to avoid if possible. In other words, the main direct clue to the real-time internal movement (displacement or deformation) of the 3D volume 191 during radiation therapy is a 2D image (digital X-ray image (DR)) generated by X-ray imaging of the 3D volume 191 in real time during radiation therapy. This real-time generated 2D image (digital X-ray image (DR)) is compared with a 2D image (digital X-ray image (DR)) generated after applying a displacement or deformation (associated with the "estimated content" of the internal movement (displacement or deformation) of the 3D volume 191) to the segment-corresponding 2D reference image (DR_ref[*]) mentioned above.

[0030] When the internal movement (displacement or deformation) of a three-dimensional volume (here, a patient's body) 191 is estimated (tracked) and a reasonable "estimate" of the displacement (position) or deformation (shape) of a target (e.g., a tumor) 192 is obtained, a treatment beam (e.g., a charged particle beam) 198 is irradiated from a gantry 196 to the target 192 in the radiation treatment room system 109 based on the "estimate" and a treatment plan formulated before radiation treatment. Here, a set of devices for irradiating the treatment beam 198, including the gantry 196, may be referred to as a "treatment beam irradiator." As described above, the treatment beam 198 is extracted from the accelerator 107, transported through the beam transport system 108, and reaches the gantry 196 (including the treatment beam irradiator). The gantry 196 may be rotatable around the three-dimensional volume 191 so that the treatment beam 198 can be irradiated to the target 192 from various angles. At this time, the accelerator 107, the beam transport system 108, and the therapeutic beam irradiator (including the gantry 196) may be controlled based on the "estimated content" of the displacement (position) or deformation (shape) of the target 192 and the treatment plan formulated before the radiation therapy. The system 102 (control system) may directly control the accelerator 107, the beam transport system 108, and the therapeutic beam irradiator. Alternatively, the system 102 (control system) may provide control information to an accelerator controller, a beam transport system controller, and a therapy controller (not shown), and then the accelerator controller may directly control the accelerator, the beam transport system controller may directly control the beam transport system, and the therapy controller may directly control the therapeutic beam irradiator.

[0031] 4. Computer Architecture for Implementing Embodiments of the Present Disclosure (FIG. 5) FIG. 5 illustrates a computer architecture 500 for implementing the system 102 (control system) of an embodiment of the present disclosure. To realize the system 102 (control system), some or all of the information processing device 501, storage device 502, non-volatile storage medium 503, external storage medium drive 504, input device 506, display or output device 507, communication device 508, external input / output port 509, and reading device 510 may be interconnected via an interconnection unit 511. (Note that some or all of the interconnection unit 511 may be a network. In that case, the system 102 (control system) is realized by a plurality of devices connected via the network.) The information processing device 501 may be, for example, a processor. Examples of this processor include a CPU, an MPU, or a GPU. Alternatively, the processor referred to here may be another semiconductor device that executes a predetermined process. The information processing device 501 may also be one or more (micro)processors. The storage device 502 may be, for example, a memory. The non-volatile recording medium 503 may be, for example, a non-volatile memory (e.g., a flash memory) or a non-volatile disk device. The external recording medium drive 504 may be, for example, a disk drive. The input device 506 may be, for example, a mouse, a keyboard, an imaging device, a sensor, a touch panel, or a pointing device. The display or output device 507 may be, for example, a display, a printer, or a speaker. The communication device 508 may be, for example, a communication device for wired communication or a communication device for wireless communication. The communication device 508 may be, for example, a network interface device (NIC) that controls communication with other systems, devices, terminals, or servers according to a predetermined protocol. The interconnection unit 511 may be, for example, a bus or a crossbar switch. The non-volatile recording medium 503 may store a program 520a (for example, a program for realizing the functional configuration according to the present disclosure; for example, various programs for realizing each of the functional units (devices) listed in FIG. 2), various databases 521, and various information 522. The various databases 521 may store information handled in the embodiments of the present disclosure (including information about various images handled in the embodiments of the present disclosure, information about the displacement or deformation of a three-dimensional volume (a three-dimensional image corresponding to the three-dimensional volume), or information about similarity). Alternatively, some or all of the information in the above-described program 520a, various databases 521, or various information 522 may be acquired from outside the device shown in FIG. 5. The external recording medium drive 504 can be connected to an external recording medium 505. The external recording medium 505 may be, for example, a portable recording disk (such as a DVD), an IC card, an SD card, a nonvolatile memory (such as a flash memory), or a portable hard disk. Note that the external recording medium 505 may transfer and store information similar to the program 520a (e.g., a program for implementing the functional configuration according to the present disclosure, e.g., various programs for implementing each of the functional units (devices) listed in FIG. 2), various databases 521, or information in the various information 522 to the nonvolatile recording medium 503 or the storage device 502. The external recording medium 505 may be used to record programs and data handled in the system 102 (control system). The external recording medium drive 504 and the external recording medium 505 may be connected to the system 102 (control system) illustrated in FIG. 5 via a network. Information of the program 520a (for example, a program for realizing the functional configuration related to the present disclosure; for example, various programs for realizing each of the functional units (devices) listed in Figure 2), various databases 521, or various information 522 may be brought via the communication device 508, the external input / output port 509, the input device 506, or the reading device 510 and stored in the non-volatile recording medium 503 or the memory device 502.

[0032] In order for the architecture of FIG. 5 to function as the system 102 (control system), each functional unit (unit) in the system 102 (control system), or a part of each functional unit (unit) (to execute one or a series of processes (steps)), a program 520a may be loaded into the storage device 502 (for example, from the non-volatile recording medium 503). The loaded program is indicated by 520b in FIG. 5. The information processing device 501 may then execute the program 520b (using information from various databases 521 or various information 522 stored in the non-volatile recording medium 503, etc., as necessary). Execution of the program 520b realizes the function of the system 102 (control system), each functional unit (unit) in the system 102 (control system), or a part of each functional unit (unit) (to execute one or a series of processes (steps)). At this time, various buffers 523 temporarily formed in the storage device 502 may also be used as appropriate.

[0033] 5. Processing performed by the embodiment of the present disclosure The following describes the processing performed by the system 102 (control system) according to an embodiment of the present disclosure. It is not necessary to realize all of the functional configurations described below and perform all of the processing. Furthermore, it is not prohibited to realize functional configurations and perform processing other than the functional configurations and processing described below. In the following, we will first explain matters related to before radiation therapy (more generally, before real-time estimation (tracking) of the movement (displacement or deformation) of the three-dimensional volume 191; hereinafter simply referred to as ``before radiation therapy''), and then we will explain matters related to during radiation therapy (more generally, during real-time estimation (tracking) of the movement (displacement or deformation) of the three-dimensional volume 191; hereinafter simply referred to as ``during radiation therapy'').

[0034] 1. Before Radiation Therapy Here, first, an overview of the image information group that the system 102 (control system) handles before radiation therapy will be explained. Next, the processing performed by the system 102 (control system) before radiation therapy will be explained. The processing performed by the system 102 (control system) before radiation therapy can be broadly divided into a process for generating a model 129 used to estimate (track) the internal movement (displacement or deformation) of the three-dimensional volume 191, and a process for preparing a segment-corresponding two-dimensional reference image (DR_ref[*]) (a process for registering the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref)). Therefore, these processes will be explained in order.

[0035] 5.1.1. Image information to be handled before radiation therapy (Figure 6) Figure 6 shows the image information groups that the system 102 (control system) handles before radiation therapy and their relationships. In Figure 6 (and Figure 7), the image information is represented by symbols. The symbols, names of the image information (hereinafter shown in ""), and descriptions of the image information used in Figure 6 (and Figure 7) are associated as follows:

[0036] 4D_ori "Four-dimensional images (time-series information of three-dimensional images)" Time-series information of three-dimensional images generated by three-dimensionally imaging the three-dimensional volume 191 (for example, by CT or MRI) before radiation therapy. 3D_ori "3D original image" A three-dimensional image is generated by three-dimensionally imaging the three-dimensional volume 191 (for example, by CT or MRI) before radiation therapy. ·3D_ori[*] (e.g. 3D_ori[1], 3D_ori[2], 3D_ori[3]...) "Segmented 3D original image" A three-dimensional image obtained by dividing the three-dimensional original image (3D_ori) into segments (e.g., parts, organs, tissues, targets (e.g., tumors), etc.) contained in the three-dimensional volume 191 before radiation therapy. 3D_ref "3D reference image" A three-dimensional image obtained by displacing or deforming the three-dimensional original image (3D_ori) before radiation therapy. 3D_ref[*] (e.g. 3D_ref[1], 3D_ref[2], 3D_ref[3]) "Segment-compatible 3D reference image" A three-dimensional image obtained by displacing or deforming the segment-corresponding three-dimensional original image (3D_ori[*]) before radiation therapy.

[0037] DRR_ref "Two-dimensional reference reconstruction image" A digitally reconstructed X-ray image (DRR) is generated based on a three-dimensional reference image (3D_ref) before radiation therapy. The two-dimensional reference reconstructed image (DRR_ref) is the target for cross-modality image similarity calculation with the two-dimensional reference image (DR_ref) described below. In other words, the two-dimensional reference reconstructed image (DRR_ref) is the target for registration with the two-dimensional reference image (DR_ref). DRR_ref[*] (e.g., DRR_ref[1], DRR_ref[2], DRR_ref[3]) "Segment-compatible two-dimensional reference reconstruction image" Digitally reconstructed radiographs (DRRs) generated based on segmented three-dimensional reference images (3D_ref[*]) before radiation treatment. DR_ref "2D reference image" A two-dimensional image (X-ray image) is generated by x-raying the three-dimensional volume 191 before radiation treatment. DR_ref[*] (e.g., DR_ref[1], DR_ref[2], DR_ref[3]) "Segment-compatible 2D reference image" A two-dimensional image generated before radiation therapy, based on the two-dimensional reference image (DR_ref), corresponding to each segment (e.g., site, organ, tissue, target (e.g., tumor), etc.) included in the three-dimensional volume 191. As will be described later, when generating the segment-corresponding two-dimensional reference image (DR_ref[*]), not only the information in the two-dimensional reference image (DR_ref) but also the information in the two-dimensional reference reconstructed image (DRR_ref) and the information in the segment-corresponding two-dimensional reference reconstructed image (DRR_ref[*]) are used.

[0038] 5.1.2. Model generation process (Figures 7 and 8) A system 102 (control system) according to an embodiment of the present disclosure performs a process of displacing or deforming a three-dimensional image generated by three-dimensionally capturing a three-dimensional volume (e.g., a body) 191, both before and during radiation therapy. The three-dimensional volume 191 includes multiple segments (e.g., a region, an organ, a tissue, a target (e.g., a tumor), etc.), and the manner of displacement or deformation may differ for each segment. Therefore, the system 102 (control system) according to an embodiment of the present disclosure uses a model 129 that models the manner of displacement or deformation for each segment included in the three-dimensional volume 191. For example, the model 129 may provide the state of displacement or deformation for each segment included in the three-dimensional volume 191 according to information on values related to a surrogate tissue (surrogate tissue measurement value ("surrogate" in FIG. 11)) contained in the three-dimensional volume 191. The input information to the model 129 is information on values (surrogate tissue measurements) regarding the surrogate tissue contained in the three-dimensional volume 191, and may be, for example, information on the position of the diaphragm contained in the three-dimensional volume 191, information on the position of the abdominal surface of the three-dimensional volume 191, and information on the amount of inhalation and exhalation between the three-dimensional volume 191 and the outside air. Note that the input information to the model 129 is not limited to the above. For example, anything related to respiratory movement can be input information to the model 129. Other anatomical changes can also be input information to the model 129. Furthermore, the output information of the model 129 is information on the state of displacement or deformation of each segment included in the three-dimensional volume 191, and this information may include information on position, shape, or velocity or acceleration. In this sense, the model 129 may be understood as a motion model or a deformation model.

[0039] 8 shows a flowchart of processing executed by the system 102 (control system) according to an embodiment of the present disclosure before radiation therapy when the system 102 (control system) generates a model 129. Not all processing steps shown in FIG. 8 are essential. In addition, the presence of processing steps other than those shown in FIG. 8 is not precluded. In the description using FIG. 8, the functional configuration 700 shown in FIG. 7 will also be referenced as appropriate.

[0040] Either step 801 or step 802 in FIG. 8 may be executed (or both may be executed). In step 801 of Fig. 8, the model generator 141 may acquire time-series information of three-dimensional images (so-called four-dimensional information (4D_ori). In the case of CT imaging, it may also be called four-dimensional CT images (4DCT)). The acquired time-series information of three-dimensional images may be, for example, (1) time-series information consisting of a collection of three-dimensional images (at different treatment fractions) acquired in three-dimensional imaging on different days (or time periods), or (2) time-series information consisting of a collection of three-dimensional images showing different breathing phases. When the time series information of the three-dimensional image as described above is used to generate the model 129, the time series information of the three-dimensional volume 191 itself, which is the subject of real-time estimation (tracking) of displacement and deformation, is reflected in the model 129, so that a model 129 customized for the three-dimensional volume 191 can be generated. As shown in FIG. 7, a certain three-dimensional image included in the four-dimensional image (4D_ori) (for example, a three-dimensional image showing a specific respiratory phase (for example, the respiratory phase immediately before the start of inhalation (0% inhaled))) may be used as a three-dimensional original image (3D_ori) to be handled later. (Alternatively, the three-dimensional original image (3D_ori) may be generated by three-dimensionally capturing the three-dimensional volume 191, independently of the four-dimensional image (4D_ori) used by the model generator 141.) In step 802 of FIG. 8, the model generator 141 may obtain case information ("database" in FIG. 7) of displacement or deformation for each segment (e.g., site, organ, tissue, target (e.g., tumor)) in the three-dimensional volume from a database that accumulates case information regarding displacement or deformation within various three-dimensional volumes. When case information from the database as described above is used to generate the model 129, many cases are reflected in the model 129, so that the model 129 can be generated such that its validity as a model is not significantly impaired. 8, the model generator 141 generates the model 129. In this case, the model generator 141 generates the model 129 using one or both of the four-dimensional image (4D_ori) of the three-dimensional volume 191, which is the target of real-time estimation (tracking) of displacement or deformation, acquired in step 801, and case information (database) of various three-dimensional volumes acquired in step 802. For example, the model generator 141 may generate the model 129 by performing statistical analysis (e.g., principal component analysis, analysis using approximation with linear or quadratic functions, or analysis using other mathematical expressions) on the information obtained in step 801 or step 802. Alternatively, the model generator 141 may use the information obtained in step 801 or step 802 as learning data for the model 129 and train the model 129 using machine learning to generate a trained model 129. As described above, the generated model 129 may provide, for example, the displacement or deformation state of each segment included in the three-dimensional volume 191 corresponding to a surrogate tissue measurement value in the three-dimensional volume 191. (Note that the model 129 may also be capable of accepting direct setting (specification) of a three-dimensional deformation vector field that directly represents the displacement or deformation state of each location (coordinate) included in the three-dimensional volume 191.) As shown in FIG. 7, the model 129 generated by the model generator 141 is to be used in the three-dimensional displacement transformer 123 .

[0041] A model 129 may be generated for one or more limited segments among the group of segments included in the three-dimensional volume 191. For example, a model 129 may be generated for only a specific region, organ, or tissue (e.g., the pancreas) (and its surrounding regions, organs, or tissues, as necessary). In this way, the cost of creating the model 129 can be reduced depending on the purpose of estimating (tracking) the displacement or deformation of the three-dimensional volume 191.

[0042] If the model 129 already exists or another system generates the model 129, the system 102 (control system) may use the already existing model 129 or the model 129 generated by the other system without generating the model 129 itself (without executing steps 801, 802, and 803 in FIG. 8). In this case, the system 102 (control system) may not need to include the model generator 141.

[0043] 5.1.3. Processing for preparing segment-compatible 2D reference images (Figure 7 and Figure 9) A system 102 (control system) according to an embodiment of the present disclosure calculates similarity (same-modality image similarity) between a two-dimensional real-time image (DR_RT) and a two-dimensional simulation image (DR_sim) during radiation therapy. Here, the two-dimensional real-time image (DR_RT) is generated by X-ray imaging of a three-dimensional volume (e.g., a body) 191. The two-dimensional simulation image (DR_sim) corresponds to an "estimated content" of displacement or deformation for each segment (e.g., a site, organ, tissue, target (e.g., a tumor), etc.) included in the three-dimensional volume 191. Furthermore, the two-dimensional simulation image (DR_sim) is generated by combining each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment. Additionally, each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) is obtained by displacing or deforming each of the segment-corresponding two-dimensional reference images (DR_ref[*]) according to the "estimated content." In view of the above, when radiation therapy (real-time estimation (tracking) of the movement (displacement or deformation) of the three-dimensional volume 191) is performed, a situation is prepared in which the system 102 (control system) of an embodiment of the present disclosure can use each of the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment included in the three-dimensional volume 191 in advance. For this preparation, the system 102 (control system) of an embodiment of the present disclosure performs a process to prepare a segment-corresponding two-dimensional reference image (DR_ref[*]) before radiation therapy (a process to register the two-dimensional reference reconstruction image (DRR_ref) and the two-dimensional reference image (DR_ref)).

[0044] FIG. 9 shows a flowchart of a process executed by the system 102 (control system) to prepare a segment-corresponding two-dimensional reference image (DR_ref[*]) before radiation therapy. Not all of the process steps shown in FIG. 9 are required. Furthermore, the presence of process steps other than those shown in FIG. 9 is not precluded. In addition, in the description using FIG. 9, the functional configuration 700 shown in FIG. 7 will also be referenced as appropriate.

[0045] In step 901 of FIG. 9, the three-dimensional image acquirer 151 acquires a three-dimensional original image (3D_ori) generated by three-dimensionally capturing (e.g., CT or MRI) a three-dimensional volume (e.g., a body) 191. As shown in FIG. 7, a certain three-dimensional image included in the four-dimensional image (4D_ori) (e.g., a three-dimensional image showing a specific respiratory phase (e.g., a respiratory phase immediately before the start of inhalation (0% inhaled), which is a stable respiratory phase)) may be used as the three-dimensional original image (3D_ori). Alternatively, the three-dimensional original image (3D_ori) may be generated by three-dimensionally capturing (e.g., CT or MRI) the three-dimensional volume 191, independently of the four-dimensional image (4D_ori) used by the model generator 141. After step 901, control transitions to step 902.

[0046] In step 902 of FIG. 9, the three-dimensional segment generator 152 divides the three-dimensional original image (3D_ori) into segment-corresponding three-dimensional original images (3D_ori[*]) for each segment (e.g., site, organ, tissue, target (e.g., tumor), etc.) included in the three-dimensional volume 191. Here, the criteria used for dividing the three-dimensional volume 191 (three-dimensional original image (3D_ori)) into segments may be, for example, one or more of the following (1) to (3). (Criteria other than the following (1) to (3) may also be used.) (1) The velocity vector (displacement or deformation pattern) of the movement of each voxel (or each region of a set of voxels) contained in the original 3D image (3D_ori). In many cases, voxels that should belong to different segments are expected to have significantly different velocity vectors at the same time, so the velocity vector is an appropriate criterion for segmentation. (2) Intensity (voxel value as image data) of each voxel contained in the 3D original image (3D_ori). In many cases, voxels that should belong to different segments are assumed to have different values (voxel intensity, voxel value) in the image data, so the values in the image data are appropriate for determining segmentation. (3) Empirical knowledge about the positions of segments contained in the three-dimensional volume 191. For example, if the three-dimensional volume 191 is a body, the approximate positions of parts, organs, tissues, etc. that can become segments are known (excluding individual differences). Therefore, empirical knowledge about the positions of segments is appropriate as a basis for determining segment division. The three-dimensional segment generator 152 may perform all of the segmentation of the three-dimensional volume 191 (three-dimensional original image (3D_ori)) automatically, or may receive input from an expert with knowledge or experience regarding each of the segments included in the three-dimensional volume 191. When the three-dimensional segment generator 152 receives input from an expert, either of the following modes (A) and (B) may be used. (Modes other than the following modes (A) and (B) may also be used.) (A) The three-dimensional segment generator 152 presents (displays or outputs) the three-dimensional original image (3D_ori) to the expert, who then inputs details that manually specify how to segment the three-dimensional volume 191 (the three-dimensional original image (3D_ori)). (B) The three-dimensional segment generator 152 presents (displays or outputs) a segment division plan for the three-dimensional original image (3D_ori) to the expert, who then inputs details to modify or confirm the segment division plan. When segmenting the three-dimensional volume 191 (three-dimensional original image (3D_ori)), the segments may be, for example, a site, an organ, a tissue, or a target (e.g., a tumor), but other things may also be used as segments. For example, the segment may be a fluid flowing inside the three-dimensional volume (e.g., a body) 191. The fluid in this case may be, for example, a gas bubble in a digestive tract or urine in a bladder.

[0047] In step 903 of FIG. 9, the two-dimensional image acquirer 121 acquires a two-dimensional reference image (DR_ref) generated by x-raying a three-dimensional volume (eg, a body) 191. As shown in FIGS. 1, 3, and 4, an X-ray source 194a emits X-rays, the emitted X-rays pass through (a region of) the three-dimensional volume 191, and the transmitted X-rays are detected by an X-ray image detector 195a to generate a two-dimensional image (X-ray image). Similarly, an X-ray source 194b emits X-rays, the emitted X-rays pass through (a region of) the three-dimensional volume 191, and the transmitted X-rays are detected by an X-ray image detector 195b. In FIGS. 1, 3, and 4, X-ray imaging is performed on the three-dimensional volume 191 from two directions. By performing X-ray imaging from two directions, it is expected that the state of each coordinate in the three-dimensional space inside the three-dimensional volume 191 can be understood to some extent. As already pointed out, the number of directions in which X-ray imaging can be performed on the three-dimensional volume 191 (for example, the number of combinations of X-ray sources and X-ray image detectors) may be three or more, or may be one. Here, the X-ray image detector 195a and the X-ray image detector 195b may be equipped with a flat panel detector (FPD) and generate a digital X-ray image (Digital Radiography (DR)) as a two-dimensional image (X-ray image). As described above, the two-dimensional image (digital X-ray image (DR)) generated by the X-ray image detector 195a or the X-ray image detector 195b is acquired by the two-dimensional image acquirer 121 operating in the system 102 (control system) and used as a two-dimensional reference image (DR_ref).

[0048] In step 904 of FIG. 9, the surrogate tissue measurement acquirer 125 (not shown in FIG. 7 but operable) may acquire surrogate tissue measurements ("surrogate" in FIG. 7). The surrogate tissue measurements are information generated by measuring the position, shape, and other conditions of a surrogate tissue (surrogate) included in the three-dimensional volume (e.g., the body) 191. The surrogate tissue measurements (surrogate) are, for example, the position of the diaphragm, the position of the abdominal surface, and the amount of external air inhaled and exhaled in the three-dimensional volume 191 due to breathing. Here, information on the position of the abdominal surface may be acquired by a camera. Information on the amount of external air inhaled and exhaled in the three-dimensional volume 191 due to breathing may be acquired by a flow meter. (However, if the position of the diaphragm can be determined based on the aforementioned two-dimensional reference image (DR_ref) by a template matching technique, a fitting technique using a quadratic function, or the like, the information on the position of the diaphragm does not need to be acquired via the surrogate tissue measurement acquirer 125.) Note that in step 905 described below, if the displacement / deformation setter 122 determines the "pre-set contents" of the displacement or deformation of each of the segment-corresponding three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]) without using surrogate tissue measurement values (surrogate), then this step 904 does not need to be performed.

[0049] The series of steps from step 905 to step 910 in Figure 9 determines the content of displacement or deformation ("pre-set content") of the three-dimensional original image (3D_ori) so that the inter-modality image similarity between the digitally reconstructed X-ray image (DIGITICALLY RECONSTRUCTED RADIOGRAPHY (DRR)) based on the three-dimensional reference image (3D_ref) generated by displacing or deforming the three-dimensional original image (3D_ori) and the two-dimensional reference image (DR_ref) acquired in step 903 is sufficiently high. In other words, a kind of matching process called registration is performed between the two-dimensional reference reconstructed image (DRR_ref), which is the digitally reconstructed X-ray image (DRR) mentioned above, and the two-dimensional reference image (DR_ref) mentioned above. Each step will be described in order below.

[0050] In step 905 of FIG. 9, the displacement / deformation setter 122 determines the details of displacement or deformation ("pre-set details") to be performed on each of the three-dimensional original image (3D_ori) acquired in step 901 and the segment-corresponding three-dimensional original image (3D_ori[*]) generated in step 902. Note that the term "pre-set details" is used here because it indicates the details of the displacement or deformation processing performed by the three-dimensional displacement / deformation setter 123 before radiation therapy. (In contrast, the term "estimated details" is used to indicate the details of the displacement or deformation processing performed by the three-dimensional displacement / deformation setter 123 during radiation therapy, which will be described later using FIG. 12.) When step 905 is executed for the first time, the displacement / deformation setter 122 may determine the "pre-set contents" using one of the following methods. (1) The displacement / deformation setting unit 122 determines the "pre-set contents" so as to indicate that no displacement or deformation is to be performed on each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). (2) The displacement / deformation setting unit 122 determines the "pre-setting contents" so that a displacement or deformation reflecting a predetermined value is applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). By appropriately determining the above-mentioned "predetermined value," various "pre-setting contents" can be generated appropriately. (3) The displacement / deformation setting unit 122 determines the "pre-setting contents" so that a displacement or deformation reflecting a value based on a random number (random) is applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). By using random numbers, various "pre-setting contents" can be generated as appropriate. (4) The displacement and deformation setting unit 122 determines the "pre-setting contents" so that a displacement or deformation reflecting the information of the three-dimensional original image (3D_ori) (or the segment-compatible three-dimensional original image (3D_ori[*])) and some or all of the information of the two-dimensional reference image (DR_ref) acquired in step 903 is applied to each of the three-dimensional original image (3D_ori) and the segment-compatible three-dimensional original image (3D_ori[*]). For example, the displacement and deformation setter 122 may determine the initial "pre-set content" based on the results of comparing the positions of landmarks (e.g., diaphragm or bones) between a digitally reconstructed X-ray image (DRR) obtained from a three-dimensional original image (3D_ori) (or a segment-corresponding three-dimensional original image (3D_ori[*])) and a two-dimensional reference image (DR_ref). (5) The displacement / deformation setter 122 determines the "pre-setting contents" so that the displacement or deformation reflecting the surrogate tissue measurement value (surrogate) acquired in step 904 is applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). (This technique (5) may be used, for example, when the timing at which the three-dimensional volume 191 is three-dimensionally imaged to generate the three-dimensional original image (3D_ori) is different from the timing at which the three-dimensional volume 191 is X-ray imaged to generate the two-dimensional reference image (DR_ref).) The "pre-set contents" (and "estimated contents" described below) may be surrogate tissue measurements (e.g., information indicating the position and shape of a predetermined portion (e.g., the diaphragm) in the three-dimensional volume 191 (or a corresponding three-dimensional image)) (information that corresponds to input information to the model 129). By determining the "pre-set contents" (and "estimated contents" described below) based on the surrogate tissue measurements (surrogate), it is possible to associate the surrogate tissue measurements (surrogate) with displacements or deformations within the three-dimensional volume 191. Additionally, the "pre-set content" (and "estimated content" described below) may indicate a three-dimensional deformation vector field to be applied to some or all segments in a three-dimensional volume (e.g., a body) 191 (or a corresponding three-dimensional image). After step 905, control transitions to step 906.

[0051] In step 906 of Figure 9, the three-dimensional displacement transformer 123 displaces or transforms the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]) for each segment based on the model 129 so as to reflect the "pre-set contents" determined in step 905. The three-dimensional displacement transformer 123 sets the results of the displacement or transformation process as the three-dimensional reference image (3D_ref) and the segment-corresponding three-dimensional reference image (3D_ref[*]) for each segment. After step 906, control transitions to step 907.

[0052] 9, the reconstructed image generator 124 generates a digitally reconstructed X-ray image (DRR) based on the three-dimensional reference image (3D_ref) and the segment-corresponding three-dimensional reference images (3D_ref[*]) for each segment generated in step 906. The reconstructed image generator 124 converts the digitally reconstructed X-ray image (DRR) obtained into a two-dimensional reference reconstructed image (DRR_ref) and a segment-corresponding two-dimensional reference reconstructed image (DRR_ref[*]). When the reconstructed image generator 124 generates the digital reconstructed X-ray image (DRR), the projection plane when generating a two-dimensional image (digital reconstructed X-ray image (DRR)) based on information of a three-dimensional image is set to include the plane when the two-dimensional image (digital X-ray image (DR)) was generated by the flat panel detector (FPD) of the X-ray image detector 195a or the X-ray image detector 195b. In other words, the two-dimensional reference reconstructed image (DRR_ref) generated in step 907 and the two-dimensional reference image (DR_ref) acquired in step 903 are set to be image information on approximately the same plane (although they have different modalities). After step 907, control is transferred to step 908.

[0053] 9, the inter-modality similarity calculator 153 calculates the inter-modality image similarity between the two-dimensional reference image (DR_ref) acquired in step 903 and the two-dimensional reference reconstructed image (DRR_ref) generated in step 907. The inter-modality image similarity may be, for example, the mutual information between two images having different modalities. Alternatively, the inter-modality image similarity may indicate the degree of proximity of the boundary lines between regions formed by high and low pixel intensities (pixel values) in the two images. It should be noted that, in step 909 described below, if the content of the expert's judgment is input to the system 102 (control system), this step 908 may not be necessary. After step 908, control transitions to step 909.

[0054] 9, the volume preset validity determiner 155 determines the validity of the "preset contents" determined in step 905. That is, the volume preset validity determiner 155 determines whether the registration (a kind of matching process) between the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref) acquired in step 903 was successful. Here, the two-dimensional reference reconstructed image (DRR_ref) is obtained as a result of the displacement or deformation made to the three-dimensional original image (3D_ori). The volume preset validity determiner 155 may determine the validity of the "preset content" based on the inter-modality image similarity calculated in step 908. For example, if the mutual information between the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref) is equal to or greater than a predetermined threshold (or is larger than a predetermined threshold), the volume preset validity determiner 155 may determine that the "preset content" is valid. Alternatively, the volume preset validity determiner 155 may present (display or output) the two-dimensional reference reconstruction image (DRR_ref) and the two-dimensional reference image (DR_ref) to a knowledgeable or experienced expert and accept input indicating the expert's judgment regarding the validity of the "pre-setting content." In this case, the above-mentioned step 908 may not be performed. Alternatively, step 908 may be performed, and the inter-modality image similarity (e.g., mutual information) may be presented (displayed or output) to the expert. Compared to during radiation therapy, which will be described later with reference to FIG. 12, before radiation therapy as shown in FIG. 9, there is more time to input the judgment of a knowledgeable or experienced expert into the system 102 (control system), so the above-mentioned operation is also possible. 9, the volume preset validity determiner 155 may determine the validity of the relationship between the two-dimensional reference image (DR_ref) and the segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment, in addition to determining whether the above-mentioned registration was successful, in order to determine the validity of the "preset contents." Specifically, first, the volume preset validity determiner 155 (or the segment-corresponding two-dimensional reference image generator 156 requested by the volume preset validity determiner 155) generates a segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment based on the two-dimensional reference image (DR_ref), the two-dimensional reference reconstruction image (DRR_ref), and the segment-corresponding two-dimensional reference reconstruction image (DRR_ref[*]) for each segment, in the same manner as described in the description of step 911 below. Next, the volume preset validity determiner 155 calculates the similarity (same-modality image similarity) between the image obtained by recombining the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment and the two-dimensional reference image (DR_ref). The volume preset validity determiner 155 may use, for example, that the calculated similarity (same-modality image similarity) is equal to or greater than a predetermined threshold (or is larger than a predetermined threshold) as one of the conditions for determining that the "preset content" is valid. By determining the validity of the relationship between the two-dimensional reference image (DR_ref) and the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment as described above, the validity of the "pre-set contents" can be determined more appropriately. Furthermore, if the similarity (similarity between images of the same modality) between the image obtained by recombining the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment and the two-dimensional reference image (DR_ref) is valid, then in either step 909 or step 911, the coefficient "k[i]" for each segment in (Equation 3) for combining the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment to obtain the two-dimensional simulation image (DR_sim) can be appropriately determined, as will be described in the explanation of step 1208 below. After step 909, control transitions to step 910.

[0055] 9, the volume preset validity determiner 155 performs branching processing in control depending on whether the "preset contents" determined in step 909 are valid. Specifically, if it is determined that the "preset contents" are valid, control transitions to step 911. If it is determined that the "preset contents" are invalid, control returns to step 905. When control is returned to step 905 (i.e., when step 905 is executed for the second time or later), the displacement / deformation setter 122 determines new content (new "pre-set content") as the content of the displacement or deformation ("pre-set content") to be performed on each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). When step 905 is executed for the second or subsequent time, the displacement / deformation setter 122 may determine the "pre-set contents" using any of the following methods. (1) The displacement / deformation setter 122 determines new "pre-set contents" so that new displacements or deformations reflecting predetermined values (different from the values when step 905 was last executed) are applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). By appropriately determining the above-mentioned "predetermined value," various "pre-setting contents" can be generated appropriately. (2) The displacement / deformation setter 122 determines new “pre-set contents” so that the new displacement or deformation obtained by modifying the previous “pre-set contents” to reflect specified values is applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). The above-mentioned "predetermined value" is determined appropriately, and by using this "predetermined value" when determining the amount of change in the "predetermined content," various "predetermined content" can be generated appropriately. (3) The displacement / deformation setter 122 determines new "pre-set contents" so that new displacements or deformations reflecting values (random) based on random numbers (different from the last time step 905 was executed) are applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). By using random numbers, various "pre-setting contents" can be generated as appropriate. (4) The displacement / deformation setter 122 determines new “pre-set contents” so that the new displacement or deformation obtained by making changes to the previous “pre-set contents” that reflect values based on random numbers (random) is applied to each of the three-dimensional original image (3D_ori) and the segment-corresponding three-dimensional original image (3D_ori[*]). By using random numbers when determining the amount of change in the "pre-set contents," various "pre-set contents" can be generated as appropriate. (5) The displacement and deformation setter 122 back projects the "difference" between the two-dimensional reference reconstructed image (DRR_ref) based on the previous "pre-set content" and the two-dimensional reference image (DR_ref) acquired in step 903 onto the three-dimensional space in which the three-dimensional volume 191 (the three-dimensional original image (3D_ori), the segment-compatible three-dimensional original image (3D_ori[*])) is located. The displacement and deformation setter 122 determines new "pre-set content" so that new displacements or deformations obtained by modifying the previous "pre-set content" to reflect the back projection results are applied to the three-dimensional original image (3D_ori) and the segment-compatible three-dimensional original image (3D_ori[*]), respectively. Furthermore, the "difference" information between the two-dimensional reference reconstruction image (DRR_ref) and the two-dimensional reference image (DR_ref) may be derived from information calculated by the inter-modality similarity calculator 153, or may be "difference" information calculated separately by the inter-modality difference calculator 154 (which may be operational but is not shown in FIG. 7). The "difference" between the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref) may be a quantity related to the mutual information between the two images, a two-dimensional deformation vector field when one image is displaced or deformed to the other, the sum of squared difference (SSD) based on the "difference (error)" of pixel intensity (pixel value) for each pixel between the two images (the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref)), or the mean squared error (MSE) based on the "difference (error)". By defining new "pre-setting contents" to reflect the "difference" between the two images (the two-dimensional reference reconstruction image (DRR_ref) and the two-dimensional reference image (DR_ref)) projected back into the above three-dimensional space, it is expected that new "pre-setting contents" will be defined that will reduce the "difference."

[0056] 9, when the "pre-set contents" are determined to be valid and control transitions to step 911, this means that successful registration (a kind of matching process) has been achieved between the two-dimensional reference reconstructed image (DRR_ref) based on the "pre-set contents" and the two-dimensional reference image (DR_ref) acquired in step 903. (Indirectly, this means that successful registration (a kind of matching process) has also been achieved between the segment-corresponding two-dimensional reference reconstructed image (DRR_ref[*]) based on the "pre-set contents" and the two-dimensional reference image (DR_ref).) In this way, step 911 is performed after the registration has been properly achieved. 9, the segment-corresponding two-dimensional reference image generator 156 generates a segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment. The segment-corresponding two-dimensional reference image (DR_ref[i]) for a certain segment (identifier is i) may be determined based on a value indicating the relationship between the two-dimensional reference reconstructed image (DRR_ref) and the segment-corresponding two-dimensional reference reconstructed image (DRR_ref[i]) for the certain segment (i) (e.g., information indicating the ratio of pixel intensity (pixel value) at the same position between the two images), and information on the two-dimensional reference image (DR_ref). When the intensity (pixel value) of a pixel at a position (x, y) on a two-dimensional image I is pi(I, x, y), the segment-corresponding two-dimensional reference image generator 156 may calculate the pixel intensity (pixel value) pi(DR_ref[i], x, y) of a segment-corresponding two-dimensional reference image (DR_ref[i]) for each segment (where i is the identifier) based on the following equation (Equation 1 or Equation 2; these equations are mathematically equivalent): In the following equation, "ln" denotes the natural logarithm (for example, "ln(x)" is the natural logarithm of x), "exp" denotes Napier's constant, "^" denotes exponentiation (for example, "A^B" is "A to the Bth power"), and N[i] denotes an adjustable parameter determined for each segment (i). ln(pi(DR_ref[i],x,y)) / ln(pi(DR_ref,x,y)) =[ln(pi(DRR_ref[i],x,y)) / ln(pi(DRR_ref,x,y))]^N[i] (Equation 1) pi(DR_ref[i],x,y) =exp^[ln(pi(DR_ref,x,y))*[[ln(pi(DRR_ref[i],x,y)) / ln(pi(DRR_ref,x,y))]^N[i]]] (Formula 2) Note that terms expressing other effects may be added to the above equations (Equation 1 or 2). For example, terms expressing effects such as scattering and beam hardening may be added. Note that step 911 may not be necessary if segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment have already been generated as part of step 909. Furthermore, as will be described later in the explanation of step 1208, the coefficient "k[i]" for each segment in (Equation 3) for synthesizing segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment to obtain a two-dimensional simulation two-dimensional image (DR_sim) may also be determined in either step 909 or step 911.

[0057] Upon completion of the processing of step 911 in FIG. 9 , the system 102 (control system) has prepared each of the segment-corresponding two-dimensional reference reconstruction images (DRR_ref[*]) and each of the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment (e.g., site, organ, tissue, target (e.g., tumor), etc.) of the three-dimensional volume (e.g., body) 191. Each of the segment-corresponding two-dimensional reference reconstruction images (DRR_ref[*]) and each of the segment-corresponding two-dimensional reference images (DR_ref[*]) are used by the system 102 (control system) in the processing during radiation therapy, which will be described with reference to FIG. 12 . Therefore, each of the segment-corresponding two-dimensional reference reconstruction images (DRR_ref[*]) and each of the segment-corresponding two-dimensional reference images (DR_ref[*]) may be recorded in some kind of recording device (recording medium). Such a recording device may be, for example, a non-volatile recording medium 503 included in the system 102 (control system), or may be any other recording device or database accessible by the system 102 (control system). In the process during radiation therapy described with reference to FIG. 12, the system 102 (control system) displaces or deforms three-dimensional images (segment-corresponding three-dimensional images for each segment) generated by previously three-dimensionally capturing the three-dimensional volume 191. Therefore, either or both of the segment-corresponding three-dimensional original images (3D_ori[*]) and the segment-corresponding three-dimensional reference images (3D_ref[*]) handled in the flowchart of FIG. 9 may be recorded in some kind of recording device (recording medium). (In addition, another segment-corresponding three-dimensional image (3D_other[*]) generated by separately capturing three-dimensional images of the three-dimensional volume 191 may be recorded in some kind of recording device (recording medium), and then the other segment-corresponding three-dimensional image (3D_other[*]) may be displaced or deformed in the process shown in the flowchart of FIG. 12.) Such a recording device may be, for example, a non-volatile recording medium 503 included in the system 102 (control system), or may be some kind of recording device or database accessible by the system 102 (control system).

[0058] As described above, the system 102 (control system) performs pre-radiation therapy processing, and therefore, in accordance with the "pre-set contents" of the internal displacement or deformation of the three-dimensional volume 191, the system 102 (control system) can sequentially generate a three-dimensional reference image (3D_ref[*]), generate segment-corresponding three-dimensional reference images (3D_ref[*]) for each segment included in the three-dimensional volume 191, generate a two-dimensional reference reconstructed image (DRR_ref), and generate segment-corresponding two-dimensional reference reconstructed images (DRR_ref[*]) for each segment. In other words, the system 102 (control system) can generate a two-dimensional reference reconstructed image (DRR_ref) in accordance with the "pre-set contents" of the internal displacement or deformation of the three-dimensional volume 191, and generate segment-corresponding two-dimensional reference reconstructed images (DRR_ref[*]) for each segment. Then, the system 102 (control system) separately acquires a two-dimensional reference image (DR_ref) generated by X-ray imaging of the three-dimensional volume 191, and compares the two-dimensional reference image (DR_ref) with the two-dimensional reference reconstructed image (DRR_ref) according to the "pre-set contents." Based on the comparison result, the system 102 (control system) can determine whether the "pre-set contents" are valid. Furthermore, once appropriate "pre-setting contents" are determined, the system 102 (control system) can generate segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment based on the various image information described above. In this way, the system 102 (control system) can prepare various image information to be used during radiation therapy.

[0059] 5-2. Radiation therapy First, an overview of the image information handled by the system 102 (control system) during radiation therapy will be explained. Next, the processing performed by the system 102 (control system) before radiation therapy will be explained. The processing performed by the system 102 (control system) during radiation therapy includes processing for estimating (tracking) the internal movement (displacement or deformation) of the three-dimensional volume 191 in real time.

[0060] 5.2.1. Image information handled during radiation therapy (Figure 10) FIG. 10 shows the image information group (and two-dimensional deformation vector field (2D-DVF) group) handled by the system 102 (control system) during radiation therapy and their relationships. In FIG. 10 (and FIG. 11), image information, etc. are represented by symbols. The symbols, names of image information, etc. (hereinafter shown in ""), and descriptions of image information, etc. used in FIG. 10 (and FIG. 11) are associated as follows:

[0061] ·3D_ori[*] (e.g. 3D_ori[1], 3D_ori[2], 3D_ori[3]...) "Segmented 3D original image" A three-dimensional image obtained by dividing the three-dimensional original image (3D_ori) into segments (e.g., parts, organs, tissues, targets (e.g., tumors), etc.) contained in the three-dimensional volume 191 before radiation therapy. 3D_ref[*] (e.g. 3D_ref[1], 3D_ref[2], 3D_ref[3]) "Segment-compatible 3D reference image" A three-dimensional image obtained by displacing or deforming the segment-corresponding three-dimensional original image (3D_ori[*]) before radiation therapy. ·3D_other[*] (e.g. 3D_other[1], 3D_other[2], 3D_other[3]...) "Another segmented 3D image" A three-dimensional image obtained by generating a three-dimensional image by three-dimensionally photographing the three-dimensional volume 191 before radiation therapy, separately from 3D_ori[*] and 3D_ref[*], and then dividing the three-dimensional image into segments contained in the three-dimensional volume 191. Hereinafter, when a segment-corresponding three-dimensional image of either 3D_ori[*], 3D_ref[*], or 3D_other[*] is sufficient, it will be referred to as a "segment-corresponding three-dimensional image" or a "segment-corresponding three-dimensional image (3D_ori[*], 3D_ref[*], or 3D_other[*])." ·3D_model[*] (e.g. 3D_model[1], 3D_model[2], 3D_model[3]...) "Segment-compatible 3D model image" A 3D image obtained by displacing or deforming a segmented 3D image (3D_ori[*], 3D_ref[*], or 3D_other[*]) during radiation therapy.

[0062] DRR_model[*] (e.g., DRR_model[1], DRR_model[2], DRR_model[3]) "Segment-compatible 2D model reconstruction image" Digitally reconstructed radiographs (DRR) are generated during radiation therapy based on segmented three-dimensional model images (3D_model[*]). DRR_ref[*] (e.g., DRR_ref[1], DRR_ref[2], DRR_ref[3]) "Segment-compatible two-dimensional reference reconstruction image" Digitally reconstructed radiographs (DRRs) generated based on segmented three-dimensional reference images (3D_ref[*]) before radiation treatment. ·2D-DVF_model[*] (e.g. 2D-DVF_model[1], 2D-DVF_model[2], 2D-DVF_model[3]...) "Segment-based two-dimensional deformation vector field" A 2D deformation vector field for each segment generated during radiation therapy based on the segment-matched 2D model reconstruction image (DRR_model[*]) and the segment-matched 2D reference reconstruction image (DRR_ref[*]). This 2D deformation vector field represents the change (displacement or deformation on a 2D plane) for each segment from the state represented by the segment-matched 2D reference reconstruction image (DRR_ref[*]) to the state represented by the segment-matched 2D model reconstruction image (DRR_model[*]). DR_ref[*] (e.g., DR_ref[1], DR_ref[2], DR_ref[3]) "Segment-compatible 2D reference image" A two-dimensional image generated before radiation therapy based on the two-dimensional reference image (DR_ref) corresponding to each segment included in the three-dimensional volume 191. As described above, when the segment-corresponding two-dimensional reference image (DR_ref[*]) is generated, not only the information of the two-dimensional reference image (DR_ref) but also the information of the two-dimensional reference reconstructed image (DRR_ref) and the information of the segment-corresponding two-dimensional reference reconstructed image (DRR_ref[*]) are used. ·DR_sim[*] (e.g., DR_sim[1], DR_sim[2], DR_sim[3]···) "Segment-compatible two-dimensional simulation image" A two-dimensional image obtained by applying displacement or deformation on a two-dimensional plane based on the segment-corresponding two-dimensional deformation vector field (2D-DVF_model[*]) to the segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment during radiation therapy. DR_sim "Two-dimensional simulation image" A 2D image obtained by combining segment-specific 2D simulation images (DR_sim[*]) obtained for each segment during radiation therapy. The 2D simulation image (DR_sim) is the target for the same modality image similarity calculation with the 2D real-time image (DR_RT) described below. DR_RT "Two-dimensional real-time images" During radiation therapy, a two-dimensional image (X-ray image) is generated by x-raying a three-dimensional volume 191 .

[0063] 5.2.2. Estimation of displacement and deformation of 3D volume during radiation therapy (Figures 11 and 12) 9 is performed, the system 102 (control system) can use each of the segment-corresponding two-dimensional reference reconstruction images (DRR_ref[*]) and each of the segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment (e.g., region, organ, tissue, target (e.g., tumor), etc.) included in the three-dimensional volume (e.g., body) 191. The system 102 (control system) can also use each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]) for each segment included in the three-dimensional volume 191. With the various image information described above available, the system 102 (control system) of an embodiment of the present disclosure performs real-time estimation (tracking) of the movement (displacement or deformation) of the three-dimensional volume 191, for example, during radiation therapy.

[0064] Compared to before radiation therapy, there is almost always less time to spare when the system 102 (control system) performs real-time estimation (tracking). Because there is relatively less time to spare when the system 102 (control system) performs real-time estimation (tracking), it is assumed that the following situations (1) and (2) will occur. (1) It is (in most cases) difficult for the system 102 (control system) to newly acquire in real time three-dimensional images generated by three-dimensionally photographing (e.g., CT or MRI) a three-dimensional volume (e.g., a body) 191. (2) It is often difficult for an expert with knowledge or expertise to visually inspect the images, and thus to verify the similarity between the images. Taking the above circumstances (1) and (2) into consideration, when the system 102 (control system) performs real-time estimation (tracking), the following (1) and (2) are performed. (1) The system 102 (control system) acquires new information of a type that does not require a long time to acquire new information, considering the timescale for real-time estimation (tracking) of displacement or deformation inside the three-dimensional volume 191. (2) The system 102 (control system) performs the verification of the similarity between images as automatically as possible (so that visual judgment by an expert is not required as much as possible). Specifically, during real-time estimation (tracking) of the movement (displacement or deformation) of the three-dimensional volume 191, the system 102 (control system) achieves the following: (1 of 1) The system 102 (control system) does not assume three-dimensional images that are generated by newly taking three-dimensional photographs of the three-dimensional volume 191, but instead utilizes each of the already existing segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). (1-2) The system 102 (control system) acquires two-dimensional images (X-ray images) generated by newly taking X-ray images of the three-dimensional volume 191 in real time. (1-3) The system 102 (control system) may acquire new values (surrogate tissue measurements) in real time, for example, obtained by measuring surrogate tissue included in the three-dimensional volume 191. (The surrogate tissue measurements here are assumed to be those that do not require a long time to acquire new information.) (2) When verifying the similarity between images, the system 102 (control system) verifies the similarity between images having the same modality. Verification (comparison) of the similarity between images having the same modality is more likely to be automated (compared to verification (comparison) of the similarity between images having different modalities). Furthermore, as described above, verification (comparison) of the similarity between images having the same modality enables real-time estimation (tracking) of displacement or deformation of a wide variety of segments (e.g., parts, organs, tissues, targets (e.g., tumors), etc.) included in the three-dimensional volume (e.g., body) 191.

[0065] Fig. 12 shows a flowchart of the processing executed by the system 102 (control system) during radiation therapy. Not all processing steps shown in Fig. 12 are required. Furthermore, the presence of processing steps other than those shown in Fig. 12 is not precluded. In the explanation using Fig. 12, the functional configuration 1100 shown in Fig. 11 will also be referenced as appropriate.

[0066] 12, the two-dimensional image acquirer 121 acquires a two-dimensional real-time image (DR_RT) generated by X-ray imaging of a three-dimensional volume (e.g., a body) 191. The manner in which the two-dimensional real-time image (DR_RT) is generated by X-ray imaging of the three-dimensional volume 191 may be similar to the manner in which the two-dimensional reference image (DR_ref) is generated by X-ray imaging of the three-dimensional volume 191, as described in the description of step 903 in FIG. 9. The two-dimensional image (digital X-ray image (DR)) generated by the X-ray image detector 195a or the X-ray image detector 195b is acquired by the two-dimensional image acquirer 121 operating in the system 102 (control system) and is used as the two-dimensional real-time image (DR_RT).

[0067] In step 1202 of FIG. 12, the surrogate tissue measurement acquirer 125 may acquire a surrogate tissue measurement value ("surrogate" in FIG. 11). The surrogate tissue measurement value is information generated by measuring the position, shape, and other conditions of a surrogate tissue (surrogate) included in the three-dimensional volume (e.g., the body) 191. The surrogate tissue measurement value is, for example, the position of the diaphragm, the position of the abdominal surface, and the amount of external air inhaled and exhaled in the three-dimensional volume 191 due to breathing. Here, information on the position of the abdominal surface may be acquired by a camera. Information on the amount of external air inhaled and exhaled in the three-dimensional volume 191 due to breathing may be acquired by a flow meter. (However, if the position of the diaphragm can be determined based on the above-mentioned two-dimensional real-time image (DR_RT) by a template matching technique, a fitting technique using a quadratic function, or the like, the information on the position of the diaphragm does not need to be acquired via the surrogate tissue measurement acquirer 125.) Note that in step 1203 described below, if the displacement / deformation setter 122 determines the "estimated content" of the displacement or deformation of each segment-corresponding three-dimensional image (3D_ori[*], 3D_ref[*], or 3D_other[*]) for each segment without using surrogate tissue measurements (surrogate), then this step 1202 does not need to be performed.

[0068] The series of steps from step 1203 to step 1211 in Figure 12 determine the displacement or deformation content ("estimated content") for each segment-corresponding three-dimensional image (3D_ori[*], 3D_ref[*], or 3D_other[*]) so that the similarity (similarity between images of the same modality) between the two-dimensional simulation image (DR_sim) and the two-dimensional real-time image (DR_RT) acquired in step 1201 is sufficiently high. Here, the two-dimensional simulation image (DR_sim) is generated (through the generation of a segment-corresponding two-dimensional model reconstruction image (DRR_model[*]), the generation of a segment-corresponding two-dimensional deformation vector field (2D-DVF_model[*]), and the generation of a segment-corresponding two-dimensional simulation image (DR_sim[*])) so as to correspond to the segment-corresponding three-dimensional model image (3D_model[*]), which is generated by displacing or deforming each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). Each step will be explained in turn below.

[0069] In step 1203 of FIG. 12, the displacement / deformation setter 122 determines the content of the displacement or deformation ("estimated content") to be performed on each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). When step 1203 is executed for the first time, the displacement / deformation setter 122 may determine the "estimated content" using one of the following methods. (1) The displacement / deformation setting unit 122 determines the "estimated content" so that a displacement or deformation reflecting a predetermined value is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). By appropriately determining the above-mentioned "predetermined value," various "estimated contents" can be generated appropriately. (2) The displacement / deformation setting unit 122 determines the “estimated content” so that a displacement or deformation reflecting a value based on a random number (random) is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). By using random numbers, various "estimated contents" can be generated appropriately. (3) The displacement / deformation setter 122 determines the "estimated content" so that the displacement or deformation reflecting the information of the two-dimensional real-time image (DR_RT) acquired in step 1201 is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). For example, the displacement / deformation setter 122 may extract information on the position of the diaphragm from the two-dimensional real-time image (DR_RT) and reflect the information on the position of the diaphragm in the "estimated content." (4) The displacement and deformation setter 122 determines the “estimated content” so that the displacement or deformation reflecting the measurement value (surrogate tissue measurement value (surrogate)) obtained from the surrogate tissue contained in the three-dimensional volume (e.g., the body) 191 is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). After step 1203, control transitions to step 1204.

[0070] In step 1204 of Fig. 12, the three-dimensional displacement transformer 123 displaces or transforms each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]) based on the model 129 so as to reflect the "estimated content" determined in step 1203. The three-dimensional displacement transformer 123 sets the results of the displacement or transformation process as each of the segment-corresponding three-dimensional model images (3D_model[*]) for each segment. After step 1204, control transitions to step 1205.

[0071] In step 1205 of Fig. 12, the reconstructed image generator 124 generates a digitally reconstructed X-ray image (DRR) based on each of the segment-corresponding three-dimensional model images (3D_model[*]) for each segment generated in step 1204. The reconstructed image generator 124 sets each of the segment-corresponding two-dimensional model reconstructed images (DRR_model[*]) for each segment as the digitally reconstructed X-ray image (DRR). The manner in which the reconstructed image generator 124 generates the digitally reconstructed X-ray image (DRR) in step 1205 may be similar to the manner in which the digitally reconstructed X-ray image (DRR) is generated as described using step 907 of Fig. 9. After step 1205, control transitions to step 1206.

[0072] 12, the two-dimensional deformation vector field generator 131 generates a segment-corresponding two-dimensional deformation vector field (2D-DVF_model[*]) for each segment. The two-dimensional deformation vector field generator 131 generates the two-dimensional deformation vector field (2D-DVF_model[i]) for each segment (where the segment identifier is "i") so that the two-dimensional deformation vector field represents a change (displacement or deformation on a two-dimensional plane) from the state represented by the segment-corresponding two-dimensional reference reconstructed image (DRR_ref[i]) to the state represented by the segment-corresponding two-dimensional model reconstructed image (DRR_model[i]). (This step 1206 may be understood as a kind of registration process (other than that shown in FIG. 9) between the segment-corresponding 2D reference reconstructed image (DRR_ref[*]) and the segment-corresponding 2D model reconstructed image (DRR_model[*]). Step 1206 may be realized by executing registration software (registration program).) After step 1206, control transitions to step 1207.

[0073] In step 1207 (two-dimensional displacement transformation step) of FIG. 12, the two-dimensional displacement transformer 132 generates a segment-corresponding two-dimensional simulation image (DR_sim[*]) for each segment. For each segment (where the segment identifier is "i"), the two-dimensional displacement transformer 132 generates a segment-corresponding two-dimensional simulation image (DR_sim[i]) by applying a displacement or deformation indicated by the segment-corresponding two-dimensional deformation vector field (2D-DVF_model[i]) to the segment-corresponding two-dimensional reference image (DR_ref[i]). After step 1207, control transitions to step 1208.

[0074] In step 1208 (two-dimensional simulation image synthesis step) of FIG. 12, the two-dimensional simulation image synthesizer 133 generates a two-dimensional simulation image (DR_sim) by synthesizing each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment. The two-dimensional simulation image synthesizer 133 may generate a two-dimensional simulation image (DR_sim) using, for example, the following linear combination equation (Equation 3): In the following linear combination equation (Equation 3), the pixel intensity (pixel value) of a pixel at a position (x, y) on the two-dimensional image I is denoted by pi(I, x, y), the segment identifier is denoted by "i", the coefficient adjustable for each segment (i) is denoted by "k[i]", and the sum related to the segment identifier i is denoted by "Σ". pi(DR_sim[i],x,y) = Σ(k[i]*pi(DR_sim[i],x,y)) = (k[1]*pi(DR_sim[1],x,y))+(k[2]*pi(DR_sim[2],x,y))+ (Formula 3) As the relationship between the two-dimensional reference image (DR_ref) and the segment-corresponding two-dimensional reference image (DR_ref[*]) for each segment is determined before radiation therapy, the coefficient "k[i]" for each segment in the above (Equation 3) may be determined in step 909 or step 911. After step 1208, control transitions to step 1209.

[0075] 12 (same-modality similarity calculation step), the same-modality similarity calculator 134 calculates the similarity (same-modality image similarity) between the two-dimensional real-time image (DR_RT) acquired in step 1201 and the two-dimensional simulation image (DR_sim) generated in step 1208. Here, the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim) have the same modality (originating from two-dimensional images (X-ray images) generated by X-ray imaging of the three-dimensional volume 191). The similarity (similarity between same-modality images) may be a value based on, for example, "pi(DR_RT,x,y)-pi(DR_sim,x,y)," which is the "difference (error)" of pixel intensity (pixel value) between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim). The same-modality similarity calculator 134 may be based on, for example, the sum of squared differences (SSD) based on the "difference (error)" for each pixel between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim), or the mean squared error (MSE) based on the "difference (error)." Here, the sum of squared differences (SSD) is the sum (for all pixels) of the squared values of the "difference (error)" for each pixel. The mean squared error (MSE) is calculated by dividing the sum of squared difference (SSD) by the number of pixels. The smaller the value of the sum of squared difference (SSD) or the sum of squared difference (SSD) (the closer it is to zero), the higher the similarity (similarity between images of the same modality) between the 2D real-time image (DR_RT) and the 2D simulated image (DR_sim). Instead of using the above-mentioned "difference (error)" for each pixel, or in addition to using the above-mentioned "difference (error)" for each pixel, the same-modality similarity calculator 134 may calculate the similarity (same-modality image similarity) between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim) by using, for example, mutual information between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim), or a two-dimensional deformation vector field that can be calculated between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim). After step 1209, control transitions to step 1210.

[0076] 12 (volume estimation validity determination step), the volume estimation validity determiner 136 determines the validity of the "estimation content" determined in step 1203. Specifically, the volume estimation validity determiner 136 determines whether the similarity (similarity between images of the same modality) is sufficiently high between the two-dimensional simulation image (DR_sim) obtained as a result of the displacement or deformation performed on each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]) and the two-dimensional real-time image (DR_RT) acquired in step 1201. For example, the volume estimation validity determiner 136 determines whether the similarity (similarity between images of the same modality) is equal to or greater than a predetermined threshold (or higher than a predetermined threshold). Because the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim) have the same modality, the similarity calculation in step 1209 can be expected to be performed appropriately for a wide variety of segments (e.g., parts, organs, tissues, targets (e.g., tumors), etc.) included in the three-dimensional volume (e.g., body) 191. Therefore, compared to step 909 in FIG. 9 , in step 1210, the system 102 (control system) may be less likely to receive input from an expert with knowledge or experience indicating a judgment regarding the validity of the “estimated content.” Considering that the time margin for executing the flowchart in FIG. 12 is (in many cases) shorter than the time margin for executing the flowchart in FIG. 9 , improving the degree of automation of the judgment regarding the validity of the “estimated content” in step 1210 brings operational benefits to the system 102 (control system) and the radiation therapy system 101. However, as a modified example, the volume estimation validity determiner 136 may present (display or output) the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim) to an expert with knowledge or experience, and may accept input indicating the expert's judgment regarding the validity of the "estimated content." Alternatively, the similarity (similarity between images of the same modality) calculated in step 1209 may be presented (displayed or output) to the expert. Although such a modified example increases the time required to judge the validity of the "estimated content," it is expected that the accuracy of the judgment of the validity of the "estimated content" will be improved (or that the expert will be more convinced). After step 1210, control transitions to step 1211.

[0077] 12, the volume estimation validity determiner 136 performs branching processing in control depending on the validity of the "estimation content" determined in step 1210. Specifically, if it is determined that the "estimation content" is valid (for example, if it is determined that the similarity (similarity between images of the same modality) is equal to or greater than a predetermined threshold (or higher than the predetermined threshold)), control transitions to step 1212. If the "estimated content" is determined to be invalid (for example, if the similarity (similarity between images of the same modality) is determined to be lower than a predetermined threshold (or equal to or less than the predetermined threshold)), control returns to step 1203. That is, the volume estimation validity determiner 136 instructs the displacement / deformation setter 122 to determine a new "estimated content" regarding the displacement or deformation of the three-dimensional volume 191 (each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*])). When control is returned to step 1203 (i.e., when step 1203 is executed for the second time or later), the displacement / deformation setter 122 determines new content (new "estimated content") as the content ("estimated content") of the displacement or deformation to be performed on each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). When step 1203 is executed for the second time or later, the displacement / deformation setter 122 may determine the "estimated content" using any of the following methods. (1) The displacement / deformation setter 122 determines new "estimated content" so that new displacements or deformations reflecting predetermined values (different from the values when step 1203 was last executed) are applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). By appropriately determining the above-mentioned "predetermined value," various "estimated contents" can be generated appropriately. (2) The displacement / deformation setter 122 determines a new “estimated content” so that the new displacement or deformation obtained by modifying the previous “estimated content” based on a predetermined value is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). The above-mentioned "predetermined value" is determined appropriately, and by using this "predetermined value" when determining the amount of change in the "estimated content," various "estimated content" can be generated appropriately. (3) The displacement / deformation setter 122 determines new "pre-set contents" so that new displacements or deformations reflecting values (random) based on random numbers (different from the last time step 1203 was executed) are applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). By using random numbers, various "estimated contents" can be generated appropriately. (4) The displacement / deformation setter 122 determines a new “estimated content” so that the new displacement or deformation obtained by modifying the previous “estimated content” to reflect a value based on a random number (random) is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). By using random numbers when determining the amount of change in the "estimated content," various "estimated content" can be generated appropriately. (5) The displacement / deformation setter 122 back projects the "difference" between the two-dimensional simulation image (DR_sim) based on the previous "estimated content" and the two-dimensional real-time image (DR_RT) acquired in step 1201 onto the three-dimensional space in which the three-dimensional volume 191 (each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*])) is located. The displacement / deformation setter 122 determines a new "estimated content" so that the new displacement or deformation obtained by modifying the previous "estimated content" to reflect the back projection results is applied to each of the segment-corresponding three-dimensional images (3D_ori[*], 3D_ref[*], or 3D_other[*]). The "difference" information between the two-dimensional simulation image (DR_sim) and the two-dimensional real-time image (DR_RT) may be derived from information calculated by the same modality similarity calculator 134, or may be "difference" information calculated separately by the same modality difference calculator 135. For example, the "difference" between a two-dimensional simulation image (DR_sim) and a two-dimensional real-time image (DR_RT) can be a quantity related to the mutual information between the two images, a two-dimensional deformation vector field when one image is displaced or deformed to the other, the sum of squared difference (SSD) based on the "difference (error)" of pixel intensity (pixel value) for each pixel between the two images, or the mean squared error (MSE) based on the "difference (error)". By defining a new "estimated content" that reflects the "difference" between two images (a two-dimensional simulation image (DR_sim) and a two-dimensional real-time image (DR_RT)) projected back onto the above three-dimensional space, it is expected that a new "estimated content" will be defined that will reduce the "difference."

[0078] It can be expected that the segment-corresponding three-dimensional model image (3D_model[*]) for each segment based on the "estimated content" at the time of reaching step 1212 in Fig. 12 is appropriate as an estimate (tracking) of the displacement or deformation of each segment included in the three-dimensional volume 191. (At the time of reaching this step 1212, it may be considered that a certain kind of registration process (different from that shown in Fig. 9 or that shown in step 1206) between the two-dimensional simulation image (DR_sim) and the two-dimensional real-time image (DR_RT) has been successfully performed.) Step 1212 may be performed if the results of estimating (tracking) the displacement or deformation of each of the segments included in the three-dimensional volume 191 are to be utilized for the radiation therapy system 101 . In step 1212 (therapeutic beam irradiation control step) of FIG. 12 , the therapeutic beam irradiation controller 137 controls irradiation of a therapeutic beam (e.g., a charged particle beam) 198 to a target (e.g., a tumor) 192 included in a three-dimensional volume (e.g., a body) 191. The therapeutic beam irradiation controller 137 may provide control information to, for example, a therapy controller, an accelerator controller, and a beam transport system controller. Based on the control information, the therapy controller may control various devices (including the gantry 196) included in the radiation therapy room system 109. Based on the control information, the accelerator controller and the beam transport system controller may control the accelerator 107 and the beam transport system 108, respectively. Alternatively, the system 102 (control system) having the therapeutic beam irradiation controller 137 may directly control part or all of the accelerator 107, the beam transport system 108, or the radiation therapy room system 109. When the therapeutic beam irradiation controller 137 performs control to irradiate the target 192 included in the three-dimensional volume 191 with the therapeutic beam 198, the result of estimation (tracking) of the displacement or deformation of each of the segments included in the three-dimensional volume 191, which is determined by the "estimation content," is utilized. In other words, since appropriate estimation (tracking) of the positions and shapes of the segments included in the three-dimensional volume 191 is realized, irradiation of the therapeutic beam 198 to the target 192 can also be performed appropriately.

[0079] As described above, the system 102 (control system) executes processing during radiation therapy, and therefore, in accordance with the "estimated content" of the internal displacement or deformation of the three-dimensional volume (e.g., the body) 191, the system 102 (control system) can sequentially generate segment-corresponding three-dimensional model images (3D_model[*]) for each segment included in the three-dimensional volume 191, generate segment-corresponding two-dimensional model reconstruction images (DRR_model[*]) for each segment, generate segment-corresponding two-dimensional deformation vector fields (2D-DVF_model[*]) for each segment, and generate segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment, and finally generate a two-dimensional simulation image (DR_sim[*]). In other words, the system 102 (control system) can generate a two-dimensional simulation image (DR_sim[*]) according to the "estimated content" of the internal displacement or deformation of the three-dimensional volume 191. The system 102 (control system) then separately acquires a two-dimensional real-time image (DR_RT) generated by X-ray imaging of the three-dimensional volume 191, and compares the two-dimensional real-time image (DR_RT) with a two-dimensional simulation image (DR_sim[*]) corresponding to the "estimated content." Based on the comparison result, the system 102 (control system) can determine whether the "estimated content" is valid. In this way, even if real-time three-dimensional imaging of the three-dimensional volume 191 is not performed during radiation therapy, the system 102 (control system) can properly estimate (track) displacements or deformations within the three-dimensional volume 191.

[0080] 6. Other (variations) The present disclosure is not limited to the above-described embodiments and includes various modifications. Part of the configurations and processes of the embodiments may be replaced with the configurations and processes of other conceivable embodiments. The configurations and processes of other conceivable embodiments may be added to the configurations and processes of the embodiments. For example, the present disclosure may include the following modified embodiments.

[0081] (α) Search for the most similar content among multiple presets and estimated contents The flowcharts in Figures 9 and 12 above show the process of determining a "preset" or "estimate" for the displacement or deformation of a three-dimensional volume, and then determining a new "preset" or "estimate" if the previously determined "preset" or "estimate" is found to be inappropriate. This method makes it possible to exploratory find appropriate "preset" or "estimate." In a modified example, a plurality of "preset contents" or "estimated contents" for displacement or deformation of a three-dimensional volume are prepared. Then, in the modified example, the volume preset validity determiner 155 or the volume estimation validity determiner 136 may select the most appropriate (most valid), the most valid, or at least the most suitable from the plurality of prepared "preset contents" or "estimated contents" using similarity (similarity between images of different modalities or similarity between images of the same modality) or the like as a basis for determination. With such a modified example, it is expected that the time required to find suitable "pre-set contents" or "estimated contents" will be stable.

[0082] (β) Omission of displacement or deformation of three-dimensional images before radiation therapy In the above embodiment, a three-dimensional reference image (3D_ref) or a segment-corresponding three-dimensional reference image (3D_ref[*]) was obtained by displacing or deforming a three-dimensional original image (3D_ori) or a segment-corresponding three-dimensional original image (3D_ori[*]). A modified example deals with the case where it is known that the inter-modality image similarity between the two-dimensional reference reconstructed image (DRR_ref) and the two-dimensional reference image (DR_ref) is sufficiently high without performing the above-described displacement or deformation of the three-dimensional image. In this modified example, the three-dimensional original image (3D_ori) or the segment-corresponding three-dimensional original image (3D_ori[*]) may be used as the three-dimensional reference image (3D_ref) or the segment-corresponding three-dimensional reference image (3D_ref[*]) as is. Such a modification can simplify the pre-radiation treatment process.

[0083] (γ) Uses other than radiotherapy The above embodiments utilize the results of real-time estimation (tracking) of displacements or deformations inside a three-dimensional volume for radiation therapy. A variant may utilize the results of real-time estimation (tracking) of displacements or deformations within a three-dimensional volume for broader applications than just radiation therapy, for example, procedures performed on a three-dimensional volume that do not involve radiation therapy. Thus, the real-time estimation (tracking) of displacements or deformations within a three-dimensional volume according to the present disclosure can be utilized in a wide range of applications.

[0084] The technical matters shown in the above-described embodiments of the present disclosure and the modified examples of the embodiments can be combined as appropriate as long as no technical contradiction occurs.

Claims

1. 1. A system comprising: a two-dimensional image acquirer for acquiring two-dimensional real-time images (DR_RT) generated by radiographing a three-dimensional volume, the two-dimensional real-time images (DR_RT) having a predetermined modality; and a two-dimensional displacement transformer that displaces or transforms each of segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment included in the three-dimensional volume to correspond to an estimated content of the displacement or deformation of the three-dimensional volume, to generate each of segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment included in the three-dimensional volume, wherein the segment-corresponding two-dimensional reference images (DR_ref[*]) and the segment-corresponding two-dimensional simulation images (DR_sim[*]) have the predetermined modality; a two-dimensional simulation image synthesizer that synthesizes each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) to generate a two-dimensional simulation image (DR_sim), wherein the two-dimensional simulation image (DR_sim) has the predetermined modality; a same-modality similarity calculator that calculates a similarity between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim); The system includes a volume estimation validity determiner that determines the validity of the estimated content based on the similarity.

2. 10. The system of claim 1, The system comprises: a displacement / deformation setting unit that determines estimated displacement or deformation of the three-dimensional volume; a three-dimensional displacement and transformation unit that displaces or transforms each of the segment-corresponding three-dimensional images for each segment included in the three-dimensional volume based on a model so as to reflect the estimated content, and generates each of the segment-corresponding three-dimensional model images (3D_model[*]) for each segment included in the three-dimensional volume; a reconstructed image generator that generates, as digitally reconstructed X-ray images, segment-corresponding two-dimensional model reconstructed images (DRR_model[*]) for each segment included in the three-dimensional volume based on the segment-corresponding three-dimensional model images (3D_model[*]); a two-dimensional deformation vector field generator that generates a segment-corresponding two-dimensional deformation vector field (2D-DVF_model[*]) for each segment included in the three-dimensional volume based on each segment-corresponding two-dimensional reference reconstruction image (DRR_ref[*]) for each segment included in the three-dimensional volume and each segment-corresponding two-dimensional model reconstruction image (DRR_model[*]); The system, wherein the two-dimensional displacement transformer generates each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) by displacing or transforming each of the segment-corresponding two-dimensional reference images (DR_ref[*]) using each of the segment-corresponding two-dimensional deformation vector fields (2D-DVF_model[*]).

3. 3. The system of claim 2, The displacement and deformation setting unit is capable of determining the estimation content based on information about a surrogate tissue contained in the three-dimensional volume, a predetermined value, or a random number.

4. 3. The system of claim 2, the volume estimation validity determiner instructs the displacement / deformation setter to determine new estimation details regarding the displacement or deformation of the three-dimensional volume when the estimation details are invalid; The system comprises: a same-modality difference calculator that generates information about the difference between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim); The displacement and transformation setter determines the new estimated content based on information about the difference projected back onto a three-dimensional space that includes the three-dimensional volume.

5. 3. The system of claim 2, the volume estimation validity determiner instructs the displacement / deformation setter to determine new estimation details regarding the displacement or deformation of the three-dimensional volume when the estimation details are invalid; The three-dimensional displacement transformer determines the new estimated content based on a random number.

6. 10. The system of claim 1, The volume estimation validity determiner selects an estimation content with high validity from among the plurality of estimation contents.

7. 3. The system of claim 2, The system comprises: a model generator for generating the model, the model generator generating the model based on three-dimensional images generated by three-dimensionally capturing the three-dimensional volume at different time periods or different respiratory phases, or based on information in a database relating to the displacement or deformation of each segment included in the three-dimensional volume; The system, wherein the model represents the displacement or deformation of each of the segments included in the three-dimensional volume.

8. 3. The system of claim 2, the two-dimensional image acquirer acquires a two-dimensional reference image (DR_ref) generated by X-ray imaging of the three-dimensional volume; the two-dimensional reference image (DR_ref) has the predetermined modality; the reconstruction image generator generates a two-dimensional reference reconstruction image (DRR_ref) as a digitally reconstructed X-ray image based on the three-dimensional reference image (3D_ref); the reconstructed image generator generates each of the segment-corresponding two-dimensional reference reconstructed images (DRR_ref[*]) as a digitally reconstructed X-ray image based on each of the segment-corresponding three-dimensional reference images (3D_ref[*]); each of the segment-corresponding three-dimensional reference images (3D_ref[*]) is a portion of the three-dimensional reference image (3D_ref) corresponding to each of the segments included in the three-dimensional volume; The system comprises: The method includes a segment-corresponding two-dimensional reference image generator that generates the two-dimensional reference reconstruction image (DRR_ref), the segment-corresponding two-dimensional reference reconstruction image (DRR_ref[*]), and each of the segment-corresponding two-dimensional reference images (DR_ref[*]) based on the two-dimensional reference image (DR_ref), The system, wherein each of the segment-corresponding two-dimensional reference images (DR_ref[*]) corresponds to a respective segment included in the three-dimensional volume.

9. 9. The system of claim 8, The system comprises: a three-dimensional image acquirer for acquiring a three-dimensional original image (3D_ori) generated by three-dimensionally photographing the three-dimensional volume, the three-dimensional original image (3D_ori) having a modality different from the predetermined modality; a three-dimensional segment generator that divides the three-dimensional original image (3D_ori) into segment-corresponding three-dimensional original images (3D_ori[*]) for each segment included in the three-dimensional volume; the displacement / deformation setting unit determines a preset content of a displacement or deformation of the three-dimensional volume; the three-dimensional displacement transformer generates the three-dimensional reference image (3D_ref) by displacing or transforming the three-dimensional original image (3D_ori) based on the model so as to reflect the preset content; the three-dimensional displacement transformer generates the segment-corresponding three-dimensional reference image (3D_ref[*]) by displacing or transforming the segment-corresponding three-dimensional original image (3D_ori[*]) based on the model so as to reflect the preset content; The system comprises: a cross-modality similarity calculator that calculates a cross-modality image similarity between the two-dimensional reference image (DR_ref) and the two-dimensional reference reconstructed image (DRR_ref); The system further comprises a volume preset validity determiner that determines the validity of the preset content based on the similarity between the different modality images.

10. 10. The system of claim 9, The three-dimensional segment generator divides the three-dimensional original image (3D_ori) into each of the segment-corresponding three-dimensional original images (3D_ori[*]) based on the velocity due to movement of each voxel contained in the three-dimensional original image (3D_ori), the intensity of each voxel in the three-dimensional original image (3D_ori), or empirical knowledge regarding the position of each segment contained in the three-dimensional volume.

11. 1. A radiation therapy system comprising: The radiation therapy system includes the system according to claim 1, an accelerator that emits a treatment beam, and a treatment beam irradiator that irradiates the treatment beam to a target included in the three-dimensional volume, The system comprises: A radiation therapy system having a treatment beam irradiation controller that controls the treatment beam irradiator to irradiate a treatment beam to a target included in the three-dimensional volume based on the estimated displacement or deformation of the three-dimensional volume.

12. A method performed by a system, comprising: a two-dimensional real-time image acquisition step for acquiring a two-dimensional real-time image (DR_RT) generated by X-raying a three-dimensional volume, the two-dimensional real-time image (DR_RT) having a predetermined modality; a two-dimensional displacement and deformation step of displacing or deforming each of segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment included in the three-dimensional volume to correspond to the estimated displacement or deformation of the three-dimensional volume, thereby generating each of segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment included in the three-dimensional volume, wherein the segment-corresponding two-dimensional reference images (DR_ref[*]) and the segment-corresponding two-dimensional simulation images (DR_sim[*]) have the predetermined modality; a two-dimensional simulation image synthesis step of synthesizing each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) to generate a two-dimensional simulation image (DR_sim), wherein the two-dimensional simulation image (DR_sim) has the predetermined modality; a same-modality similarity calculation step of calculating a similarity between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim); The method further comprises a volume estimation validity determination step of determining the validity of the estimated content based on the similarity.

13. 13. The method of claim 12, A method comprising a treatment beam irradiation control step of controlling irradiation of a treatment beam to a target included in the three-dimensional volume based on the estimated displacement or deformation of the three-dimensional volume.

14. A program, To the system, a two-dimensional real-time image acquisition step for acquiring a two-dimensional real-time image (DR_RT) generated by X-raying a three-dimensional volume, the two-dimensional real-time image (DR_RT) having a predetermined modality; a two-dimensional displacement and deformation step of displacing or deforming each of segment-corresponding two-dimensional reference images (DR_ref[*]) for each segment included in the three-dimensional volume to correspond to the estimated displacement or deformation of the three-dimensional volume, thereby generating each of segment-corresponding two-dimensional simulation images (DR_sim[*]) for each segment included in the three-dimensional volume, wherein the segment-corresponding two-dimensional reference images (DR_ref[*]) and the segment-corresponding two-dimensional simulation images (DR_sim[*]) have the predetermined modality; a two-dimensional simulation image synthesis step of synthesizing each of the segment-corresponding two-dimensional simulation images (DR_sim[*]) to generate a two-dimensional simulation image (DR_sim), wherein the two-dimensional simulation image (DR_sim) has the predetermined modality; a same-modality similarity calculation step of calculating a similarity between the two-dimensional real-time image (DR_RT) and the two-dimensional simulation image (DR_sim); A program for executing a volume estimation validity determination step of determining the validity of the estimation content based on the similarity.

15. 15. The program according to claim 14, In the system, A program for executing a treatment beam irradiation control step that controls irradiation of a treatment beam to a target included in the three-dimensional volume based on the estimated content of the displacement or deformation of the three-dimensional volume.

Images