Image processing apparatus and image processing method

Abstract

To provide an image processor capable of coping with the change of an anatomical structure by using a three dimensional volume image.SOLUTION: The volume estimating unit 24 acquires an estimated three dimensional volume image obtained by estimating the three dimensional volume of the patient at the pretreatment stage based on the three dimensional volume image at the treatment planning stage acquired by the volume acquiring unit 21 and the first and second proxy operations acquired by the proxy operation acquiring unit 23. A generation part including a projection part 25 and an optimization part 26 generates an optimized three dimensional volume image obtained by deforming the estimated three dimensional volume image according to the two dimensional perspective image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and an image processing method. [Background technology]

[0002] In radiation therapy, it is important to obtain structural information that accurately represents a patient's anatomical structures and to accurately irradiate targets such as tumors with radiation based on that structural information. Fan-beam planning CT images, which are three-dimensional volume images of the patient, are typically acquired using fan-beam CT (Computed Tomography), and treatment plans for radiation therapy are created based on these fan-beam planning CT images. However, because radiation therapy is usually performed several days after the treatment plan is created, the patient's anatomical structures may have changed since the treatment plan was created, making it difficult to accurately deliver radiation.

[0003] In response to this, adaptive treatment, which takes into account structural changes in the patient's body, has recently attracted attention. In adaptive treatment, new structural information about the patient is acquired on the day of treatment, and the treatment plan is modified based on this new structural information. To reduce the burden on the patient, the structural information acquired on the day of treatment is typically a series of 2D CT images acquired using cone-beam computed tomography (CBCT) or four-dimensional cone-beam CT (4DCBCT) instead of fan-beam planned CT images. However, these same-day CT images have the disadvantage of lower image quality compared to fan-beam planned CT images. Furthermore, when the patient is performing movements such as breathing, motion artifacts can occur when the CBCT projection data is reconstructed to create same-day CT images.

[0004] For this reason, attention has been focused on using detailed fan-beam planned CT images for positioning patients in the treatment room on the day of treatment. For example, Patent Document 1 discloses a technique for positioning a patient by comparing a fluoroscopic image acquired in the treatment room with a DRR (Digitally Reconstructed Radiograph) image generated from the fan-beam planned CT image. Furthermore, Patent Document 2 discloses a technique for comparing a fluoroscopic image with a DRR image generated from the fan-beam planned CT image and modifying the fan-beam planned CT image to match the patient's condition on that day. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-180910 [Patent Document 2] U.S. Patent No. 10,580,147 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when a patient is performing an action such as breathing, a change occurs in the patient's anatomical structure, and the techniques described in Patent Documents 1 and 2 have the problem that they cannot handle such a case.

[0007] An object of the present disclosure is to provide an image processing device and an image processing method that can also accommodate changes in anatomical structures using three-dimensional volume images. [Means for solving the problem]

[0008] An image processing device according to one aspect of the present disclosure includes a volume acquisition unit that acquires a three-dimensional volume image showing changes over time in the anatomical structure of a subject in a first stage; an image acquisition unit that acquires a plurality of two-dimensional perspective images of the subject from each of a plurality of imaging angles in a second stage that is later than the first stage; a proxy action acquisition unit that acquires a first proxy action that is the action of a surrogate placed on the body surface of the subject in the first stage and a second proxy action that is the action of a surrogate placed on the body surface of the subject in the second stage; an estimation unit that acquires an estimated three-dimensional volume image that estimates the three-dimensional volume image showing changes over time in the anatomical structure of the subject in the second stage based on the three-dimensional volume image, the first proxy action, and the second proxy action; and a generation unit that generates an optimized three-dimensional volume image by deforming the estimated three-dimensional volume image according to the two-dimensional perspective image. [Effects of the Invention]

[0009] According to the present invention, it is possible to deal with changes in anatomical structures using three-dimensional volume images. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an image processing system according to a first embodiment of the present disclosure. [Figure 2] FIG. 1 is a front view showing an example of a three-dimensional volume imaging device. [Figure 3] 1 is a side view showing an example of a three-dimensional volume imaging device. [Figure 4] FIG. 1 is a front view showing an example of an image capturing device. [Figure 5] FIG. 1 is a side view illustrating an example of an image capturing device. [Figure 6] 10 is a flowchart illustrating an example of an operation of the image processing device. [Figure 7] 4 is a flowchart for explaining the minimization process of the first embodiment in more detail. [Figure 8]10 is a flowchart for explaining the minimization process of the second embodiment in more detail. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] Fig. 1 is a diagram showing an image processing system according to a first embodiment of the present disclosure. The image processing system 1 shown in Fig. 1 is a system for performing image processing on images used in radiation therapy, which treats an affected area of ​​a patient by irradiating the affected area with radiation, and includes a three-dimensional volumetric imaging device 11, an image imaging device 12, and an image processing device 13.

[0013] The three-dimensional volumetric imaging device 11 is an imaging device for acquiring three-dimensional volumetric images showing temporal changes in the anatomical structure of a patient, which is the subject. In this embodiment, the three-dimensional volumetric imaging device 11 is a fan-beam CT device, and images the patient in the treatment planning stage, which is the first stage in creating a treatment plan for radiation therapy for the patient. The three-dimensional volumetric images are treatment planning images used to create the treatment plan. The patient's anatomical structures include, for example, tumors and organs, and their positions change due to movements such as respiratory movement.

[0014] The imaging device 12 is an imaging device for acquiring multiple 2D fluoroscopic images of the patient's anatomical structure while changing the imaging angle. In this embodiment, the imaging device 12 is a cone beam CT device, and images the patient in the pretreatment stage, which is the second stage of pretreatment for radiation therapy based on a treatment plan. The pretreatment stage is usually performed immediately before the start of radiation therapy on the treatment day, which is several days after the treatment planning stage. The 2D fluoroscopic images are positioning images for positioning the patient in pretreatment.

[0015] FIG. 2 is a front view showing an example of the three-dimensional volume imaging device 11, and FIG. 3 is a side view showing an example of the three-dimensional volume imaging device 11. As shown in FIG.

[0016] The three-dimensional volume imaging device 11 shown in FIGS. 2 and 3 is a fan beam CT device that images a patient 51 placed on a treatment table 50a, and includes an X-ray source 111a and an X-ray detector 112a.

[0017] The X-ray source 111a irradiates X-rays as radiation for imaging in a fan beam format toward a patient 51 placed on a treatment couch 50a. The X-ray detector 112a detects the X-rays irradiated from the X-ray source 111a through the patient 51.

[0018] The treatment couch 50a moves in a predetermined direction (direction X indicated by an arrow in FIG. 3). The X-ray source 111a is installed so as to be rotatable around the patient 51 about the X-axis, and multiple X-ray detectors 112a are installed around the patient 51, or one is installed so as to be rotatable around the patient 51 in the same manner as the X-ray source 111a.

[0019] The three-dimensional volume imaging device 11 also includes a surrogate motion acquisition device 15 that acquires the motion of a surrogate (substitute target) 151 placed on the body surface of the patient 51. The surrogate motion acquisition device 15 has a motion tracking camera 152 that tracks the motion of the surrogate 151.

[0020] FIG. 4 is a front view showing an example of the image capturing device 12, and FIG. 5 is a side view showing an example of the image capturing device 12. As shown in FIG.

[0021] The image capturing device 12 shown in FIGS. 4 and 5 is a cone beam CT device that captures an image of a patient 51 placed on a treatment table 50b, and includes an X-ray source 111b and an X-ray detector 112b.

[0022] The X-ray source 111a irradiates the patient 51 placed on the treatment couch 50b with X-rays in a cone beam form as radiation for imaging. The X-ray detector 112b detects the X-rays irradiated from the X-ray source 111b through the patient 51.

[0023] The X-ray source 111b is installed so as to be rotatable around the patient 51, and multiple X-ray detectors 112b are installed around the patient 51, or one X-ray detector 112b is installed so as to be rotatable around the patient 51 in the same manner as the X-ray source 111b.

[0024] The image capturing device 12 also includes a surrogate motion acquiring device 16 that acquires the motion of a surrogate 161 placed on the body surface of the patient 51. The surrogate motion acquiring device 16 has a motion tracking camera 162b that tracks the motion of the surrogate 161.

[0025] Returning to the explanation of FIG. 1, the image processing device 13 is configured, for example, by a computer system including a processor (computer) and memory (neither of which is shown). In this case, each component and function of the image processing device 13 described below is realized, for example, by the processor reading a program and executing the read program. The program can be recorded on a computer-readable recording medium such as memory. The image processing device 13 may also be connected to an input device that receives various information from a user who uses the image processing device 13, an output device that outputs various information to the user, and a network interface that transmits and receives various information via a communication network such as the Internet.

[0026] The image processing device 13 has, as functional components, a volume acquisition unit 21, an image acquisition unit 22, a proxy action acquisition unit 23, a volume estimation unit 24, a projection unit 25, and an optimization unit 26.

[0027] The volume acquisition unit 21 acquires a three-dimensional volume image of the patient 51 based on the imaging data (projection data) obtained by the three-dimensional volume imaging device 11.

[0028] The image acquisition unit 22 acquires a plurality of two-dimensional perspective images of the patient 51 taken from a plurality of imaging angles based on the imaging data obtained by the image capturing device 12.

[0029] The proxy action acquisition unit 23 acquires the actions of the surrogates 151 and 161 based on imaging data from the surrogate action acquisition devices 15 and 16 mounted on the three-dimensional volume imaging device 11 and the image imaging device 12, respectively. Hereinafter, the action of the surrogate 151 may be referred to as a first proxy action, and the action of the surrogate 161 may be referred to as a second proxy action.

[0030] The volume estimation unit 24 acquires an estimated three-dimensional volume image that estimates a three-dimensional volume image of the patient in the pre-treatment stage based on the three-dimensional volume image acquired by the volume acquisition unit 21 and the first proxy action and second proxy action acquired by the proxy action acquisition unit 23. In this embodiment, the volume estimation unit 24 generates a motion model that estimates a three-dimensional volume image from the action of the surrogate based on the three-dimensional volume image and the first proxy action, and estimates an estimated three-dimensional volume by inputting the proxy action into the motion model.

[0031] The projection unit 25 and optimization unit 26 constitute a generation unit that generates an optimized three-dimensional volume by transforming the estimated three-dimensional volume acquired by the volume estimation unit 24 according to the two-dimensional perspective image acquired by the image acquisition unit 22.

[0032] The projection unit 25 generates a DRR image, which is a two-dimensional projection image obtained by projecting the estimated three-dimensional volume onto a virtual plane corresponding to each imaging angle of the two-dimensional perspective image.

[0033] The optimization unit 26 generates an optimized three-dimensional volume image by transforming the estimated three-dimensional volume so that the two-dimensional perspective image and the DRR image coincide with each other.

[0034] 6 is a flowchart for explaining an example of the operation of the image processing device 13. The operation of the image processing device 13 includes a treatment planning stage 301 in which a treatment plan for radiation therapy is created, and a pretreatment stage 302 on the treatment day of radiation therapy.

[0035] First, in the treatment planning stage 301, the volume acquisition unit 21 of the image processing device 13 acquires a three-dimensional volume image of the patient 51 based on the imaging data obtained by the three-dimensional volume imaging device 11 (step S101). Note that the patient is engaged in movements such as breathing, and the three-dimensional volume image changes over time in response to these movements.

[0036] Furthermore, the surrogate action acquisition unit 23 acquires a first surrogate action s, which is the action of the surrogate 151 placed on the patient 51, based on the imaging data acquired by the surrogate action acquisition device 15 mounted on the image processing device 13. a (t) is acquired (step S102).

[0037] The proxy action acquisition unit 23 acquires the three-dimensional volume image and the first proxy action s a Based on (t), a motion model is generated to estimate a three-dimensional volume image from the motion of the surrogate (step S103).

[0038] The motion model is created using a statistical method such as principal component analysis, which generalizes the motion of the patient's tissue into eigenvectors, and in this embodiment, is expressed by the following equation 1.

number

[0039] Next, in the pretreatment stage 302, the image acquisition unit 22 acquires a plurality of 2D fluoroscopic images at different imaging angles based on imaging data from the imaging device 12 (step S201). Here, the imaging device 12 images the patient while changing the imaging angle. That is, the imaging angle is expressed as a function of time θ(t).

[0040] The proxy action acquisition unit 23 acquires a second proxy action s, which is the action of the surrogate 161, based on the imaging data of the surrogate action acquisition device 16 mounted on the image capturing device 12. b (t) is acquired (step S202).

[0041] The proxy action acquisition unit 23 acquires the second proxy action s b (t) is input to the motion model to obtain the second surrogate motion s b The estimated 3D volume [s b (t)] is acquired (step S203).

[0042] The projection unit 25 projects an estimated three-dimensional volume [s b (t)] onto a virtual plane according to the imaging angle θ(t) of the two-dimensional perspective image to generate a DRR image (step S204).

[0043] The optimization unit 26 optimizes the three-dimensional volume image [s b (t)] is transformed to generate an optimized three-dimensional volume image (step S205), and the process ends.

[0044] FIG. 7 is a flowchart for explaining in more detail the minimization process (the processes of steps S203 to S206) for generating an optimized volume in the process described with reference to FIG.

[0045] In the optimization process, first, the proxy action acquisition unit 23 acquires the second proxy action s b (t) is input to the motion model to obtain the 3D volume image of the patient during treatment [sb (t)] is generated (step S401).

[0046] The projection unit 25 projects an estimated three-dimensional volume [s b (t)] onto a virtual plane according to the imaging angle θ(t) of the two-dimensional perspective image to generate a DRR image (step S402).

[0047] The optimization unit 26 calculates the similarity between the 2D perspective image and the DRR image (step S403). The type of similarity is not particularly limited, and it may be a value indicating the degree of similarity between images, such as mutual information. Furthermore, the similarity between the 2D perspective image and the DRR image may be a statistical value (such as an average value) of individual similarities, which are the similarities between the 2D perspective image and the DRR projection image for each imaging angle θ(t).

[0048] The optimization unit 26 determines whether the similarity is greater than a threshold value (step S403).

[0049] If the similarity is equal to or smaller than the threshold (step S404: No), the optimization unit 26 optimizes the three-dimensional volume image [s b (t)] is transformed to generate an optimized three-dimensional volume image (step S405), and the process returns to step S403.

[0050] In this embodiment, the three-dimensional volume image [s b The deformation of the 3D volume image [s (t)] is performed by deforming the motion model so that the 2D perspective image and the DRR projection image coincide with each other, and then inputting the second proxy motion to the optimized motion model, which is the deformed motion model. b (t)] and the motion model are both optimized. For example, the 3D volume image [s b (t)] is a function u of the motion model (expressed in Equation 1). i (t)) product E u E s -1 This is done by optimizing parameters such as:

[0051] The 3D volumetric images and motion models can also be optimized using a deformation field with optimizable parameters. The deformation field is realized as an affine transformation matrix that applies to the organ to change its shape, such as scaling, rotation, or translation. This type of matrix can be optimized by mapping the gradient of a function between the matrix parameters and a similarity index. The matrix parameters or deformation field can also be randomly generated.

[0052] On the other hand, if the similarity is greater than the threshold value (step S404: Yes), the optimization unit 26 outputs the three-dimensional volume image at that time as an optimized three-dimensional volume image (step S406), and ends the process.

[0053] As described above, according to this embodiment, the volume estimation unit 24 acquires an estimated 3D volume image that estimates the 3D volume of the patient in the pre-treatment stage based on the 3D volume image of the treatment planning stage acquired by the volume acquisition unit 21 and the first and second proxy actions acquired by the proxy action acquisition unit 23. The generation unit, which includes the projection unit 25 and the optimization unit 26, generates an optimized 3D volume image by transforming the estimated 3D volume image in accordance with the 2D perspective image. Therefore, it is possible to generate an optimized 3D volume image that takes into account the second proxy action, which is a proxy action in the pre-treatment stage, and it is also possible to respond to changes in the anatomical structure using the 3D volume image.

[0054] In this embodiment, the projection unit 25 generates a DRR image by projecting the estimated 3D volume image onto a virtual plane corresponding to each imaging angle of the 2D perspective image. The optimization unit 26 generates an optimized 3D volume image by deforming the estimated 3D volume image so that the 2D perspective image and the DRR image coincide with each other. In this case, it is possible to appropriately generate the optimized 3D volume image.

[0055] In this embodiment, the volume estimation unit 24 generates a motion model that estimates the 3D volume image from the motion of the surrogate based on the 3D volume image and the first proxy motion, and obtains the estimated 3D volume image by inputting the second proxy motion to the motion model. In this case, it is possible to appropriately generate the estimated 3D volume image.

[0056] In this embodiment, the generator acquires an estimated 3D volume image by inputting the second substitute motion to an optimized motion model in which the parameters of the motion model are adjusted so that the 2D perspective image and the DRR projection image coincide with each other. In this case, it is also possible to optimize the motion model.

[0057] Next, a second embodiment will be described. In this embodiment, an example will be described in which a motion model is not generated in the treatment planning stage 301. In this case, the processing of step S103 is skipped.

[0058] FIG. 8 is a flowchart illustrating an example of the optimization process according to the second embodiment.

[0059] In the optimization process, the volume estimation unit 24 performs a first proxy operation s a (t) and the second proxy action s b (t) and the second substitute action s b At each time point (t), the second representative action s b (t) and the closest first proxy action s a A three-dimensional volume image corresponding to (t) is acquired as an estimated three-dimensional volume image (step S501).

[0060] The subsequent processing is the same as steps S402 to S406. However, in the processing of step S405, deformation of the motion model is omitted. However, if patient motion estimation is required in radiation therapy, the optimized 3D volume image and the second substitute motion s b Based on (t) and (t), a motion model may be constructed.

[0061] As described above, according to this embodiment, it is possible to deal with changes in anatomical structures using three-dimensional volume images without creating a motion model.

[0062] Furthermore, in this embodiment, when patient motion estimation is required in radiation therapy, a motion model is constructed, making it possible to more appropriately respond to changes in anatomical structures using three-dimensional volume images.

[0063] The above-described embodiments of the present disclosure are merely illustrative examples of the present disclosure, and are not intended to limit the scope of the present disclosure to these embodiments alone. Those skilled in the art may implement the present disclosure in various other forms without departing from the scope of the present disclosure. [Explanation of symbols]

[0064] 1: Image processing system 11: 3D volume imaging device 12: Image imaging device 13: Image processing device 15: Surrogate motion acquisition device 16: Surrogate motion acquisition device 21: Volume acquisition unit 22: Image acquisition unit 23: Surrogate motion acquisition unit 24: Volume estimation unit 25: Projection unit 26: Optimization unit

Claims

1. a volume acquisition unit that acquires a three-dimensional volume image showing a time change of an anatomical structure of a subject in a first stage; an image acquisition unit that acquires a plurality of two-dimensional perspective images of the subject from a plurality of imaging angles in a second stage subsequent to the first stage; a proxy action acquisition unit that acquires a first proxy action that is a motion of a surrogate placed on the body surface of the subject in the first stage and a second proxy action that is a motion of a surrogate placed on the body surface of the subject in the second stage; an estimation unit that acquires an estimated three-dimensional volume image by estimating a three-dimensional volume image showing a time change of the anatomical structure of the subject in the second stage based on the three-dimensional volume image, the first proxy action, and the second proxy action; a generating unit that generates an optimized three-dimensional volume image by transforming the estimated three-dimensional volume image in accordance with the two-dimensional perspective image.

2. The generation unit a projection unit that generates a two-dimensional projected image by projecting the estimated three-dimensional volume image onto a virtual surface corresponding to each imaging angle of the two-dimensional perspective image; 2. The image processing apparatus according to claim 1, further comprising: an optimization unit that generates the optimized three-dimensional volume image by deforming the estimated three-dimensional volume image so that the two-dimensional perspective image and the two-dimensional projection image coincide with each other.

3. The image processing device according to claim 2 , wherein the optimization unit deforms the estimated three-dimensional volume image using a deformation field.

4. The image processing device according to claim 1, wherein the estimation unit generates a motion model that estimates the three-dimensional volume image from the motion of the surrogate based on the three-dimensional volume image and the first proxy motion, and obtains the estimated three-dimensional volume image by inputting the second proxy motion into the motion model.

5. The generation unit a projection unit that generates a two-dimensional projected image by projecting the estimated three-dimensional volume image onto a virtual surface corresponding to each imaging angle of the two-dimensional perspective image; 5. The image processing device according to claim 4, further comprising: an optimization unit that acquires the estimated three-dimensional volume image by inputting the second proxy movement into an optimized motion model in which parameters of the motion model are adjusted so that the two-dimensional perspective image and the two-dimensional projection image coincide with each other.

6. The image processing device according to claim 1 , wherein the estimation unit acquires, at each time point of the second proxy action, the three-dimensional volume image corresponding to the time point of the first proxy action that is closest to the second proxy action as the estimated three-dimensional volume image.

7. The image processing apparatus according to claim 6 , wherein the generator generates a motion model that estimates the three-dimensional volume image from the motion of the surrogate, based on the optimized three-dimensional volume image and the second surrogate motion.

8. An image processing method by an image processing device, In a first step, a three-dimensional volume image showing a time-varying change in an anatomical structure of a subject is acquired; acquiring a plurality of two-dimensional perspective images of the subject taken from a plurality of imaging angles in a second stage subsequent to the first stage; acquiring a first substitute action that is a motion of a surrogate placed on the body surface of the subject in the first stage and a second substitute action that is a motion of a surrogate placed on the body surface of the subject in the second stage; acquiring an estimated three-dimensional volume image that estimates a three-dimensional volume image showing a time change of the anatomical structure of the subject at the second stage based on the three-dimensional volume image, the first proxy action, and the second proxy action; an image processing method for generating an optimized three-dimensional volume image by deforming the estimated three-dimensional volume image in accordance with the two-dimensional perspective image;

Images