Information processing apparatus, information processing system, and program

Abstract

An object of the present invention is to provide an information processing device, an information processing system, and a program that can improve the accuracy of tracking an object inside a subject. [Solution] According to an embodiment, an information processing apparatus includes a template selection unit and a tracking unit. The template selection unit selects at least one template from a plurality of templates. The tracking unit tracks an object in an image of a subject using the at least one template selected by the template selection unit. Furthermore, the tracking unit switches between one or more templates used to track the object in the image of the subject, using information indicating which of two or more templates included in the plurality of templates has a similarity to the object in the image of the subject that satisfies a predetermined condition.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing system, and a program.

Background Art

[0002] There is known an information processing apparatus that detects the position of an object in a subject using an image obtained by photographing the object in the subject by a template matching method. In the case of such a method, when the number of templates increases, the calculation load increases, and it may be difficult to further improve the tracking accuracy of the object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide an information processing apparatus, an information processing system, and a program capable of improving the tracking accuracy of an object in a subject.

Means for Solving the Problems

[0005] The information processing apparatus according to the embodiment includes a display processing unit, an input reception unit, a template selection unit, and a tracking unit. The display processing unit causes a display unit to display an image of a subject including a marker of the subject and a plurality of candidate templates related to the shape of the marker in the subject. The input reception unit receives a designation operation of a template to be used for tracking by detecting a designation of a position within an image of a candidate template displayed on the display unit. The template selection unit selects at least one template from among the plurality of candidate templates displayed on the display unit by the designation operation. The tracking unit is configured to be able to track the marker of the subject using the at least one template selected by the template selection unit and a similar template similar to the at least one template, to track the marker of the subject using only the at least one template without using the similar template, and to track the marker of the subject using the similar template when the similar template is more similar to the marker in the image than the at least one template. of the and is configured to be able to track the marker of the subject using the similar template when the similar template is more similar to the marker in the image than the at least one template.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Embodiments for Carrying Out the Invention

[0007] Hereinafter, an information processing apparatus, an information processing system, and a program according to embodiments will be described with reference to the drawings. In the following embodiments, an example in which the information processing apparatus, the information processing system, and the program are applied to an apparatus and a system related to radiation therapy will be taken and described. However, the information processing apparatus, the information processing system, and the program are not limited to the above example, and may be an information processing apparatus, an information processing system, and a program applied to other uses different from the treatment. In the following description, the same reference numerals are given to configurations having the same or similar functions. And the overlapping descriptions of those configurations may be omitted.

[0008] Also, as used in this application, "based on XX" means "based on at least XX", and includes cases where it is based on other elements in addition to XX. Also, "based on XX" is not limited to the case where XX is directly used, and includes cases where it is based on something obtained by performing operations or processing on XX. "XX" is an arbitrary element (for example, arbitrary information).

[0009] (First Embodiment) FIG. 1 is a diagram showing a configuration example of a treatment system 1 including an information processing apparatus 100 according to the first embodiment. The treatment system 1 is, for example, a treatment system used for radiation irradiation type treatment. The "radiation" mentioned here includes electromagnetic waves such as X-rays and γ-rays, and particle beams such as proton beams and heavy particle beams. The treatment system 1 is an example of an "information processing system".

[0010] As shown in FIG. 1, the treatment system 1 includes, for example, a radiation irradiation apparatus 10 and an information processing apparatus 100. Here, for convenience of explanation, the radiation irradiation apparatus 10 and the information processing apparatus 100 are described separately, but the information processing apparatus 100 may be provided as a part of the radiation irradiation apparatus 10. The treatment system 1 may further include a plan management apparatus 20 that manages a treatment plan in relation to the radiation irradiation apparatus 10 and the information processing apparatus 100. Here, the radiation irradiation apparatus 10 and the plan management apparatus 20 will be described first, and then the information processing apparatus 100 will be described.

[0011] Here, the "subject" and "object" are defined. The "subject" broadly refers to the object of information processing by the information processing apparatus 100. For example, the "subject" means the object on which the detection process of the object by the information processing apparatus 100 is performed. When the information processing apparatus 100 is used in the treatment system 1, the "subject" is, for example, a patient. The "object" may be a tumor (lesion) in the tissue of the subject, or a marker or the like placed in the subject by a percutaneous treatment. In the present embodiment, the marker is made of metal. The marker is, for example, non-spherical, such as a rod shape or a wedge shape, but may have other shapes. Hereinafter, an example in which a metal marker placed in the subject corresponds to the object is shown. In other words, all the "markers" in the following description may be read as "objects".

[0012] [Radiation Irradiation Device] The radiation irradiation device 10 is a device that irradiates radiation toward the subject P. The radiation irradiation device 10 includes, for example, a couch 11, radiation sources (fluoroscopic radiation irradiation units) 12-1 and 12-2, radiation detectors 13-1 and 13-2, an irradiation gate 14 (treatment beam irradiation unit), a control unit 15, and an input display unit 16.

[0013] The subject P is fixed to the couch 11 by a fixture (not shown). The subject P is supported by the fixture so that the posture of the subject P does not change.

[0014] The radiation source 12-1 irradiates the subject P with fluoroscopic radiation r-1. The radiation source 12-2 irradiates the subject P with fluoroscopic radiation r-2 from a position (different angle) different from that of the radiation source 12-1. The fluoroscopic radiations r-1 and r-2 are, for example, X-rays.

[0015] The radiation detector 13-1 faces the radiation source 12-1 with the subject P in between. The radiation detector 13-1 includes detection elements arranged in an array, and detects the fluoroscopic radiation r-1 irradiated from the radiation source 12-1. The radiation detector 13-1 detects the energy of the radiation r-1, digitally converts the energy, and generates a first image (first fluoroscopic image). The radiation detector 13-1 outputs the generated first image to the image acquisition unit 102 of the information processing apparatus 100. However, the generation of the first image may be performed by the image acquisition unit 102 that receives the detection result of the radiation detector 13-1.

[0016] The radiation detector 13-2 faces the radiation source 12-2 with the subject P in between. The radiation detector 13-2 includes detection elements arranged in an array, and detects the fluoroscopic radiation r-2 irradiated from the radiation source 12-2. The radiation detector 13-2 detects the energy of the radiation r-2, digitally converts the energy, and generates a second image (second fluoroscopic image). The radiation detector 13-2 outputs the generated second image to the image acquisition unit 102 of the information processing apparatus 100. However, the generation of the second image may be performed by the image acquisition unit 102 that receives the detection result of the radiation detector 13-2.

[0017] The radiation detectors 13-1 and 13-2 are, for example, flat panel detectors (FPDs), image intensifiers, color image intensifiers, etc. In FIG. 1, two sets of radiation sources and radiation detectors are shown, but the present invention is not limited thereto. The radiation irradiation apparatus 10 may include three or more sets of radiation sources and radiation detectors. Hereinafter, for convenience of explanation, the first image and the second image will be described as the image IM, except when the first image and the second image are separately described.

[0018] The irradiation gate 14 irradiates the subject P with the treatment beam B during the treatment stage. The treatment beam B includes, for example, X-rays, γ-rays, electron beams, proton beams, neutron beams, heavy particle beams, etc. The treatment beam B is an example of "energy". The irradiation gate 14 is an example of an "irradiation unit". In FIG. 1, only one irradiation gate 14 is shown. However, the radiation irradiation apparatus 10 may include a plurality of irradiation gates 14. Further, the position of the irradiation gate 14 is not limited to the position shown in FIG. 1, and may be a position arranged in a substantially horizontal direction with the subject P, etc.

[0019] The control unit 15 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Further, the control unit 15 may be realized by hardware (including a circuit unit; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by the cooperation of software and hardware. The control unit 15 may have, for example, substantially the same configuration as the hardware configuration of the information processing apparatus 100 described later with reference to FIG. 2.

[0020] The control unit 15 controls the radiation sources 12-1 and 12-2 to irradiate the fluoroscopic radiation r-1 and r-2 at a predetermined interval (predetermined period) based on the treatment plan described later. Further, the control unit 15 controls the irradiation gate 14 to irradiate the treatment beam B based on the treatment plan. For example, the control unit 15 controls the irradiation gate 14 based on the position of the marker MK tracked by the tracking unit 105 of the information processing apparatus 100 described later.

[0021] The input display unit 16 includes, for example, an input device such as a touch panel, a general-purpose keyboard, a mouse, a dedicated key, a dial, or a microphone, and a display device such as a liquid crystal display or an organic electro-luminescence (organic EL) display. The input display unit 16 displays an image output from the information processing apparatus 100. Note that the input display unit 16 may be integrated with the input display unit 103 of the information processing apparatus 100 described later.

[0022] [Treatment Planning Device] The treatment planning device 20 has a plan data storage unit 25 that stores treatment plans in radiation therapy. The plan data storage unit 25 is, for example, an HDD (Hard Disk Drive), a flash memory, a ROM (Read Only Memory), or the like. Here, the "treatment plan" will be described. The treatment plan includes the energy of the radiation (for example, the treatment beam B) to be irradiated to the subject P, the irradiation direction, the shape of the irradiation range, the distribution of the dose to be irradiated in multiple portions, and the like. In creating the treatment plan, for the CT (Computed Tomography) image taken at the time of creating the treatment plan, the boundaries between the tumor and the normal region and the boundaries between the important organs around the tumor are specified, and based on the depth of the tumor position from the body surface and the size of the tumor, the direction and intensity of the treatment beam B to be irradiated are determined. At this time, the position of the marker MK placed in the subject P is also registered in the treatment plan.

[0023] Specifying the boundary between the tumor and the normal region means specifying the position and volume of the tumor. This volume is called, for example, GTV (Gross Target Volume), CTV (Clinical Target Volume), ITV (Internal Target Volume), PTV (Planning Target Volume), etc. GTV is the volume of the tumor that can be visually confirmed from the image and is the site where a sufficient dose must be irradiated in radical treatment. CTV is the volume that includes the GTV and the potential tumor to be treated. ITV is the volume obtained by adding a margin to the CTV in consideration of the movement of the CTV due to predicted physiological movements, etc. PTV is the volume obtained by adding a margin to the ITV in consideration of the misalignment error during treatment. Therefore, these volumes have a relationship of GTV ∈ CTV ∈ ITV ∈ PTV. In this way, a margin considering the possible errors that occur during actual treatment set in the treatment plan is added, and the irradiation field of the treatment beam B is determined. Examples of such errors include the displacement of the position of the subject P in the positioning of the subject P.

[0024] [Information processing device] When the tumor and the marker MK of the subject P move due to the movements of respiration and heartbeat in organs such as the lungs and liver, even if the position of the hospital bed 11 is adjusted, the position of the tumor to be irradiated moves. In such a case, a method of specifying the position of the tumor to be irradiated and irradiating the treatment beam B is used. Such irradiation methods include tracking irradiation in which the tumor (or marker MK) is tracked and the treatment beam B is irradiated to the moving tumor, and ambush irradiation in which the treatment beam B is irradiated when the tumor (or marker MK) comes to the position set in the treatment plan. These irradiation methods are called, for example, respiratory synchronization irradiation methods.

[0025] The information processing apparatus 100 of the present embodiment is an apparatus used in, for example, the respiration-synchronized irradiation method as described above, and is an apparatus that tracks the marker MK in the subject P based on the image IM acquired from the radiation irradiation apparatus 10. The information processing apparatus 100 includes, for example, a template acquisition unit 101, an image acquisition unit 102, an input display unit 103, a template selection unit 104, a tracking unit 105, a display processing unit 106, and a storage unit 107.

[0026] These components are realized, for example, when a hardware processor such as a CPU executes a program (software). Also, some or all of these components may be realized by hardware (including a circuit unit; circuitry) such as an LSI, an ASIC, an FPGA, or a GPU, or may be realized by the cooperation of software and hardware. The storage unit 107 is, for example, an HDD, a flash memory, a ROM, or the like.

[0027] FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus 100. The information processing apparatus 100 includes, for example, a CPU 100A, a RAM (Random Access Memory) 100B, a nonvolatile storage device 100C, a portable storage medium drive device 100D, an input / output device 100E, and a communication interface 100F. The information processing apparatus 100 may include an arbitrary processor such as a GPU instead of the CPU 100A. Also, some of the components shown in FIG. 2 may be omitted.

[0028] The CPU 100A performs various processes described below by expanding and executing a program stored in the non-volatile storage device 100C or a program stored in a portable storage medium mounted on the portable storage medium drive device 100D in the RAM 100B. The RAM 100B is used as a working area by the CPU 100A. The non-volatile storage device 100C is, for example, an HDD, a flash memory, a ROM, or the like. Portable storage media such as DVDs, CDs (Compact Discs), and SD (registered trademark) cards are mounted on the portable storage medium drive device 100D. The input / output device 100E includes, for example, a keyboard, a mouse, a touch panel, a display device, and the like. The communication interface 100F functions as an interface when the information processing device 100 communicates with other devices.

[0029] Returning to FIG. 1 for explanation, the template acquisition unit 101 acquires a plurality of templates 210 regarding the marker MK (object) in the subject P. The "template" means data indicating the typical shape of a certain element (the marker MK in the present embodiment) in the subject P. In the present embodiment, the "template" means data indicating the outer shape of the image of the marker MK (hereinafter referred to as "marker image MKI") that is reflected in the image IM acquired by the image acquisition unit 102. The "marker image MKI" is a projection image of the marker MK viewed from a certain position (for example, the position of the radiation source 12-1 or 12-2). Also, "acquiring a template" includes both the case where the template acquisition unit 101 generates the template 210 and the case where the template acquisition unit 101 acquires the template 210 generated by another device by receiving it. The template acquisition unit 101 may directly output the acquired plurality of templates 210 to the template selection unit 104, the tracking unit 105, and the display processing unit 106, or may store them in the template information DB107a of the storage unit 107. The plurality of templates 210 stored in the template information DB107a are readable by the template selection unit 104, the tracking unit 105, and the display processing unit 106. Also, in the present application, "acquiring a template" also corresponds to the act of reading out the information once stored in the template information DB107a from the template information DB107a.

[0030] As types of the template 210 as described above, for example, there are a template 210 in which the posture of the marker MK in the two-dimensional plane is considered and a template 210 in which the posture of the marker MK in the three-dimensional space is considered. In the present embodiment, an example where the marker MK is a rod-shaped marker will be taken and described.

[0031] FIG. 3 is a diagram showing an example of a template 210 in which the attitude of the marker MK in the two-dimensional plane is considered. The template 210 includes a rectangle MM corresponding to the outer shape (outer line) of the marker image MKI. That is, the inside of the rectangle MM is a region (first region Z1, marker corresponding region) corresponding to the marker image MKI in the template 210. On the other hand, the outside of the rectangle MM is a region (second region Z2) corresponding to a position outside the marker image MKI in the template 210. L in FIG. 3 indicates the length of the long side of the rectangle MM. L corresponds to the length in the longitudinal direction of the marker image MKI. T in FIG. 3 indicates the length of the short side of the rectangle MM. T corresponds to the thickness of the marker image MKI. Θ in FIG. 3 indicates the inclination of the rectangle MM with respect to the reference line REF. The reference line REF is a line segment extending in a specific direction predetermined with respect to the template 210. That is, Θ indicates the orientation of the marker MK in the two-dimensional plane. The quadrangle FW is a quadrangle parallel to the reference line REF and circumscribing the rectangle MM. M in FIG. 3 indicates the margin with respect to the quadrangle FW. M is an amount that determines the size of the template 210. That is, the length obtained by adding 2M to the length of each side of the quadrangle FW is the size of the template 210, and the outer shape TPZ of the template 210 is determined. The template acquisition unit 101 acquires a plurality of templates 210 corresponding to various orientations of the marker MK, for example, by generating a plurality of templates 210 with various changes in the above Θ.

[0032] FIG. 4 is a diagram for explaining a template 210 in which the posture of the marker MK in a three-dimensional space is considered. FIG. 4 extracts and shows only the radiation source 12-1, the radiation detector 13-1, and the radiation r-1 from the radiation irradiation apparatus 10 shown in FIG. 1, and further shows a state in which the bar-shaped marker MK is virtually arranged in the three-dimensional space. As shown in FIG. 4, the template acquisition unit 101 generates a plurality of templates 210 by varying the rotation amounts θ, φ, and η around three axes C1, C2, and C3 orthogonal to each other, thereby obtaining a plurality of templates 210 corresponding to various orientations (various postures in the three-dimensional space) of the marker MK. The axis C1 is, for example, an axis along the path direction of the radiation r-1. The template acquisition unit 101 generates a plurality of templates 210 based on the three-dimensional shape of the marker MK. Note that elements such as the above-described rectangle MM, the first region Z1, the second region Z2, the length L, the width T, and the size M may use the same definitions as the above-described definitions even for the template 210 in which the posture in the three-dimensional space is considered.

[0033] Note that, in either case of the template 210 in which the posture in the two-dimensional plane is considered and the template 210 in which the posture in the three-dimensional space is considered, when there are a plurality of radiation sources and radiation detectors, a set of templates 210 corresponding to the set of marker images MKI detected by a set of radiation detectors composed of a plurality of radiation detectors may be prepared.

[0034] Further, the template acquisition unit 101 obtains one or more similar templates 250 in which one or more parameters (for example, posture parameters (θ, φ, η)) defining the template 210 are slightly changed based on the content of the template 210 selected or specified by the template selection unit 104. The template acquisition unit 101 outputs information indicating the one or more obtained similar templates 250 to the tracking unit 105. Note that this content will be described in detail later.

[0035] Returning to FIG. 1 for explanation, the image acquisition unit 102 acquires the image IM of the subject P by receiving the image IM of the subject P from the radiation detectors 13-1 and 13-2. The image IM of the subject P is an image showing the inside of the subject P, for example, a fluoroscopic image of the subject P. The marker image MKI is captured in the image IM of the subject P. Here, the marker MK is made of metal. The X-rays used when capturing the image IM are easily absorbed by metal. Therefore, in the image IM of the subject P, the portion corresponding to the marker image MKI is displayed darker than the surrounding area. The image acquisition unit 102 outputs the acquired image IM of the subject P to the tracking unit 105 and the display processing unit 106.

[0036] The input display unit 103 includes, for example, an input reception unit 1031 and a display unit 1032. The input reception unit 1031 is an input device such as a touch panel, a general-purpose keyboard, a mouse, a dedicated key, a dial, or a microphone, for example. The input reception unit 1031 receives the input operation of the user. The "input operation of the user" is not limited to the operation of operating a touch panel or a keyboard, and may also be voice input to a microphone. In the present embodiment, the input reception unit 1031 receives the selection operation of the user for selecting (designating) one or more templates 210 included in the plurality of templates 210. In the present embodiment, the input reception unit 1031 receives the selection operation of the user for selecting one or more templates 210 from among two or more templates 210 displayed on the display unit 1032.

[0037] The display unit 1032 is a display device such as a liquid crystal display or an organic EL display. The display unit 1032 has a display screen that can be seen by the user during the input operation on the input reception unit 1031. Note that a user terminal device such as a notebook personal computer or a tablet terminal device may be communicably connected to the information processing apparatus 100 by wire or wirelessly. In this case, the input display unit 103 may not have an input device and a display device. In this case, the input reception unit 1031 is a reception unit that receives the content of the input operation of the user input to the user terminal device from the user terminal device to receive the input operation of the user.

[0038] The template selection unit 104 selects or specifies at least one template 210 (hereinafter referred to as "selected template 230"), which is less than the number of the plurality of templates 210, from among the plurality of templates 210 acquired by the template acquisition unit 101. The selected template 230 is a template used for tracking the marker MK by the tracking unit 105. For example, the template selection unit 104 selects at least one selected template 230 based on the image IM acquired by the image acquisition unit 102. Note that an example in which the selected template 230 is selected based on the image IM acquired by the image acquisition unit 102 will be described in detail among the modified examples of the present embodiment. In the present embodiment, the template selection unit 104 specifies at least one selected template 230 based on the input operation of the user received by the input reception unit 1031. For example, the template selection unit 104 specifies at least one selected template 230 based on the selection operation of the user for selecting one or more selected templates 230 received by the input reception unit 1031. For example, the template selection unit 104 acquires, from the input reception unit 1031, identification information (for example, an identification ID registered corresponding to each template 210) for identifying one or more selected templates 230 received by the input reception unit 1031, and collates the template 210 corresponding to the identification information from among the plurality of templates 210, thereby specifying the selected template 230 selected by the user.

[0039] FIG. 5 is a diagram showing an example of an operation screen Z110 displayed on the display unit 1032. As shown in FIG. 5, the operation screen Z110 includes a template selection image display area Z111 and an image display area Z112. At least the template selection image display area Z111 and the image display area Z112 are arranged side by side on the operation screen Z110. On the template selection image display area Z111, a template selection screen S is displayed based on the control by the display processing unit 106. On the template selection screen S, two or more candidate templates 220 included in a plurality of templates 210 (hereinafter referred to as "candidate templates 220") acquired by the template acquisition unit 101 are displayed. On the other hand, on the image display area Z112, an image IM acquired by the image acquisition unit 102 is displayed based on the control by the display processing unit 106. The image IM acquired by the image acquisition unit 102 includes a marker image MKI in the subject P.

[0040] Note that the layout of the operation screen Z110 is not limited to the above example. For example, the arrangement positions of the template selection image display area Z111 and the image display area Z112 are not limited to the above example. Also, information other than the above information may be displayed within the operation screen Z110. For example, a CT image, a DRR, various information about the subject P, etc. may be displayed. Instead of collectively displaying the template selection screen S and the image IM within the operation screen Z110, the template selection screen S and the image IM may be displayed in separate windows. Also, the display unit 1032 may separately include a first monitor for displaying the template selection screen S and a second monitor for displaying the image IM.

[0041] FIG. 6 is a diagram showing an example of the content of the template selection screen S. As shown in FIG. 6, a plurality of candidate templates 220 that are visually different from each other are displayed on the template selection screen S. For example, the longitudinal directions of the rectangles MM indicating the marker corresponding areas of the plurality of candidate templates 220 are different from each other.

[0042] The plurality of candidate templates 220 shown in FIG. 6 are a plurality of templates in which the orientation of the rectangle MM within the outer shape TPZ of the template is sequentially changed. For example, each candidate template 220 is a template generated by changing at least one of the parameters (θ, φ, η) representing the posture of the marker MK according to a predetermined rule. Each of the parameters (θ, φ, η) is an example of the parameters defining the template 210. The display order of the candidate templates 220 may be arranged according to the direction in which the parameters (θ, φ, η) are changed according to the above-mentioned predetermined rule. The change of the parameters (θ, φ, η) may be substantially continuous or may be at a predetermined interval such as at 15-degree intervals.

[0043] In the present embodiment, the display processing unit 106 further causes the image display area Z112 to display an image IM including the marker image MKI within the subject P acquired by the image acquisition unit 102. Then, the input reception unit 1031 receives a selection operation of the user to select one or more selection templates 230 from among two or more candidate templates 220 displayed on the template selection screen S in a state where the image IM including the marker image MKI within the subject P is displayed in the image display area Z112 by the display processing unit 106. That is, the user observes the marker image MKI in the image IM and selects a selection template 230 having a high similarity to the marker image MKI from among the candidate templates 220 displayed in the template selection screen S. The method of the selection operation may be such that the user presses a button displayed on the operation screen Z110 using a mouse or the like to make a selection. Alternatively, when a touch panel overlapping the template selection screen S is provided, the selection template 230 may be selected by touching a specific portion where the selection template 230 is displayed within the template selection screen S.

[0044] As a result, the template selection unit 104 identifies the selected template 230 based on the user's selection operation. The template selection unit 104 outputs information indicating the identified selected template 230 to the tracking unit 105. Note that the "information indicating the template" may be the image data of the template itself or the identification information for identifying the template.

[0045] The tracking unit 105 uses the image IM acquired by the image acquisition unit 102 and at least one selected template 230 selected or identified by the template selection unit 104 to track the marker MK (for example, the position of the marker MK). In the present embodiment, the tracking unit 105 uses the selected template 230 to track the position of the marker MK in the image IM by means of a template matching method.

[0046] FIG. 7 is a diagram for explaining the tracking process of the position of the marker MK. The tracking unit 105 superimposes (i.e., synthesizes) the selected template 230 on the image IM acquired by the image acquisition unit 102 at a predetermined interval, and scans the selected template 230 over a predetermined range (calculation region). The predetermined range may be the entire area of the image IM or a partial area of the image IM. For example, the tracking unit 105 may limit the predetermined range to include near the trajectory of the marker MK based on the position of the marker MK obtained in the past treatment.

[0047] The above calculation region means the region overlapping with the selected template 230 in the image IM and the range for calculating the similarity with the selected template 230. For example, the tracking unit 105 calculates the similarity between the image IM and the selected template 230 in the calculation region, and identifies the position where the similarity is equal to or greater than the threshold value and the similarity is maximized as the position of the marker MK in the image IM. The method for calculating the similarity by the tracking unit 105 may be based on the normalized cross-correlation or the mutual information amount obtained by calculating the correlation for each pixel between the image IM and the selected template 230. In the present embodiment, the tracking unit 105 uses the separation degree shown below as an index of the similarity.

[0048] FIG. 8 is a diagram for explaining the calculation process of the resolution. (a) in FIG. 8 shows an example of the selection template 230. The inner region of the outer shape TPZ of the selection template 230 is divided into a first region Z1 surrounded by a rectangle MM and a second region Z2 other than that. The first region Z1 is a region corresponding to the marker image MKI. The second region Z2 is a region not corresponding to the marker image MKI.

[0049] (b) in FIG. 8 shows the selection template 230 arranged at the first position T1 and the selection template 230 arranged at the second position T2 with respect to the image IM acquired by the image acquisition unit 102. The first position T1 is a position where the selection template 230 and the marker image MKI do not overlap. The second position T2 is a position where the selection template 230 and the marker image MKI overlap.

[0050] FIG. 8(c) shows the luminance distribution (pixel value) of the composite image IMZ10 obtained by combining the image IM and the selection template 230 when the selection template 230 is arranged at the first position T1. As shown in FIG. 8(c), when the selection template 230 is arranged at the first position T1, the histograms of the pixel values of the first region Z1 and the second region Z2 at the first position T1 show the same tendency as each other. That is, the difference between the histogram HG11 of the pixel values of the first region Z1 and the histogram HG12 of the pixel values of the second region 2 (for example, the difference in the magnitude of the pixel value indicating the maximum value of the frequency) is less than a preset threshold value.

[0051] In Fig. 8(d), when the selection template 230 is arranged at the second position T2, the luminance distribution (pixel value) of the composite image IMZ20 obtained by combining the image IM and the selection template 230 is shown. As shown in Fig. 8(d), when the selection template 230 is arranged at the second position T2, the histograms of the pixel values of the first region Z1 and the second region Z2 at the second position T2 show different tendencies from each other. That is, the two histograms are separated. That is, the difference between the histogram HG21 of the pixel values of the first region Z1 and the histogram HG22 of the pixel values of the second region Z2 (for example, the difference in the magnitude of the pixel value indicating the maximum frequency) is equal to or greater than the threshold value.

[0052] This is because in the histogram HG21 when the first region Z1 and the marker image MKI overlap, in the first region Z1, the frequency of darker pixels is higher than that in the second region Z2. On the other hand, in the second region Z2, the frequency of brighter pixels is relatively higher, and the tendency to be separated from the first region Z1 becomes more prominent. In contrast, in the histogram when the first region Z1 and the marker image MKI do not overlap, there is no significant difference in the tendency of the frequency distribution between the first region Z1 and the second region Z2. Thus, for the two regions Z1 and Z2 in the selection template 230, by analyzing the brightness distribution for each analysis unit (pixel), it is possible to determine whether the position of the selection template 230 is at a position corresponding to the position of the marker MK.

[0053] For example, the tracking unit 105 quantifies the separability of the histograms using the Fisher discriminant criterion method. In the process by the tracking unit 105, the average of the variances (intra-class variances) of the pixel values (luminances) within each of the first region Z1 and the second region Z2 and the variance (inter-class variance) of the pixel values (luminances) between the first region Z1 and the second region Z2 are calculated. This ratio is defined as the separation degree.

[0054] With the above configuration, the tracking unit 105 detects the position of the marker MK using the selection template 230 and tracks the position of the marker MK. The tracking unit 105 outputs information indicating the position of the tracked marker MK to the display processing unit 106.

[0055] The display processing unit 106 is a control unit that controls the display of the display unit 1032. The display processing unit 106 generates information indicating the desired content to be displayed on the display unit 1032, and outputs the generated information to the display unit 1032, thereby causing the display unit 1032 to display the desired content.

[0056] In the selection stage of the template 210, the display processing unit 106 causes the display unit 1032 to display one or more candidate templates 220 included in the plurality of candidate templates 220 related to the marker MK in the subject P. Further, the display processing unit 106 causes the display unit 1032 to display the image IM acquired by the image acquisition unit 102.

[0057] In the tracking stage of the marker MK by the tracking unit 105, the display processing unit 106 causes the display unit 1032 to display information indicating the position of the marker MK tracked by the tracking unit 105. For example, the display processing unit 106 causes the display unit 1032 to display the position of the marker MK tracked by the tracking unit 105 in association with the image IM acquired by the image acquisition unit 102. "Display in association" means that, for example, a display indicating the position of the marker MK tracked by the tracking unit 105 is superimposed and displayed on the image IM. Further, the display processing unit 106 may display the template 210 used for tracking during the tracking of the marker MK by the tracking unit 105. In this case, the position of the template 210 may be a position superimposed on the image IM, or a position arranged side by side with the image IM.

[0058] The memory unit 107 stores the template information DB 107a. FIG. 9 is a diagram showing an example of the content of the template information DB 107a. In the template information DB 107a, data such as template identification information, template image information, feature amounts, similarity, and information indicating that it is being selected are stored. In the template identification information, identification information for uniquely identifying the template 210 is stored. In the template image information, the image data of the template 210 is stored. In the feature amounts, data indicating the features of the template 210 is stored. The feature amounts may include, for example, data such as length L, width T, angle θ, angle φ, angle η, and size M. The angle θ, angle φ, and angle η are pose parameters representing the pose of the marker MK. The length L, width T, and size M are data indicating features other than the pose of the marker MK. The similarity stores, for example, data indicating the similarity between the template 210 and at least one other template 210. In the information indicating that it is being selected, data indicating that the template 210 is in a selected state is stored.

[0059] Each template 210 such as ID_i, ID_j, ID_k, ID_l shown in the column of the template identification information is, for example, an example of a template with the angle θ changed. The aforementioned FIG. 7 is an example when each of the above templates 210 is displayed.

[0060] Next, an example of the treatment flow using the treatment system 1 of the present embodiment will be described. This treatment flow is roughly divided into a treatment plan creation stage, a template selection stage, and a treatment stage.

[0061] [Treatment plan creation stage] The treatment plan is created in advance (for example, about one week before) the day when the treatment is performed. In creating the treatment plan, CT imaging is performed to obtain three-dimensional information of the inside of the subject P including the position, shape, and size of the tumor in the subject P. Then, based on the three-dimensional information, the irradiation conditions (such as irradiation position and irradiation timing) of the treatment beam B for the subject P are set.

[0062] [Template Selection Stage] The selection of the selected template 230 is performed, for example, immediately before the treatment using the treatment beam B, after the positioning of the subject P with respect to the radiation irradiation apparatus 10 (positioning of the bed 11 on which the subject P lies). Here, the positioning of the subject P is performed, for example, in the following procedure. First, a fluoroscopic image of the subject P is acquired using the radiation sources 12-1, 12-2 and the radiation detectors 13-1, 13-2 with the subject P lying on the bed 11. Then, in order to align the positions of the tumor and bone in the subject P included in this fluoroscopic image with the positions of the tumor and bone in the subject P in the treatment plan, the fluoroscopic image is image-matched with a DRR (Digitally Reconstructed Radiograph) in which a virtual fluoroscopic image is reconstructed from the three-dimensional CT image taken at the treatment planning stage. Thereby, the deviation between the position of the subject P at the current time and the position of the subject P in the treatment plan is obtained. The positional deviation of the subject P is obtained by searching for the position of the CT image in which the DRR most similar to the fluoroscopic image is reconstructed. Then, the bed 11 is moved so as to eliminate the positional deviation of the subject P. After moving the bed 11, a fluoroscopic image of the subject P is re-acquired using the radiation sources 12-1, 12-2 and the radiation detectors 13-1, 13-2. Then, image matching is performed between the re-acquired fluoroscopic image and the DRR, and if the positional deviation of the subject P is within the threshold value, the positioning of the subject P is completed.

[0063] In this embodiment, the process of selecting the selection template 230 from the plurality of candidate templates 220 is performed after the positioning of the subject P. That is, the process of selecting the selection template 230 is performed after the positioning of the subject P based on the image IM acquired by the image acquisition unit 102. In this embodiment, the process of selecting the selection template 230 is performed after the subject P is aligned with the position in the image IM acquired by the image acquisition unit 102 where the marker image MKI in the subject P is included. From another perspective, it can be said that the process of selecting the selection template 230 is performed after the subject P is positioned at a position where the marker image MKI included in the image IM acquired by the image acquisition unit 102 and the marker corresponding region (first region Z1) included in the template 210 can be compared.

[0064] FIG. 10 is a flowchart showing an example of the flow of the process for selecting the selection template 230. First, the template acquisition unit 101 acquires a plurality of candidate templates 220 (S101). The process of S101 may be performed before the positioning of the subject P or the creation of the treatment plan. The template acquisition unit 101 may directly output the acquired plurality of candidate templates 220 to the template selection unit 104 or the display processing unit 106, or may store them in the template information DB107a of the storage unit 107.

[0065] Next, the display processing unit 106 causes two or more candidate templates 220 included in the plurality of candidate templates 220 acquired by the template acquisition unit 101 to be displayed in the template selection image display area Z111 of the display unit 1032 (S102). Further, the display processing unit 106 causes the image IM including the marker image MKI in the subject P acquired by the image acquisition unit 102 to be displayed in the image display area Z112 of the display unit 1032 (S103). Thereby, the user can consider the selection template 230 to be selected from among the two or more candidate templates 220 while viewing the image IM including the marker image MKI in the subject P. Note that the image IM including the marker image MKI displayed on the display unit 1032 in S103 may be the image IM acquired for positioning the subject P (for example, a fluoroscopic image re-acquired to confirm that the positional deviation of the subject P is within the threshold value at the last stage of positioning the subject P), or the image IM newly acquired by the image acquisition unit 102 after the positioning of the subject P is completed, or the image IM taken during past treatment. Also, the process of S102 and the process of S103 may be performed simultaneously, or either may be performed first.

[0066] Next, the input reception unit 1031 receives a selection operation of the user to select one or more selection templates 230 in a state where two or more candidate templates 220 and the image IM including the marker image MKI are displayed on the display unit 1032 (S104). Next, the template selection unit 104 identifies one or more selection templates 230 based on the selection operation of the user (S105). The template selection unit 104 outputs information indicating the identified selection template 230 to the tracking unit 105 and the template acquisition unit 101 (S106).

[0067] Next, the template acquisition unit 101 acquires one or more similar templates 250 based on the content of the selection template 230 identified by the template selection unit 104 (for example, parameters defining the selection template 230 (for example, posture parameters (θ, φ, η))) (S107).

[0068] FIG. 11 is a diagram showing a similar template 250. As shown in FIG. 11, for example, the template acquisition unit 101 makes a slight change to one or more parameters (for example, posture parameters (θ, φ, η)) that define the reference template 240, using the selection template 230 specified by the template selection unit 104 as the reference template 240, thereby obtaining one or more similar templates 250. "Making a slight change to the parameters" means, for example, changing the parameters within a range where the similar template 250 is similar to the selected selection template 230, as compared to the candidate template 220 that was not selected among the two or more candidate templates 220 displayed on the display unit 1032. "Making a slight change to the parameters" means, for example, a change for absorbing errors in the size and orientation of the marker image IMK due to physiological phenomena of the subject P or positioning errors of the subject P. For example, the template acquisition unit 101 obtains one or more similar templates 250 with a slight change in the parameter θ, one or more similar templates 250 with a slight change in the parameter φ, and one or more similar templates 250 with a slight change in the parameter η, using the selection template 230 selected by the template selection unit 104 as a reference. Note that the similar template 250 may be a template included in the plurality of candidate templates 220 initially obtained by the template acquisition unit 101. That is, the similar template 250 does not necessarily have to be a template generated after the selection template 230 is selected. The template acquisition unit 101 outputs information indicating these similar templates 250 to the tracking unit 105 (S108).

[0069] Thereafter, the tracking unit 105 confirms whether the marker MK can be tracked. That is, using the images IM acquired at a predetermined interval by the image acquisition unit 102, the selection template 230, and the similar template 250, the tracking unit 105 confirms whether the marker MK can be tracked. This confirmation is performed, for example, over the time lengths of several breaths of the subject P. When it is confirmed by the tracking unit 105 that the marker MK can be tracked, a series of processes related to the selection of the template 210 ends.

[0070] [Treatment stage] FIG. 12 is a flowchart showing an example of the processing flow in the treatment stage. As shown in FIG. 12, the image acquisition unit 102 acquires an image IM including the marker MK in the subject P at a predetermined interval (S201). The image acquisition unit 102 outputs the acquired image IM to the tracking unit 105 and the display processing unit 106.

[0071] The tracking unit 105 tracks the position of the marker MK in the subject P using the selection template 230 specified by the template selection unit 104 and the similar template 250 acquired by the template acquisition unit 101 (S202). That is, the tracking unit 105 tracks the position of the marker MK in the subject P by scanning the image IM using the selection template 230 and the similar template 250. The tracking unit 105 outputs the tracked position of the marker MK in the subject P to the radiation irradiation device 10 and the display processing unit 106.

[0072] The control unit 15 of the radiation irradiation device 10 controls the irradiation gate 14 to irradiate the treatment beam B based on the position of the marker MK in the subject P tracked by the tracking unit 105. For example, when the marker MK in the subject P tracked by the tracking unit 105 enters a predetermined region, the control unit 15 controls the irradiation gate 14 (for example, by sending a control signal to the irradiation gate 14) to irradiate the treatment beam B (S203). The predetermined region is determined based on the relationship between the position of the tumor obtained in the treatment plan and the position of the marker MK and the PTV, etc. For example, a three-dimensional region with a margin added centered on the position of the marker MK on the CT image corresponds to the predetermined region. Also, a region obtained by projecting this region onto the image IM may be used as the predetermined region. In addition, considering the state of the subject P immediately before treatment, the predetermined region may be determined according to the margin added at that time.

[0073] The display processing unit 106 generates information associating the image IM acquired by the image acquisition unit 102 at a predetermined period with the position of the marker MK tracked by the tracking unit 105, and causes the generated information to be displayed on the display unit 1032 (S204). Note that the processes of S203 and S204 may be performed simultaneously, or either one may be performed first. The processes from S201 to S204 described above are continuously repeated until the treatment ends.

[0074] According to the above-described embodiment, the information processing apparatus 100 includes an image acquisition unit 102, a template selection unit 104, and a tracking unit 105. The template selection unit 104 specifies at least one selected template 230, which is less than the number of the plurality of candidate templates 220 regarding the marker MK, based on a user input operation from among the plurality of candidate templates 220. The tracking unit 105 tracks the marker MK included in the image IM acquired by the image acquisition unit 102 using at least one selected template 230 specified by the template selection unit 104.

[0075] According to such a configuration, based on a user input operation, the marker MK can be tracked using the selected template 230 narrowed down from among the plurality of candidate templates 220. Therefore, compared with the case of scanning the marker MK using all the candidate templates 220, the computational load on the information processing apparatus 100 can be reduced. Thereby, the real-time performance of tracking the marker MK can be further enhanced. For this reason, the tracking accuracy of the marker MK can be improved. From another perspective, by tracking the marker MK using the selected template 230 narrowed down from among the plurality of candidate templates 220, it is possible to avoid using a template having a high similarity to a portion other than the marker for tracking. Therefore, the possibility of false detection in tracking can be reduced. From such a perspective as well, the tracking accuracy of the marker MK can be improved.

[0076] In this embodiment, the input reception unit 1031 receives, as a user input operation, a user selection operation (designation operation) for selecting (designating) at least one selection template 230 included in the plurality of candidate templates 220. The template selection unit 104 identifies at least one selection template 230 based on the user selection operation received by the input reception unit 1031. With such a configuration, the user can directly specify a selection template 230 having a high similarity to the marker image MKI. Thereby, scanning can be performed using a selection template 230 that is more similar to the marker image MKI, and further improvement in the tracking accuracy of the marker MK can be achieved.

[0077] In this embodiment, the information processing apparatus 100 further includes a display processing unit 106 that causes the display unit 1032 to display two or more candidate templates 220 included in the plurality of candidate templates 220. The input reception unit 1031 receives a user selection operation for selecting one or more selection templates 230 from among the two or more candidate templates 220 displayed on the display unit 1032. With such a configuration, the user can select the selection template 230 with reference to the marker image MKI in the subject P displayed on the display unit 1032. Thereby, the user can be made to select a selection template 230 that is more similar to the marker image MKI. Thereby, further improvement in the tracking accuracy of the marker MK can be achieved.

[0078] In this embodiment, the input reception unit 1031 receives a user selection operation in a state where the image IM including the marker image MKI in the subject P is displayed on the display unit 1032 by the display processing unit 106. With such a configuration, the user can select the selection template 230 while observing the marker image MKI in the subject P displayed on the display unit 1032. Thereby, the user can be made to select a selection template 230 that is more similar to the marker image MKI.

[0079] In this embodiment, the input reception unit 1031 receives a user's input operation. The template selection unit 104 identifies at least one selected template 230 from among a plurality of templates 210 regarding the shape of the marker MK in the subject based on the user's input operation received by the input reception unit 1031. The tracking unit 105 tracks the marker MK in the image including the marker MK using the identified at least one selected template 230. Thereby, it is possible to further improve the tracking accuracy of the marker MK.

[0080] However, the embodiment is not limited to the above example. For example, the method of tracking the marker MK by the tracking unit 105 is not limited to the above example. For example, the tracking unit 105 may track the marker MK using only the selected template 230 identified by the template selection unit 104 without using the similar template 250 acquired by the template acquisition unit 101.

[0081] Also, the template acquisition unit 101 outputs information indicating a similar template 250 similar to at least one selected template 230 identified by the template selection unit 104. For example, the template acquisition unit 101 acquires one or more similar templates 250 in which one or more parameters defining at least one selected template are changed based on the content of at least one selected template 230 identified by the template selection unit 104. In this case, when the similar template 250 is more similar to the marker MK in the image IM than at least one selected template 230 identified by the template selection unit 104, the marker MK is tracked using the similar template 250. At that time, the tracking unit 105 may track the marker MK using at least one selected template 230 identified by the template selection unit 104 and one or more similar templates 250 acquired by the template acquisition unit 101. According to such a configuration, even when the shape of the marker image MKI changes according to respiratory movement or heartbeat fluctuation in the body, the marker MK can be accurately tracked by the similar template 250.

[0082] Next, some modifications of the first embodiment will be described. In the configuration of each modification, the configuration other than those described below is the same as that of the first embodiment.

[0083] (First Modification Example) FIG. 13 is a diagram showing an example of the content of the template selection screen S of the first modification example. As shown in FIG. 13, the display processing unit 106 of this modification example causes the display unit 1032 to display two or more candidate templates 220 (only one candidate template 220 is shown in FIG. 13). For example, the display processing unit 106 causes the display unit 1032 to display one or two or more candidate templates 220 in a state of being superimposed on the image IM acquired by the image acquisition unit 102 on the template selection screen S of the display unit 1032. In this case, the display processing unit 106 may synthesize an image obtained by cutting out an arbitrary position of the pre-treatment image IM acquired by the image acquisition unit 102 and the image of the binarized candidate template 220 by an image synthesis process such as α-blending, and display the synthesized image on the template selection screen S. α-blending is a method of image synthesis for superimposing two images, and by this method, another image can be superimposed on a specific position of the image to be used as the background. Note that the display processing unit 106 may cause the display unit 1032 to display each of the two or more candidate templates 220 in a state of being superimposed on the image IM.

[0084] According to such a configuration, the user can view the candidate template 220 in a state where it is easy to compare with the marker image MKI included in the image IM of the actual subject P acquired by the image acquisition unit 102. As a result, the user can select a selection template 230 that is more similar to the marker image MKI. Thereby, further improvement in the tracking accuracy of the marker MK can be achieved.

[0085] (Second Modification Example) FIG. 14 shows an example of the content of the template selection screen S of the second modification. As shown in (a) in FIG. 14, in this modification, the display processing unit 106 causes the display unit 1032 to display at least the first template (ID_i) and the second template (ID_j) included in the plurality of candidate templates 220. For example, on the template selection screen S, two candidate templates 220 (the first template (ID_i), the second template (ID_j)) with different orientations of the rectangle MM in the template 210 are arranged and displayed side by side. The two candidate templates 220 are templates included in the plurality of candidate templates 220 acquired by the template acquisition unit 101. Then, the input reception unit 1031 receives a selection operation of the user to select any one of the two candidate templates 220 displayed on the template selection screen S.

[0086] When the user selects the first template (ID_i) and the input reception unit 1031 detects the selection operation for it, the template selection unit 104 identifies the template (ID_i) without identifying the template (ID_j). In this case, the template selection unit 104 selects other candidate templates 220 (the third template (ID_(i - 1)), the fourth template (ID_(i + 1))) that are more similar to the selected template (ID_i) than the template (ID_j) that was not selected from among the plurality of candidate templates 220 as the candidate templates 220 to be next displayed on the template selection screen S. As a result, the display processing unit 106 can display on the display unit 1032 other candidate templates 220 (the template (ID_(i - 1)), the template (ID_(i + 1))) that are more similar to the template (ID_i) than the template (ID_j) from among the plurality of candidate templates 220. Note that the template (ID_(i - 1)) is an example of the "third template", and the template (ID_(i + 1)) is an example of the "fourth template".

[0087] As shown in (b) of FIG. 14, next, the display processing unit 106 causes the template selection screen S to transition to the next screen (transition screen 1). On this transition screen 1, two candidate templates 220 (template (ID_(i - 1)), template (ID_(i + 1))) that are similar to the selected template (ID_i) rather than the template (ID_j) not selected in the previous selection operation are displayed. Then, the input reception unit 1031 receives a selection operation of the user to select any one of the two candidate templates 220 displayed on the transition screen 1. In addition, the input reception unit 1031 receives a selection operation of the user indicating whether to select a new candidate template 220 or the candidate template 220 selected in the previous selection.

[0088] The above description shows an example where the user selects the first template (ID_i). However, the same applies when the user selects the second template (ID_j). In this case, as shown in (c) of FIG. 4, the display processing unit 106 causes the template selection screen S to transition to a transition screen 2 different from the transition screen 1. On this transition screen 2, two candidate templates 220 (template (ID_(j - 1)), template (ID_(j + 1))) that are similar to the selected template (ID_j) rather than the template (ID_i) not selected in the previous selection operation are displayed. Then, the input reception unit 1031 receives a selection operation of the user to select any one of the two candidate templates 220 displayed on the transition screen 2. In addition, the input reception unit 1031 receives a selection operation of the user indicating whether to select a new candidate template 220 or the candidate template 220 selected in the previous selection.

[0089] The information processing apparatus 100 repeatedly performs the process of accepting the above-described selection, and specifies a selection template 230 to be used for tracking when the selection of a new candidate template 220 is no longer required. Although the above description relates to the binary two-selection method, a method (multiple-selection method) may be implemented in which the number of candidate templates 220 displayed at one time is increased to three or more, and one selection template 230 is specified from more candidates.

[0090] According to such a configuration, since the number of candidate templates 220 displayed on the template selection screen S at one time is limited, the user can relatively easily determine which candidate template 220 is similar to the marker image MKI. Therefore, the user can easily select a selection template 230 having a higher degree of similarity to the marker image MKI. Thereby, it is possible to further improve the tracking accuracy of the marker MK.

[0091] (Third Modification Example) FIG. 15 shows an example of the content of the template selection screen S of the third modification example. As shown in FIG. 15, in this modification example, a plurality of candidate templates 220 that are visually different from each other are displayed on the template selection screen S. The plurality of candidate templates 220 shown in this FIG. 15 are candidate templates for the user to select. The difference from the figure shown in FIG. 6 is that when there are two sets of radiation sources and radiation detectors, that is, the radiation source 12-1 and the radiation detector 13-1, and the radiation source 12-2 and the radiation detector 13-2, a plurality of sets of candidate templates 220 corresponding to the sets of marker images MKI created by the respective detectors are displayed.

[0092] For example, ID_Gi, ID_Gj, ID_Gj*, ID_Gk, ID_Gk*, etc. are identification information (template set identification information) of one set of candidate templates 220 (template set). The template set identified by ID_Gi includes two candidate templates 220, ID_i1 and ID_i2. The same applies to other sets.

[0093] As described above, the candidate template 220 grouped in pairs is managed using, for example, the template information DB 107a shown in FIG. 16. FIG. 16 is a diagram showing the template information DB 107a of this modified example. In the template information DB 107a shown in this FIG. 16, data such as template set identification information, template identification information, template image information, feature amounts, similarity, and being selected is stored. The template set identification information has information related to a template set identified by a plurality of template identification information associated therewith.

[0094] According to this modified example, by handling a plurality of images IM having mutually different photographing directions in a group, it is possible to easily find the candidate template 220 similar to the posture of the marker MK in the three-dimensional space.

[0095] (Fourth Modified Example) FIGS. 17 and 18 are diagrams showing an example of the content of the operation screen Z110 of the fourth modified example. As shown in FIGS. 17 and 18, in this modified example, an image display area Z112 is allocated to the operation screen Z110 of the input display unit 103, and the template selection image display area Z111 is not allocated. The display of the operation screen Z110 may display only the image IM. That is, the display processing unit 106 may display only the image IM on the operation screen Z110 of the input display unit 103 and not display the candidate template 220.

[0096] For example, instead of selecting the selection template 230, the user designates the position of the marker image MKI on the image IM by touching the vicinity of the marker image MKI on the image IM displayed on the operation screen Z110 of the input display unit 103. For example, as shown in FIG. 17, the user designates a point overlapping or in the vicinity of the marker image MKI on the image IM.

[0097] When one point is specified in this way, the template selection unit 104 extracts a partial region Z113 of a predetermined size that includes the one point. The predetermined size may be determined in advance based on, for example, the size of the marker MK placed in the subject P. The template selection unit 104 selects, as the selected template 230, the candidate template 220 that is most similar to the marker image MKI from among a plurality of candidate templates 220 for the image IM within the partial region Z113 that includes the marker image MKI. The similarity of the candidate template 220 to the image IM is determined using normalized cross-correlation, mutual information, or the separation degree described above.

[0098] Instead of specifying one point, the range of the partial region Z113 of the image IM may be specified. In this case, for example, as shown in FIG. 18, the user traces the edge of the region that includes the marker image MKI and specifies the partial region Z113 of the image IM as the range. The template selection unit 104 associates the range with a partial region Z113 of a predetermined size. Then, the template selection unit 104 selects, as the selected template 230, the candidate template 220 that is most similar to the marker image MKI from among a plurality of candidate templates 220 for the image IM within the partial region Z113.

[0099] Specifically, the template selection unit 104 may perform the following processing. For example, the template selection unit 104 selects at least one selected template 230 based on the similarity of one or more candidate templates 220 included in a plurality of templates 210 for the partial region Z113 that includes one point within the image IM, or for the partial region Z113 of the image IM when the range is specified.

[0100] For example, the template selection unit 104 may select at least one selected template 230 based on the above similarity threshold. That is, the template selection unit 104 may select a candidate template 220 with a similarity equal to or higher than the threshold as the selected template 230. Thereby, the template selection unit 104 can select a selected template 230 with a high similarity to the marker image MKI.

[0101] Alternatively, instead of the above, the template selection unit 104 derives the similarity of each of two or more candidate templates 220 with respect to the partial region Z113 including one point in the image IM or the partial region Z113 of the image IM when a range is specified. Further, the template selection unit 104 may select at least one selected template 230 from among two or more candidate templates 220 in descending order of the similarity. Thereby, the template selection unit 104 can select a selected template 230 with a high similarity to the marker image MKI.

[0102] That is, in this modification, the display processing unit 106 causes the display unit 1032 to display the image IM acquired by the image acquisition unit 102. The input reception unit 1031 receives, as a user input operation, a designation operation for designating one point in the image IM displayed on the display unit 1032 or a partial region of the image IM. The template selection unit 104 selects at least one selected template 230 based on the user designation operation received by the input reception unit 1031. Thereby, the user can cause the information processing apparatus 100 to select the selected template 230 without performing a selection operation of selecting the selected template 230 from among a plurality of candidate templates 220. Thereby, the convenience of the user is improved.

[0103] (Second Embodiment) Next, a second embodiment will be described. The information processing apparatus 100 of this embodiment is different from the first embodiment in that it dynamically switches the template 210 during tracking instead of using only the selected selection template 230. Thereby, even when the posture of the marker MK placed in the subject P changes greatly due to respiratory movement or heartbeat fluctuations in the body, the marker MK can be tracked with high accuracy. The configuration other than that described below is the same as that of the first embodiment described above.

[0104] Referring to FIG. 19, the functional configuration of the information processing apparatus 100 of the embodiment will be described. FIG. 19 is a diagram showing a configuration example of the treatment system 1 of the embodiment. The information processing apparatus 100 includes a template acquisition unit 101A, an image acquisition unit 102, an input display unit 103, a template selection unit 104, a tracking unit 105A, and a display processing unit 106. The differences from the information processing apparatus 100 in FIG. 1 described above are the template acquisition unit 101A and the tracking unit 105A, and the description will focus on them here.

[0105] The tracking unit 105A first tracks the marker MK in the same manner as the tracking unit 105 in the first embodiment, using the selected template 230 and one or more similar templates 250 selected or specified by the template selection unit 104. Then, in this modified example, the tracking unit 105A evaluates the similarity of the selected template 230 and one or more similar templates 250 to the marker image MKI during the tracking operation. Then, among the selected template 230 and one or more similar templates 250, the tracking unit 105A specifies the selected template 230 or the similar template 250 whose similarity to the marker image MKI satisfies a predetermined condition (for example, the similarity is the maximum) as the specific template. Note that the selected template 230 and one or more similar templates 250 are an example of two or more templates included in a plurality of templates. Then, the tracking unit 105A outputs the parameters (for example, pose parameters (θ, φ, η)) that define the specific template to the template acquisition unit 101A. Instead of this, the tracking unit 105A may output information indicating the specific template (for example, the identification information of the specific template) to the template acquisition unit 101, or may output information indicating the similarity of each of the selected template 230 and one or more similar templates 250 to the template acquisition unit 101A.

[0106] The template acquisition unit 101A re - sets, as a new reference template 240, the template (that is, the above - mentioned specific template) whose similarity to the marker image MKI satisfies a predetermined condition (for example, the similarity is the maximum) among the templates 210 used for tracking the marker MK in the tracking unit 105A. Then, the template acquisition unit 101A obtains one or more new similar templates 250 by slightly changing one or more parameters (for example, pose parameters (θ, φ, η)) that define the new reference template 240. The template acquisition unit 101A outputs information indicating the new reference template 240 and the one or more obtained new similar templates 250 to the tracking unit 105A.

[0107] Then, based on the information output by the template acquisition unit 101A, the tracking unit 105A switches one or more templates 210 used for tracking the marker MK. For example, the tracking unit 105A tracks the marker MK based on the newly specified reference template 240 and the newly acquired one or more similar templates 250 specified by the template acquisition unit 101A. Thereafter, by repeating the above processing, the tracking unit 105A tracks the marker MK while sequentially switching (updating) the template 210. Note that the similar template 250 may be a template included in the plurality of candidate templates 220 initially acquired by the template acquisition unit 101. That is, the similar template 250 does not have to be a template generated after the reference template 240 is newly set.

[0108] FIG. 20 is a flowchart showing the processing flow in the treatment stage of the present embodiment. Note that the processing from S201 to S204 in FIG. 20 is the same as the processing from S201 to S204 in the first embodiment.

[0109] In the present embodiment, as a result of the tracking already performed, the tracking unit 105A outputs information regarding a parameter that defines a specific template that satisfies a predetermined condition with respect to the marker image MKI among the templates 210 used for tracking by the tracking unit 105A, or information indicating the specific template itself (S205). For example, the information output from the tracking unit 105A includes a parameter (for example, attitude parameters (θ, φ, η)) that defines the specific template.

[0110] Then, the template acquisition unit 101A reset - sets the specific template to the new reference template 240 (S206). Then, based on the content of the new reference template 240, the template acquisition unit 101A acquires one or more new similar templates 250 (S207). The tracking unit 105A switches the template 210 used for tracking the marker MK based on the new reference template 240 specified by the template acquisition unit 101A and the one or more new similar templates 250 acquired (S208).

[0111] According to this embodiment, not only the initially selected or specified template 210 is used, but it is also possible to dynamically switch the template 210 during tracking. Thereby, even when the posture of the marker MK placed in the subject P changes greatly due to respiratory movement or heartbeat fluctuations in the body, it becomes possible to track the position of the marker MK.

[0112] (Third Embodiment) Next, the third embodiment will be described. The information processing apparatus 100 of this embodiment displays the position of the marker MK being tracked on the image IM and the position of the tumor estimated from the position of the marker MK so that the user can confirm the position of the marker MK and the position of the tumor.

[0113] Referring to FIG. 1 described above, the functional configuration of the information processing apparatus 100 of the embodiment will be described. The information processing apparatus 100 includes a template acquisition unit 101, an image acquisition unit 102, an input - display unit 103, a template selection unit 104, a tracking unit 105, a display processing unit 106A, and a storage unit 107. The difference from the first embodiment described above is the display processing unit 106A, and the description will be centered on the display processing unit 106A here.

[0114] The display processing unit 106A acquires the image IM from the image acquisition unit 102 and acquires the position of the marker MK from the tracking unit 105. The display processing unit 106A displays a display H1 indicating the position of the marker MK on the image IM.

[0115] FIG. 21 is a diagram showing an example of an image IM displayed by the display processing unit 106A. As shown in FIG. 21, for example, the display processing unit 106A displays a rectangle MM of a template 201 indicating the position of the marker MK on the image IM as a display H1. At this time, the display processing unit 106A may display information indicating the template 210 used to track the marker MK.

[0116] In the present embodiment, the display processing unit 106A displays the position of the tumor estimated from the position of the marker MK in the image IM displayed on the display unit 1032. A method for estimating the position of the tumor from the position of the marker MK will be described. The position y of the tumor is calculated from the position xm of the marker MK using Equation (1) indicating a linear relationship.

[0117] y = Axm + T ···(1)

[0118] In the above Equation (1), xm and y are vectors indicating the respective positions. A is a 2x2 matrix. T is a 1x2 vector. A and T are determined by linear regression from the positions of one or more markers MK obtained before treatment and the position of the tumor.

[0119] As the position of the marker MK and the position of the tumor, for example, the position of the tumor and the position of the marker MK shown in the CT image taken at the time of treatment planning can be used. That is, based on a plurality of CT images (three-dimensional images) taken during one breath, A and T in the above Equation (1) are obtained by linear regression, and the relationship between the position of the tumor and the position of the marker MK can be obtained. For example, the relationship between the position of the tumor and the position of the marker is obtained by linear regression from the centroid of the tumor region projected when the CT image is converted into a DRR and the centroid of the region corresponding to the marker MK.

[0120] The display processing unit 106A calculates the position of the tumor based on the position of the marker MK tracked by the tracking unit 105 and the above Equation (1). Then, the display processing unit 106A displays the calculated position of the tumor on the image IM as a display H2.

[0121] According to this embodiment, the position of the tumor estimated from the position of the marker MK is displayed on the image IM. As a result, the user can continue the treatment while easily confirming the position of the tumor.

[0122] As described above, several embodiments and modifications have been explained, but the embodiments are not limited to the above examples. For example, in the described embodiments and modifications, an example of tracking the rod-shaped marker MK placed in the subject P has been explained. However, the marker MK to be tracked may have a wedge shape or other shapes. For example, when tracking a wedge-shaped marker MK, the shape of MM in the template 210 becomes wedge-shaped. Further, the "object" is not limited to the tumor or the marker MK, and may be a catheter in catheter surgery. In this case, by using the template 210 corresponding to the shape of the catheter, it becomes possible to track the catheter image appearing in the image IM of the subject P taken during the surgery.

[0123] Note that the tracking unit 105 may fix one template 210 used during the treatment, or may switch a plurality of templates 210 according to the state of the subject P. For example, the posture of the marker MK included in the subject P in the three-dimensional space changes due to the breathing of the subject P. The tracking unit 105 approximates the movement of the subject P due to breathing or the change in the posture of the marker MK with a breathing model modeled corresponding to one cycle of breathing, and selects the template 210 corresponding to the phase based on the cycle of the breathing model, and may perform the tracking process of the marker MK.

[0124] Alternatively, as described above, instead of displaying a plurality of candidate templates 220 on the display unit 1032 and accepting the user's selection operation, the information processing apparatus 100 may accept the user's designation operation for designating one point or a portion of the image IM acquired by the image acquisition unit 102, and select the selection template 230 based on the designation operation. Alternatively, instead of this, the template selection unit 104 acquires information regarding the treatment plan of the subject P, sets a predetermined region in the image IM acquired by the image acquisition unit 102 based on the approximate position of the marker MK included in the treatment plan, obtains the similarity of each of all the candidate templates 220 with respect to the predetermined region, and may select the candidate template 220 whose similarity satisfies a predetermined condition (for example, the similarity is maximized) as the selection template 230. In this case, the above-described designation operation by the user becomes unnecessary.

[0125] According to at least one of the embodiments described above, the information processing apparatus 100 includes an input reception unit, a template selection unit, and a tracking unit. The input reception unit 1031 accepts the user's input operation. The template selection unit 104 identifies at least one selection template 230 from among a plurality of templates 210 regarding the shape of the marker MK in the subject based on the user's input operation accepted by the input reception unit 1031. The tracking unit 105 tracks the marker MK in the image including the marker MK using the identified at least one selection template 230. With such a configuration, it is possible to improve the tracking accuracy of the object in the subject.

[0126] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0127] 1... treatment system (information processing system), 14... irradiation gate (irradiation unit), 15... control unit, 100... information processing device, 101... template acquisition unit, 102... image acquisition unit, 1031... input reception unit, 1032... display unit, 104... template selection unit, 105... tracking unit, 106... display processing unit, 210... template, 220... candidate template, 230... selected template, 240... reference template (specific template), 250... similar template, IM... image (first image, second image), P... subject, MK... marker (object), MKI... marker image.

Claims

1. A display processing unit that causes a display unit to display an image of a subject including a marker of the subject and a plurality of candidate templates related to the shape of the marker in the subject; An input reception unit that receives a designation operation of a template to be used for tracking by detecting a designation of a position within an image of the candidate template displayed on the display unit; A template selection unit that selects at least one template from among the plurality of candidate templates displayed on the display unit by the designation operation; Tracking the marker of the subject using the at least one template selected by the template selection unit and a similar template similar to the at least one template; Tracking the marker of the subject using only the at least one template without using the similar template; A tracking unit configured to be able to track the marker of the subject using the similar template when the similar template is more similar to the marker in the image than the at least one template; An information processing apparatus comprising the above.

2. The input reception unit receives, as a user input operation, the designation operation of designating one point or a partial region of the image displayed on the display unit, The display processing unit causes the display unit to display at least a first template and a second template included in the plurality of candidate templates, The display processing unit when the template selection unit identifies the first template without identifying the second template, a third template and a fourth template that are more similar to the first template than the second template among the plurality of candidate templates are selected as the at least one template, and the third template and the fourth template are displayed on the display unit. The information processing apparatus according to claim 1.

3. The template selection unit performs either selecting at least one template from among the plurality of templates related to the marker of the subject or identifying at least one template from among the plurality of templates. The information processing apparatus according to claim 1.

4. The input reception unit As a user input operation, accepting the designation operation of designating one point within the image displayed on the display unit or a partial area of the image, The template selection unit selects the at least one template based on the designation operation received by the input reception unit. The information processing apparatus according to any one of claims 1 to 3.

5. The template selection unit selects the at least one template based on the similarity of the partial area image of the image to the marker image of the subject of a specific candidate template among the plurality of candidate templates, with at least each template included in the plurality of candidate templates not being displayed on the display unit. The information processing apparatus according to claim 4.

6. The template selection unit selects the at least one template based on the similarity threshold. The information processing apparatus according to claim 5.

7. further comprising a template acquisition unit that outputs information indicating a similar template similar to the at least one template specified by the template selection unit, The tracking unit tracks the marker of the subject using the at least one template specified by the template selection unit and the information output by the template acquisition unit. The information processing apparatus according to any one of claims 1 to 6.

8. The display processing unit superimposes the position of the marker of the subject tracked by the tracking unit on the image of the subject acquired by the image acquisition unit and displays it on the display unit. The information processing apparatus according to any one of claims 1 to 7.

9. further comprising a template acquisition unit that outputs information indicating a similar template similar to the at least one template specified by the template selection unit, The tracking unit tracks the marker of the subject using the similar template when the similar template is more similar to the object in the image of the subject than the at least one template specified by the template selection unit. The information processing apparatus according to any one of claims 1 to 8.

10. The object placed inside the subject is the metal marker. The object in the image of the subject indicates the shape of the object placed in the subject. The template selection unit selects at least one template from among a plurality of templates based on the shape of the marker of the subject. The information processing apparatus according to any one of claims 1 to 9.

11. further comprising a template acquisition unit that generates the plurality of templates based on the three-dimensional shape of the marker placed in the subject. The information processing apparatus according to claim 3.

12. a display processing unit that causes a display unit to display an image of the subject including the marker of the subject and a plurality of candidate templates related to the shape of the marker in the subject; an input reception unit that receives a designation operation of a template to be used for tracking by detecting a designation of a position in the image of the candidate template displayed on the display unit; a template selection unit that selects at least one template from among the plurality of candidate templates displayed on the display unit by the designation operation; two or more templates are extracted from the at least one template selected by the template selection unit and a similar template similar to the at least one template, and the template determined from among the two or more templates is switched to track the marker of the subject; tracking the marker of the subject using only the at least one template without using the similar template; a tracking unit configured to be able to track the marker of the subject using the similar template when the similar template is more similar to the marker in the image than the at least one template; An information processing system comprising.

13. On a computer, causing a display unit to display an image of the subject including the marker of the subject and a plurality of candidate templates related to the shape of the marker in the subject; receiving a designation operation of a template to be used for tracking by detecting a designation of a position in the image of the candidate template displayed on the display unit; causing at least one template to be selected from among the plurality of candidate templates displayed on the display unit by the designation operation; Tracking the markers of the subject using the selected at least one template and a similar template similar to the at least one template; Tracking the markers of the subject using only the at least one template without using the similar template; Formed to enable tracking the markers of the subject using the similar template when the similar template is more similar to the markers in the image than the at least one template; Program.

Images