Processing apparatus, processing program, processing method, and processing system

Abstract

Efficiently adjust the relative position of the subjects. [Solution] During the treatment planning stage, a reference image is acquired in which at least a part of the subject is included as the subject. A judgment image is acquired in which the subject is included as the subject when the subject is positioned on the treatment table for treatment. The acquired reference image and the judgment image are input to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, respectively. A process is executed to generate control information for generating position adjustment information to adjust the relative position of the subject based on the judgment result indicating the error obtained from the trained judgment model.

Description

Technical Field

[0001] The present disclosure relates to a processing apparatus, a processing program, a processing method, and a processing system configured to perform processing related to controlling the relative position of a subject to be treated.

Background Art

[0002] Conventionally, in a radiation therapy apparatus for detecting a disease position such as a tumor and irradiating a treatment beam to the disease position to treat the disease such as a tumor, it has been known that there is a need to control a treatment table for positioning a patient. For example, Patent Document 1 describes "a treatment table on which a patient is placed during radiation therapy, comprising a top plate on which the patient is placed, a positioning mechanism for adjusting the position of the top plate, and a force sensor installed at the tip of the positioning mechanism, wherein the top plate has an opening directly above the force sensor, and the patient placed on the top plate is positioned while correcting the deflection of the positioning mechanism using the output of the force sensor."

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Therefore, based on the above-described technology, an object of the present disclosure is to provide a processing apparatus, a processing program, a processing method, and a processing system capable of efficiently adjusting the relative position of a subject according to various embodiments.

Means for Solving the Problems

[0005] According to one aspect of the present disclosure, a processing device is provided comprising at least one processor, wherein the at least one processor is configured to acquire a reference image in which at least a part of a subject is included as the subject during the treatment planning stage, acquire a determination image in which the part of the subject is included as the subject, input the acquired reference image and the determination image, respectively, to a trained determination model for determining the error in the position of the subject included in the reference image and the determination image, and to perform a process for generating position adjustment information for adjusting the relative position of the subject based on a determination result indicating the error obtained from the trained determination model.

[0006] According to one aspect of the present disclosure, a processing program is provided which, executed by at least one processor, acquires a reference image in which at least a portion of a subject is included as the subject during the treatment planning stage, acquires a judgment image in which the portion of the subject is included as the subject, inputs the acquired reference image and the judgment image, respectively, to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, and causes the at least one processor to function to generate position adjustment information for adjusting the relative position of the subject based on the judgment result indicating the error obtained from the trained judgment model.

[0007] According to one aspect of the present disclosure, a processing method is provided which is performed by at least one processor and includes the steps of: acquiring a reference image in which at least a part of a subject is included as the subject during the treatment planning stage; acquiring a determination image in which the part of the subject is included as the subject; inputting the acquired reference image and the determination image into a trained determination model for determining the error in the position of the subject included in the reference image and the determination image, respectively; and generating position adjustment information for adjusting the relative position of the subject based on a determination result indicating the error obtained from the trained determination model.

[0008] According to one aspect of the present disclosure, a processing system is provided comprising at least one processor, wherein the at least one processor is configured to acquire a reference image in which at least a part of a subject is included as the subject during the treatment planning stage, acquire a judgment image in which the part of the subject is included as the subject, input the acquired reference image and the judgment image, respectively, to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, and to perform a process for generating position adjustment information for adjusting the relative position of the subject based on a judgment result indicating the error obtained from the trained judgment model. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide a processing device, a processing program, a processing method, and a processing system that can efficiently adjust the relative position of a subject.

[0010] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved. [Brief explanation of the drawing]

[0011] [Figure 1A] Figure 1A is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. [Figure 1B] Figure 1B is a diagram conceptually showing the configuration of a medical device 200 according to one embodiment of the present disclosure. [Figure 2] Figure 2 is a block diagram showing the configuration of a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 3A] Figure 3A is a conceptual diagram showing an example of a target management table stored in a processing device 100 according to one embodiment of the present disclosure. [Figure 3B]Figure 3B is a conceptual diagram showing an example of a CT image and a simulation image according to one embodiment of the present disclosure. [Figure 4] Figure 4 is a diagram showing the processing flow performed in the processing apparatus 100 according to one embodiment of the present disclosure. [Figure 5A] Figure 5A is a conceptual diagram illustrating an example of a position error calculation process according to one embodiment of the present disclosure. [Figure 5B] Figure 5B is a conceptual diagram illustrating an example of a position error calculation process according to one embodiment of the present disclosure. [Figure 5C] Figure 5C is a conceptual diagram illustrating an example of a position error calculation process according to one embodiment of the present disclosure. [Figure 6] Figure 6 is a conceptual diagram illustrating an example of a simulation image generation process according to one embodiment of the present disclosure. [Figure 7A] Figure 7A shows an example of a simulation image according to one embodiment of the present disclosure. [Figure 7B] Figure 7B shows an example of an X-ray image according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0012] 1. Configuration of Processing System 1 The processing system 1 described herein is a system used to adjust the relative position of a subject on a treatment table during treatment performed by irradiating the subject with a therapeutic beam. Specifically, the processing system 1 acquires a reference image in which at least a portion of the subject is included as the subject during the treatment planning stage. The processing system 1 also acquires a judgment image in which the relevant portion of the subject, positioned on the treatment table for treatment, is included as the subject. The processing system 1 then inputs the acquired reference image and the judgment image into a trained judgment model, and obtains a judgment result from the trained judgment model that indicates the error in the position of the subject that may be included in the reference image and the judgment image. Based on the acquired judgment result, the processing system 1 generates position adjustment information for adjusting the relative position of the subject.

[0013] More specifically, although not limited to the following, the processing system 1 acquires a CT image in the treatment planning stage that includes at least a portion of the subject's body as the subject. This CT image is accompanied by information indicating the diseased area identified by a medical professional or other means, or control information indicating the irradiation position and direction of the treatment beam. The processing system 1 also acquires an X-ray image in the positioning stage that includes the same body part of the subject who is positioned on the treatment table for treatment. The CT image acquired in the treatment planning stage and the X-ray image acquired in the positioning stage are then input into a trained judgment model for determining the error in the subject's position, and a judgment result indicating the error in the subject's position is obtained. Based on this judgment result, at least one of the subject's position and posture on the treatment table is adjusted to eliminate the error. In this way, the processing system 1 can accurately irradiate the subject's position, which has been adjusted on the treatment table, with a treatment beam according to the treatment plan information planned in the treatment planning stage. In other words, the processing system 1 can use the generated control information to irradiate the treatment beam to the target in a more accurate position and direction, thereby enabling treatment of that target.

[0014] In the treatment stage, the treatment beams mainly include radiations such as X-rays, proton beams, and heavy particle beams. In the treatment planning stage, the irradiation position, irradiation direction, etc. of the treatment beam are preset so that it can be appropriately irradiated to the disease position specified through CT images or the like. That is, in the treatment stage, it is extremely important for the treatment beam to irradiate the preset irradiation position with the intensity (e.g., energy amount), irradiation direction, irradiation field, dose fractionation, and irradiation dose preset in the treatment planning stage. Conversely, in the positioning stage prior to the treatment stage, it is extremely important to position the subject at the same location as the irradiation position of the treatment beam determined in the treatment planning stage. However, in the actual positioning stage, due to subtle changes in the physical state or posture of the subject with the passage of time from the treatment planning stage, an error may occur between the disease position (or irradiation position) specified in the treatment planning stage and the disease position of the subject located on the treatment table on the treatment day. Therefore, it is necessary to more efficiently and accurately correct this error in the positioning stage.

[0015] Generally, in the positioning stage, the similarity is calculated between the so-called DRR (Digital Reconstructed Radiograph) image generated from the CT image obtained in the treatment planning stage and the X-ray image obtained before treatment, and the error between the disease position (or irradiation position) specified in the treatment planning stage and the disease position of the subject located on the treatment table on the treatment day is calculated. Then, the relative position of the subject (e.g., the position of the treatment table) is adjusted by the amount of the calculated error.

[0016] Then, an X-ray image is acquired again. This time, a medical staff visually compares the DRR image with the X-ray image acquired again, and manually adjusts the position of the treatment table. This process of acquiring the X-ray image again and adjusting the relative position of the subject (for example, the position of the treatment table) is repeated. When the adjustment is completed, in the treatment stage, the treatment beam will be irradiated in the irradiation direction set by the control information for the determined disease position (or irradiation position). Here, in the treatment planning stage, the disease position, the irradiation direction of the treatment beam, etc. are three-dimensionally and stereoscopically specified in a three-dimensional space using the CT image taken three-dimensionally. On the other hand, the determination image (for example, the X-ray image) is two-dimensionally represented. Therefore, the above-mentioned manual adjustment is carried out by the medical staff's rule of thumb, considering the three-dimensional shape of the disease position and the direction of the treatment beam from the two-dimensional X-ray image. In addition, internal organs, which are soft tissues in the body, may shift or deform between the treatment planning stage and the treatment stage. Also, even parts such as bones with little deformation are affected by the movement of joints and the like. Therefore, it is difficult to completely match the positions in the treatment planning stage and the treatment stage. Due to such circumstances, generally, it is necessary to be positioned while considering which position should be emphasized by the medical staff. At this time, the points to be emphasized include avoiding exposure to important normal sites, positioning while emphasizing the displacement and deformation at important sites (for example, sites with diseases) rather than considering the displacement and deformation at unimportant normal sites (for example, sites without the disease to be treated), and positioning so that the influence of movements such as respiration, pulsation, and peristaltic movement occurring during treatment is small.

[0017] In addition, since the DRR image used in the calculation of errors and visual comparison is usually generated from the CT image which is the reference image, its fineness is lower than that of the determination image (for example, the X-ray image). Therefore, it will affect the accuracy and time of the calculation of similarity and visual comparison.

[0018] In this positioning stage, processing system 1 inputs a reference image (e.g., CT image) acquired during the treatment planning stage and a judgment image (e.g., X-ray image) acquired on the treatment table before treatment into a trained judgment model. This results in a judgment result indicating the error between the disease location (or irradiation location) identified on each CT image during the treatment planning stage and the disease location of the patient on the treatment table on the day of treatment. Based on this judgment result, processing system 1 generates position adjustment information to adjust the relative position of the patient. This enables efficient adjustment of the relative position of the treatment table. Furthermore, while conventional positioning was done while considering which locations medical professionals prioritized, using the trained judgment model allows for efficient position adjustment while taking these points into consideration.

[0019] Furthermore, the trained judgment model used in processing system 1 utilizes training information consisting of a combination of a reference image (e.g., a CT image) and a judgment image (e.g., an X-ray image). In addition, the amount of movement of the treatment table, which has been manually adjusted by a medical professional, can also be added as further training information. By using the trained judgment model trained with such training information, processing system 1 can reproduce the amount of movement adjusted by the medical professional's rules of thumb (e.g., deciding which position to prioritize when adjusting the position), enabling more efficient position adjustment.

[0020] Furthermore, processing system 1 performs high-resolution processing on the DRR image using a trained generative model. This high-resolution DRR image can be used for various purposes, such as training information (virtual X-ray images) for generating the trained judgment model, as a DRR image for calculating similarity, or as a DRR image for comparison with X-ray images visually inspected by medical professionals. This allows for more accurate and rapid error adjustment, and more efficient position adjustment.

[0021] Figure 1A is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. According to Figure 1A, the processing system 1 includes at least a medical device 200 which includes an imaging device 200-1 for photographing a person to be treated and an irradiation device 200-2 for irradiating the person with a treatment beam, and a processing device 100 for performing the process of generating the position adjustment information. These processing devices 100 and medical devices 200 are communicated with each other by at least one of a wireless communication network and a wired communication network.

[0022] Here, Figure 1B is a diagram conceptually showing the configuration of a medical device 200 according to one embodiment of the present disclosure. Specifically, Figure 1B is a diagram conceptually showing a radiation irradiation device, which is an example of the medical device 200 described in Figure 1A. According to Figure 1B, the medical device 200 is used to irradiate a patient lying on a treatment table 210, also called a treatment table, couch, or operating table, which is installed on the medical device 200, with a treatment beam. Therefore, the medical device 200 includes a head 200-2a of an irradiation device 200-2 that emits a treatment beam. The medical device 200 is capable of moving the gantry 200-2b on which the head 200-2a is installed two-dimensionally or three-dimensionally relative to the base 200-2c so that the irradiation direction of the treatment beam is predetermined. Note that the treatment table 210 does not need to be positioned with the patient lying down, and the patient may be positioned in various positions such as sitting or standing.

[0023] Furthermore, the medical device 200 includes, as an imaging device 200-1, an X-ray tube 200-1b for irradiating X-rays to image the subject, and a flat panel detector (FPD) 200-1a for imaging the X-rays irradiated from the X-ray tube 200-1b and transmitted through the subject. This captures a judgment image used to determine the error in the location of the disease in the subject.

[0024] In Figure 1B, an example of such a judgment image is an X-ray image obtained using the X-ray tube 200-1b and its flat panel detector 200-1a. However, it is not limited to X-ray images alone. The judgment image can be any image that can be used to determine the location of the disease, and in addition to X-ray images, for example, MR images, CT images, PET images, infrared images, 3D point cloud data, or any combination thereof can be suitably used.

[0025] Furthermore, although Figure 1B illustrates an imaging device 200-1 consisting of one set of X-ray tubes 200-1b and its flat panel detector 200-1a, the judgment image may be acquired using two sets of X-ray tubes and flat panel detectors with different X-ray irradiation directions. Alternatively, three or more sets of X-ray tubes and flat panel detectors with different X-ray irradiation directions may be installed, and the judgment image may be acquired by selecting the optimal set of one or two X-ray tubes and flat panel detectors according to the orientation of the subject, the location of the disease, the orientation of the gantry 200-2b, etc. In addition, although Figure 1B describes the use of a flat panel detector as the detector, it is also possible to use other detectors such as scintillators.

[0026] Returning to Figure 1A, in processing system 1, typically, when a judgment image captured by the imaging device 200-1 is transmitted to the processing device 100 via a communication network, the processing device 100 inputs the received judgment image and a reference image acquired during the treatment phase into a trained judgment model. The processing device 100 then obtains a judgment result from the trained judgment model that shows the error between the disease location in the judgment image and the disease location (irradiation location) in the reference image. Based on the acquired judgment result, the processing device 100 generates position adjustment information to adjust the relative position of the treatment table. Based on this position adjustment information, the relative position of the subject is adjusted. The processing device 100 also generates control information for irradiating the treatment beam to the position set during the treatment planning phase and transmits it to the medical device 200. Based on the received control information, the medical device 200 controls the treatment beam irradiated from the irradiation device 200-2. As a result, the treatment beam is irradiated to the subject, who is placed on the treatment table 210 and whose relative position has been adjusted, and the disease is treated.

[0027] In this disclosure, the processing system 1 is given as an example consisting only of the processing unit 100 and the medical device 200. However, in addition to these devices, it is also possible to combine various devices depending on the application and purpose, such as an electronic medical record device, a database device, a medical interview device, various diagnostic devices, a model generation device, and a treatment planning device. Specifically, as will be explained in detail below, the processing system 1 is divided into multiple stages, such as the treatment planning stage, the positioning stage, and the treatment stage, and each stage is carried out accordingly. For example, the learning process for generating the trained judgment model and trained generation model used in the positioning stage is performed by the server device and the model generation device. In addition, the processing related to the generation of the treatment plan in the treatment planning stage is performed by the treatment planning device. Furthermore, the processing related to positioning in the treatment stage and the processing related to the generation of control information for controlling the treatment beam are performed by the processing unit 100. By distributing the processing across multiple devices in this way, it becomes possible to, for example, perform treatment planning and learning for other patients in parallel, even while actually performing treatment by irradiating with a treatment beam.

[0028] Furthermore, in this disclosure, "subjects" can refer to any person who may be eligible for treatment, and are not limited to those with specific attributes. Therefore, such subjects may include any person, such as patients, test subjects, diagnostic subjects, and healthy individuals. In addition, the term "person" is sometimes used in this disclosure, and this may refer to subjects or to subjects plus other people.

[0029] Furthermore, in this disclosure, “disease” can be any disease that is treatable with a therapeutic beam, and is not limited to any specific disease. Examples of such diseases include a wide range of conditions, such as brain tumors, head and neck cancers, esophageal cancers, lung cancers, breast cancers, hepatobiliary and pancreatic cancers, rectal cancers, uterine cancers, prostate cancers, skin cancers, malignant lymphomas, or combinations thereof, benign tumors, leukemias, tuberculous lymphadenitis, skin diseases, gastric ulcers, keloids, hemangiomas, eye diseases, heart diseases, arteriovenous malformations, or other gynecological diseases.

[0030] Furthermore, in this disclosure, "the relative position of the subject" refers to the relative positional relationship between the subject on the treatment table and the irradiation device 200-2. Therefore, when position adjustment information is generated to adjust the relative position of the subject, the treatment table or the position of the subject located on the treatment table may be adjusted according to this information, but the position of the irradiation device 200-2 or the irradiation position of the treatment beam emitted from the irradiation device 200-2 may also be adjusted, or both may be adjusted in combination (for example, adjusting the translational deviation by adjusting the position of the treatment table or the subject located on the treatment table and adjusting the rotational deviation by adjusting the position of the irradiation device 200-2 or the irradiation position of the treatment beam emitted from the irradiation device 200-2, or vice versa).

[0031] Furthermore, in this disclosure, "control information" may refer to any information used to control the therapeutic beam. Such control information is not limited to the examples given below, but • Information to instruct whether to turn the treatment beam on or off. • Information to indicate at least one of the irradiation direction and intensity of the treatment beam. • Information indicating the location of the disease based on the diagnostic image. • Combination of the above information These are just a few examples. In the following, unless otherwise specified, we will describe cases in which information is used as control information to instruct at least one of the following: turning the treatment beam on or off.

[0032] 2. Configuration of the processing unit 100 Figure 2 is a block diagram showing the configuration of a processing unit 100 according to one embodiment of the present disclosure. According to Figure 2, the processing unit 100 includes a processor 111, a memory 112, and a communication interface 113. Each of these components is electrically connected to the others via control lines and data lines. The processing unit 100 does not need to have all of the components shown in Figure 2; it is possible to omit some components or add other components. For example, it is possible to use an external memory connected to the processing unit 100 via communication, or a database device, server device, terminal device, etc., together with the processing unit 100. It is also possible to distribute some processing among other devices. In other words, the processing unit 100 is not limited to a single device, but also includes cases where it is distributed among multiple devices depending on the handling of information and the processing load.

[0033] Such a processing device 100 can utilize various forms of devices, such as a server device installed on the cloud, an on-premise server device installed in a specific facility or equipment, a terminal device used by medical personnel at a medical institution or other facility that operates the medical device 200, or a control device connected to the medical device 200 via a communication network.

[0034] The processor 111 functions as a control unit that controls other components of the processing system 1 based on a processing program stored in the memory 112. Specifically, the processor 111 executes processing related to generating control information for the treatment beam based on the processing program. The processor 111 executes, in particular, the following based on the processing program stored in the memory 112: "the process of acquiring a reference image in which at least a part of the subject is included as the subject during the treatment planning stage," "the process of acquiring a judgment image in which a part of the subject is included as the subject," "the process of inputting the acquired reference image and judgment image into a trained judgment model for determining the error in the position of the subject included in the reference image and judgment image," and "the process of generating position adjustment information for adjusting the relative position of the subject based on the judgment result showing the error obtained from the trained judgment model." The processor 111 is mainly composed of one or more CPUs, but a GPU or FPGA may be combined as appropriate.

[0035] Furthermore, the processes related to the generation of the trained decision model and the trained generation model do not necessarily have to be executed by the processing unit 100, but may be executed by other devices other than the processing unit 100.

[0036] Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 112 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs. Specifically, Memory 112 stores programs for execution by the processor 111, such as "a process to acquire a reference image in which at least a part of the subject is included as the subject during the treatment planning stage," "a process to acquire a judgment image in which a part of the subject is included as the subject," "a process to input the acquired reference image and judgment image into a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image," and "a process to generate position adjustment information for adjusting the relative position of the subject based on the judgment result showing the error obtained from the trained judgment model." In addition to these programs, Memory 112 also stores information for the subject management table and the subject management table. Memory 112 also stores the trained judgment model and the trained generation model as programs.

[0037] Furthermore, memory 112 may also include a database device for storing information from the subject management table and the target person management table, via a communication network. In addition, the programs for the trained judgment model and the trained generation model do not necessarily have to be stored in memory 112, but may be stored in the memory of another device other than the processing unit 100.

[0038] The communication interface 113 functions as a communication unit for sending and receiving judgment images and control information between the medical device 200 and other devices via a wired or wireless network. Examples of the communication interface 113 include various types of connectors for wired communication such as USB, SCSI, and wired LAN, wireless communication transceivers such as wireless LAN, Bluetooth®, LTE, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0039] 3. Other devices Although the specific configuration of the medical device 200 is not shown in the diagram, it includes a processor that functions as a control unit, memory that functions as a storage unit, and a communication interface that functions as a communication unit. Each of these elements is electrically connected to the others via control lines and data lines. The medical device 200 does not need to have all of these components; it is possible to omit some or add other components.

[0040] 4. Various information used in the processing of processing system 1 Figure 3A is a conceptual diagram showing a patient management table stored in a processing device 100 according to one embodiment of the present disclosure. According to Figure 3A, the patient management table stores CT images, treatment plan information, simulation images, disease location information, and X-ray images, respectively, associated with patient ID information.

[0041] "Patient ID information" is information used to identify each patient receiving treatment, and is unique to each patient. Patient ID information is newly generated, for example, each time a patient registers for treatment at a medical institution or each time they are registered as a patient for treatment using a therapeutic beam.

[0042] A "CT image" is an image obtained during the treatment planning stage by continuously taking cross-sectional images in the thickness direction along the opposite axis (for example, from the head to the legs) at a fixed spatial interval (e.g., slice interval = 5 mm) over a predetermined area (one or more parts or the whole body) of the human body. A CT image is an image that includes at least a part of the subject as the subject and is used as one of the reference images. Such CT images are used to calculate errors, generate simulation images, and generate treatment planning information and disease location information.

[0043] Furthermore, each CT image can be annotated with disease location information or treatment beam irradiation location information by a medical professional or a trained specific model. Therefore, a CT image may be the image itself taken by the CT device, or it may be an image to which the disease location information or irradiation location information has been input. Also, a CT image may be image data that is the image itself taken by the CT device, or it may be processed image data after image processing has been performed on the image data. In addition, a CT image may be a still image or a video, its image format may be black and white or color, and it may be not limited to two-dimensional images but may also be three-dimensional or four-dimensional images.

[0044] Furthermore, the reference image can be any image that can be used to determine the error in the position of the subject (including the subject itself, a specific part of the subject, or a part of the subject), along with the judgment image, for example, an X-ray image. Therefore, in addition to CT images, MRI images, PET images, or combinations thereof can also be suitably used as reference images.

[0045] "Treatment planning information" is information that indicates the plan for radiation therapy, generated prior to treatment, for example, based on CT images obtained by scanning the subject with a CT scanner. Such treatment planning information includes various setting information such as the intensity of the treatment beam (e.g., energy amount), irradiation direction, irradiation field, dose fractionation, and irradiation dose. Of this information, the irradiation field indicates the irradiation position of the treatment beam, and identifies the position in the three-dimensional coordinate space of a simulated body model reconstructed from the subject's CT images (including both a specific point and a range including that point). This irradiation position is identified by, for example, a medical professional such as a physician identifying the location of the disease in each CT image.

[0046] A "simulation image" is an image that simulates a two-dimensional X-ray image by combining the CT values ​​of multiple CT images taken consecutively along the body axis, along a projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a. Typically, DRR images (Digital Reconstructed Radiograph images) are used for simulation images, but it is also possible to use simulation X-ray images generated by Monte Carlo simulation or other methods based on CT images. Such simulation images can also be generated using a pre-trained machine learning model that can output a simulation image by taking each CT image as input.

[0047] Here, Figure 3B is a conceptual diagram showing an example of a CT image and a simulation image according to one embodiment of the present disclosure. Specifically, Figure 3B is a diagram showing the relationship between a CT image stored in the subject management table of Figure 3A and a simulation image generated from said CT image.

[0048] CT images are typically acquired by taking cross-sectional images in the thickness direction from the head to the legs of the body, at a spatially constant interval (e.g., slice interval = 5 mm) over a predetermined area (one or more parts or the whole body) of the human body. In Figure 3B, multiple CT images, including at least CT images Bm-7 to Bm and CT images Bn-7 to Bn, are taken at a spatially constant interval (e.g., slice interval = 5 mm) across one or more parts of the body.

[0049] As described above, the multiple CT images obtained in this way were taken at regular intervals in the direction from the head to the legs. Therefore, by sequentially superimposing each CT image in this direction and combining CT images at the same coordinate positions, it is possible to virtually form a three-dimensional body model. The simulation image is a pseudo-X-ray image obtained by virtually transmitting this pseudo-body model from a predetermined direction. That is, according to Figure 3B, simulation image Cm is generated as a pseudo-transmissive X-ray image of the body model reconstructed from CT images Bm-7 to CT image Bm, and simulation image Cn is generated as a pseudo-X-ray image of the body model reconstructed from CT images Bn-7 to CT image Bn.

[0050] The CT images and simulation images obtained in this manner are typically used as the CT images and simulation images shown in Figure 3A.

[0051] In addition to images generated from CT images using the methods exemplified above, simulation images can also be generated from CT images using a pre-trained generative model, images drawn using operator input, simulated two-dimensional X-ray images, or pseudo-X-ray images generated from CT images using Monte Carlo simulation. Furthermore, image processing such as contrast adjustment, noise addition, and edge enhancement may be randomly applied to the generated simulation images within a predetermined range, and multiple simulation images with different contrast, noise levels, and edge characteristics may be used.

[0052] "Disease location information" is information indicating the location of a disease contained in the diagnostic image. Such disease location information is information indicating the location where a disease exists, identified by a medical professional such as a physician while referring to the CT image, and represents coordinates in three-dimensional space. Specifically, it is generated by using information indicating the location of the disease where the disease exists (for example, contour information such as CTV (clinical target volume), PTV (planned specimen volume), VOI (Volume of Interest), and ROI (Region of Interest)) for each CT image acquired as described above. Such contour information can be generated by projecting contour information attached to the CT image in the same direction along the projection line used when generating the simulation image. Note that disease location information does not need to be identified by a medical professional as described above; it may also be identified by inputting the CT image into a trained identification model, for example.

[0053] An "X-ray image" is an image taken during the positioning stage so as to include the part of the subject that is positioned on the treatment table 210 for treatment, and is used as one of the judgment images. Such an X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a with the flat panel detector 200-1a. In other words, an X-ray image is an image that visualizes the internal structure of the subject by utilizing the different X-ray absorption rates of each tissue, such as organs and bones.

[0054] The judgment image can be any image that can be used to determine the error in the subject's position in comparison to a reference image, such as a CT image. Therefore, in addition to X-ray images, MR images, CT images, PET images, or any combination thereof can be suitably used as the judgment image.

[0055] Although not specifically illustrated in Figure 3A, it is also possible to add various types of information, such as medical interview information, electronic medical record information, and subject attribute information, to the subject management table in association with the subject ID information.

[0056] 5. Processing flow executed by the processing unit 100 Therapeutic methods using therapeutic beams (e.g., radiotherapy) are important treatments alongside surgery and chemotherapy for tumors such as brain tumors, head and neck cancers, esophageal cancers, lung cancers, breast cancers, hepatobiliary and pancreatic cancers, rectal cancers, uterine cancers, prostate cancers, skin cancers, malignant lymphomas, or combinations thereof. However, these treatments require minimizing the dose irradiated to normal cells surrounding tumor cells in order to cure the tumor or alleviate symptoms. Therefore, it is necessary to create an appropriate and individualized treatment plan for each case, including the intensity of the therapeutic beam, irradiation direction, irradiation field, dose fractionation, irradiation dose, and chemotherapy to be used in combination. For this reason, these treatments are generally divided into a treatment planning stage, which is carried out in advance of the day of treatment; a positioning stage, which corrects the difference in the position and posture of the subject during CT imaging in the treatment planning stage and the position and posture of the subject during X-ray imaging in the treatment stage; and a treatment stage, in which the treatment is carried out.

[0057] Figure 4 is a diagram showing the processing flow performed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 4 is a diagram showing the processing flow performed by the processor 111 of the processing device 100 from the treatment planning stage to the treatment stage. This processing flow is mainly performed by the processor 111 of the processing device 100 reading and executing a program stored in the memory 112.

[0058] In Figure 4, the steps of each process are shown chronologically, but the steps do not need to be performed consecutively. For example, after each step of the treatment planning stage is completed, the steps of the positioning stage and the treatment stage may be performed after an arbitrary period of time.

[0059] (A) Treatment planning stage As shown in Figure 4, during the treatment planning stage, the processor 111 acquires CT images (S101) by receiving CT images taken by the CT scanner, which include at least a portion of the subject as the subject, along with the subject ID information, from the CT scanner or PACS (Picture Archiving and Communication System) via the communication interface 113.

[0060] As explained in Figure 3A, the processor 111 accepts input operations from medical professionals for each stored CT image via an arbitrary input interface, and inputs information indicating the location of the disease where the disease exists (for example, contour information such as CTV, PTV, VOI, or ROI) or information indicating the irradiation position of the treatment beam. The processor 111 then stores each CT image into which information indicating the disease location has been input as reference information, as a reference image, and associates it with the patient ID information in the patient management table.

[0061] Furthermore, the processor 111 stacks each CT image along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a to reproduce a body model containing three-dimensional contour information and generate a simulation image. The processor 111 then stores treatment plan information set by medical professionals such as doctors, based on the generated body model and simulation image, in the patient management table, associating it with the patient ID information (S102). As explained in Figure 3A, this treatment plan information includes various information such as the intensity of the treatment beam (e.g., energy amount), irradiation direction, irradiation field indicating the irradiation position of the treatment beam, dose fractionation, and irradiation dose. The processor 111 also stores the generated simulation image as a reference image in the patient management table, associating it with the patient ID information. Although not specifically shown, the processor 111 may also store the acquired CT images and the body model (image data) in three-dimensional coordinate space reproduced based on the CT images, associating them with the patient ID information.

[0062] Each step in the treatment planning stage involves various considerations, such as creating fixation devices to secure the patient to the treatment table 210 and developing a treatment plan. Therefore, these steps are carried out in advance, rather than on the day of treatment.

[0063] It should be noted that, conventionally, systems using markers (e.g., tumor markers), such as the technology described in Japanese Patent Publication No. 2000-167072, have been known. Therefore, it is possible to use the technology using such markers in conjunction with processing system 1. This makes it possible to switch to processing using markers if, for some reason, it is difficult to detect the location of the disease in processing system 1, which does not use markers. However, in the case of technology using markers, the marker images will be included in the CT images taken in advance to implant markers in the subject's body and in the simulation images reproduced from those CT images. If such images are used as first training medical images, the presence and location of the disease will be judged based on the presence or absence of markers, which is undesirable as training images. Therefore, in such cases, it is possible to remove the marker images from the CT images or simulation images by image processing.

[0064] (B) Positioning stage Immediately before treatment or on the day treatment is performed, a positioning stage is carried out. This positioning stage begins by positioning the subject on the treatment table 210 installed on the medical device 200, with the subject's movement restricted using restraints or the like. Subsequently, when an X-ray image is taken by the imaging device 200-1 of the medical device 200, the processor 111 of the processing unit 100 receives the X-ray image from the medical device 200 along with the subject ID information via the communication interface 113 (S103). The processor 111 stores the received X-ray image as a judgment image in the subject management table, associating it with the subject ID information.

[0065] Next, the processor 111 reads the CT image stored as the reference image in S102 and the X-ray image stored as the judgment image in S103, and calculates the error in the position of the subject included in both images (S104). Specifically, the processor 111 inputs both images into the trained judgment model in order to determine this error. The processor 111 then obtains a judgment result from the trained judgment model indicating the error in the position of the subject as output. Details of this process by the trained judgment model will be explained in Figures 5A to 5C, etc.

[0066] Furthermore, in calculating positional errors, it is possible for medical professionals to specify in advance which parts of the body should be judged as miscalculated. Specifically, the processor 111 accepts input from any terminal device available to medical professionals and obtains information specifying any part of the body to be focused on in determining the error. This specified information may be information indicating coordinates, the name of the part, the outline of the part, etc. The processor 111 then calculates the positional error at the arbitrary part identified by the acquired specified information. This makes it possible for medical professionals to specify in advance, for example, important parts such as disease locations and vital organs, parts that receive a large amount of radiation from the treatment beam, or parts that are easily projected in X-ray images such as bones, and to perform positioning that reflects the points to focus on in calculating the error.

[0067] When the processor 111 obtains a judgment result, it stores the judgment result information in the memory 112 and outputs the judgment result information to any terminal device accessible to medical personnel via the communication interface 113 (S105).

[0068] The judgment result may be any information indicating the amount of displacement between the position of the subject's body in the CT image stored as the reference image and the position of the subject's body in the X-ray image stored as the judgment image. Furthermore, this position may be the entire body of the subject, any part of the subject, or the location of the subject's disease. In other words, the judgment result or the judgment result information indicating said judgment result may include, but is not limited to, the following information. • Information indicating a positional error between the two images • Information indicating the amount of positional error between the two images.

[0069] Next, the processor 111 determines whether the position error is within an acceptable range. This acceptable range is a pre-set range. If the error falls within this range, the processor 111 determines that no adjustment of the treatment table 210's position is necessary and proceeds to S110. On the other hand, if the error is not within the range, the processor 111 generates position adjustment information to adjust the relative position of the treatment table 210 based on the acquired determination result (S107), and outputs this position adjustment information to the medical device 200 via the communication interface 113 (S108). When the medical device 200 receives this position adjustment information via the communication interface, it adjusts the relative position of the patient by adjusting at least one of the settings, such as the position of the treatment table, the position of the irradiation device 200-2, and the irradiation direction of the treatment beam, based on the position adjustment information to correct the determined error. Then, the processor 111 returns to S103.

[0070] As described above, once the relative position of the subject is adjusted, the processor 111 again executes the process of acquiring the X-ray image in S103, the process of calculating the position error in S104, the process of outputting the judgment result in S105, and the process of determining whether the error is within an acceptable range in S106. If the processor 111 does not determine in S106 that the error is within an acceptable range, it executes the process of generating position adjustment information in S107 and the process of outputting position adjustment information in S108. As a result, the relative position of the subject is adjusted again based on the position adjustment information by adjusting at least one of the settings such as the position of the treatment table, the position of the irradiation device 200-2, and the irradiation direction of the treatment beam. In other words, the processor 111 repeats each of the processes from S103 to S108 and the adjustment of the relative position of the subject until the error in S106 is within an acceptable range.

[0071] The position adjustment information may include, but is not limited to, the following. Alternatively, the judgment result information may be output directly as position adjustment information. • Information indicating the amount of adjustment to be made to the position of the treatment table or the person on the treatment table in order to correct the positional error between the two images. • Information indicating the amount of adjustment to be made to the position of the irradiation device 200-2 in order to correct the positional error between the two images. • Information indicating the amount of adjustment needed for settings such as the direction of irradiation of the treatment beam in order to correct positional errors between the two images. - A combination of information indicating the amount of adjustment to the position of the treatment table or the patient on the treatment table, information indicating the amount of adjustment to the position of the irradiation device 200-2, and information indicating the amount of adjustment to the setting of the irradiation direction of the treatment beam, in order to correct the positional error between the two images.

[0072] Furthermore, although not particularly essential, it is possible to add a step of visual adjustment by a medical professional during the positioning stage. Specifically, after it is determined that the error is within an acceptable range, when an X-ray image is taken again by the imaging device 200-1 of the medical device 200, the processor 111 of the processing unit 100 receives the X-ray image from the medical device 200 along with the patient ID information via the communication interface 113 (S109). The processor 111 associates the received X-ray image with the patient ID information and stores it in the patient management table as a judgment image for visual inspection. The processor 111 also reads the simulation image stored in association with the patient ID information during the treatment planning stage and outputs the X-ray image acquired for visual inspection and the simulation image to a terminal device accessible to a medical professional via the communication interface 113 (S110).

[0073] Here, the simulation image output in S110 is generated from the CT image as described above, and therefore has lower resolution than the X-ray image. This may affect the accuracy of visual confirmation by medical professionals. Therefore, the processor 111 can input the simulation image into a trained generation model and perform a high-resolution processing on the simulation image. Details of this processing are explained in Figure 6 and other figures.

[0074] Medical personnel refer to the X-ray image and the simulation image (or the high-resolution simulation image) to check for any errors in the patient's position. If an error is found, the processor 111 accepts input from the medical personnel via an optional input interface and generates position adjustment information to adjust the position of the treatment table 210 or the patient on the treatment table 210, and outputs this position adjustment information to the medical device 200 via the communication interface 113. Upon receiving the position adjustment information via the communication interface, the medical device 200 adjusts the position of the treatment table based on the position adjustment information to correct the determined error.

[0075] The processor 111 then repeats the above processes in S109 and S110 until the medical professional accepts the error. With this, the processor 111 terminates the processing in the positioning stage.

[0076] By performing this processing at the positioning stage, the coordinate space in which the X-ray image is formed and the coordinate space in which the irradiation position of the treatment beam is set correspond precisely to each other, making it possible to accurately irradiate the treatment beam to the position set at the treatment planning stage. Furthermore, as will be explained in detail in Figures 5A to 5C, a trained judgment model or a trained generation model is used to calculate errors and adjust the position at the positioning stage. Therefore, it becomes possible to perform the processing at the positioning stage more efficiently.

[0077] (C) Treatment stage During the treatment phase, the processor 111 performs a control information generation process to generate control information for the treatment beam (S111), and transmits the generated control information to the medical device 200 via the communication interface 113 (S112). Specifically, the processor 111 reads treatment plan information from the patient management table based on the patient ID information. Then, based on the treatment plan information, the processor 111 generates control information to control the irradiation device 200-2 of the medical device 200 in order to irradiate the treatment beam to the patient's position adjusted during the positioning phase, using information such as the intensity (e.g., energy amount), irradiation direction, irradiation field, dose fractionation, and irradiation dose of the treatment beam. The processor 111 transmits the generated control information to the irradiation device 200-2 of the medical device 200 via the communication interface 113.

[0078] When the medical device 200 receives control information, it operates the irradiation device 200-2 based on the control information and performs the process of irradiating the subject with a therapeutic beam. In this way, during the treatment phase, steps S110 to S111 in the processing unit 100 and the irradiation process in the medical device 200 are repeatedly executed according to the treatment plan information. With this, the processor 111 terminates the processing in the treatment phase.

[0079] 6. Processing related to the calculation of errors in the positioning stage. Figures 5A to 5C conceptually illustrate an example of a position error calculation process according to one embodiment of the present disclosure. Specifically, Figures 5A to 5C show details of the process for generating a trained judgment model used in the position error calculation process and the error calculation process using the generated trained judgment model. In particular, Figure 5A shows the process when an X-ray image is used as training information, Figure 5B shows the process when a virtual X-ray image is used as training information, and Figure 5C shows the process when the amount of movement of the treatment table manually adjusted by a medical professional is used as training information. Each process is mainly performed by the processor 111 of the processing unit 100 reading and executing a program stored in the memory 112.

[0080] (A) Processing when X-ray images are used as training information. As shown in Figure 5A, processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as an object in the reference image and the judgment image (S210). Here, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0081] First, the processor 111 receives CT images from a CT scanner (S211) that include a part of a person (which may or may not include the subject) as the subject, as training information to be used for deep learning. Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (e.g., contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112. It is also possible to use training simulation images (e.g., training DRR images) generated from the CT images as training CT images.

[0082] Next, the processor 111 acquires an X-ray image from the medical device 200 by positioning the person who was the subject of the CT image on the treatment table 210 (S212). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. This X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a judgment image in the memory 112, associating it with the reference image.

[0083] Next, the processor 111 reads the CT image acquired in S211 and the X-ray image received in S213 from memory 112 as a combination of a training CT image (i.e., a training reference image) and a training X-ray image (i.e., a training judgment image) (S213). The processor 111 similarly prepares combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) for multiple people, and acquires positional error information between the two images prepared for training as the correct label. Then, the processor 111 reads the combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) acquired from multiple people, as well as the positional error information (correct label) between the two images, as training information.

[0084] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and critical area location information, which are generated based on the CT images acquired in S211, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., which areas will receive how much radiation). For example, a slight misalignment in an area with low radiation exposure may have little impact, but even a slight misalignment in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be accurately aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation by the treatment beam is undesirable (e.g., non-disease areas), and misalignment in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0085] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learners that constitute the deep neural network (DNN) and repeats the learning process while adjusting the parameters of each neuron that constitute the deep neural network (DNN) (e.g., the number of layers, the number of nodes in each layer, the connection method of nodes between layers, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix). Then, the processor 111 acquires a trained decision model (e.g., the deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in the memory 112. With this, the processor 111 finishes generating the trained decision model.

[0086] In the above example, the trained decision model is generated using a deep neural network (DNN). However, it is also possible to generate the model using methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; methods using gradient boosting decision trees (GBDT) such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0087] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0088] Next, the processor 111 reads the trained judgment model generated in S210 from memory 112 and performs position error determination processing (S220). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the location of the disease, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0089] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0090] Thus, by using a pre-trained classification model in determining positional errors, it becomes possible to make determinations more efficiently compared to methods such as calculating similarity.

[0091] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0092] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and critical site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training critical site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0093] (B) Processing when virtual X-ray images are used as learning information. As shown in Figure 5B, the processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as an object in the reference image and the judgment image (S310). Here, as in Figure 5A, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0094] First, the processor 111 receives CT images from a CT scanner (S311) that include a part of a person (which may or may not include the subject) as the subject, as training information to be used for deep learning. Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (e.g., contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112. It is also possible to use training simulation images (e.g., training DRR images) generated from the CT images as training CT images.

[0095] In deep learning, a large amount of training information is required for learning. In Figure 5A, X-ray images taken of a person positioned on the treatment table 210 were used as training information, but there are many time constraints in efficiently gathering people with diseases and acquiring this information. Therefore, the processor 111 generates a simulation image based on the CT image acquired in S311 as a virtual X-ray image (S312). Specifically, the processor 111 generates a DRR image as a virtual X-ray image by combining CT values ​​in the direction along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a for the body model reproduced based on the CT image. In addition to this, the processor 111 generates multiple virtual X-ray images by, for example, applying an arbitrary position error to the CT image or the DRR image generated from the CT image using the following method. Method 1: The processor 111 acquires a simulated image generated by combining CT values ​​in a direction that is randomly shifted from the above direction. Method 2: The processor 111 generates an image in which the disease location identified in the CT image is randomly shifted, and obtains a simulated image generated based on that image. Method 3: The processor 111 generates an image in which the disease location identified in the DRR image is randomly shifted, and acquires this image as a simulation image.

[0096] In this way, the processor 111 generates multiple simulated images as virtual X-ray images, each with a pseudo-arbitrary position error applied to a set of CT images. The processor 111 stores the generated virtual X-ray images in memory 112 as judgment images, associating them with the reference images.

[0097] Next, the processor 111 reads the CT images acquired in S311 and each virtual X-ray image generated in S312 from memory 112 as a combination of a training CT image (i.e., training reference image) and a training virtual X-ray image (i.e., training judgment image) (S313). The processor 111 similarly prepares combinations of training CT images (i.e., training reference images) and training virtual X-ray images (i.e., training judgment images) for multiple people, and acquires position error information between the two images prepared for training (for example, information indicating the amount of an arbitrary position error given in S312) as the correct label. Then, the processor 111 reads the combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) acquired from multiple people, and the position error information (correct label) between the two images as training information.

[0098] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and critical area location information, which are generated based on the CT images acquired in S311, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., how much radiation exposure each area will receive). For example, a slight misalignment in an area with low radiation exposure may have little impact, but even a slight misalignment in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be precisely aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation by the treatment beam is undesirable (e.g., non-disease areas), and misalignment in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0099] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learners that constitute the deep neural network (DNN), similar to Figure 5A, and repeats the learning process while adjusting the parameters of each neuron that constitute the deep neural network (DNN) (e.g., number of layers, number of nodes in each layer, connection method of nodes between layers, activation function, error function, gradient descent algorithm, pooling region, kernel, weight coefficients, and weight matrix). Then, the processor 111 acquires a trained decision model (e.g., a deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in memory 112. With this, the processor 111 finishes generating the trained decision model.

[0100] In the above example, the trained decision model is generated using a deep neural network (DNN). However, it is also possible to generate the model using methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; methods using gradient boosting decision trees (GBDT) such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0101] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0102] Next, the processor 111 reads the trained judgment model generated in S310 from the memory 112 and performs position error determination processing (S320). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the disease location, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0103] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0104] Thus, by using a pre-trained classification model in determining positional errors, it becomes possible to determine them more efficiently than by methods such as calculating similarity. Furthermore, in generating the pre-trained classification model, instead of using X-ray images as training information, virtual X-ray images with an arbitrary positional error are used, allowing for more efficient collection of training information and even more efficient determination of positional errors.

[0105] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0106] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and critical site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training critical site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0107] (C) Processing when the amount of movement of the treatment table manually adjusted by a medical professional is used as learning information. As shown in Figure 5C, processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as an object in the reference image and the judgment image (S410). Here, as with Figures 5A and 5B, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0108] First, the processor 111 receives CT images from a CT scanner (S411) that include a part of a person (which may or may not include the subject) as training information for deep learning. Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (e.g., contour information such as CTV, PTV, VOI, ROI) identified by a medical professional or a trained specific model referring to each CT image. The processor 111 stores the CT images accompanied by the disease location information as reference images in the memory 112. It is also possible to use training simulation images (e.g., training DRR images) generated from the CT images as training CT images.

[0109] Next, the processor 111 acquires an X-ray image from the medical device 200 by positioning the person who was the subject of the CT image on the treatment table 210 (S412). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. This X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a judgment image in the memory 112, associating it with the reference image.

[0110] As explained in S108 and S109 of Figure 4, in actual treatment, in addition to adjusting the treatment table 210 based on the calculated error, medical professionals may manually adjust the position of the treatment table 210 based on empirical rules while visually comparing the simulation image and the X-ray image. Therefore, the processor 111 acquires information indicating the amount of movement of the treatment table 210 that has been manually adjusted by the medical professional as the correct label (S413). The processor 111 stores the acquired information indicating the amount of movement of the treatment table 210 in memory 112, associating it with the reference image.

[0111] Next, the processor 111 reads the CT image acquired in S411, the X-ray image generated in S412, and the information indicating the amount of movement of the treatment table 210 acquired in S413 from the memory 112 as a combination of a learning CT image (i.e., a learning reference image), a learning X-ray image (i.e., a learning judgment image), and learning movement amount information (S414). The processor 111 similarly prepares combinations of learning CT images (i.e., learning reference images), learning X-ray images (i.e., learning judgment images), and learning movement amount information for multiple people. The processor 111 then uses the combinations of learning CT images (i.e., learning reference images), learning X-ray images (i.e., learning judgment images), and learning movement information acquired from multiple people as learning information.

[0112] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and critical area location information, which are generated based on the CT images acquired in S411, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., how much radiation exposure each area will receive). For example, a slight misalignment in an area with low radiation exposure may have little impact, but even a slight misalignment in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be precisely aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation by the treatment beam is undesirable (e.g., non-disease areas), and misalignment in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0113] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learners that constitute the deep neural network (DNN), similar to Figure 5A, and repeats the learning process while adjusting the parameters of each neuron that constitute the deep neural network (DNN) (e.g., number of layers, number of nodes in each layer, connection method of nodes between layers, activation function, error function, gradient descent algorithm, pooling region, kernel, weight coefficients, and weight matrix). Then, the processor 111 acquires a trained decision model (e.g., a deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in memory 112. With this, the processor 111 finishes generating the trained decision model.

[0114] In the above example, the trained decision model is generated using a deep neural network (DNN). However, it is also possible to generate the model using methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; methods using gradient boosting decision trees (GBDT) such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0115] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0116] Next, the processor 111 reads the trained judgment model generated in S310 from memory 112 and performs position error determination processing (S420). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the location of the disease, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0117] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0118] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0119] Thus, by using a pre-trained judgment model in determining positional errors, it becomes possible to make judgments more efficiently compared to methods such as calculating similarity. Furthermore, in generating the pre-trained judgment model, information indicating the amount of manual movement of the treatment table 210 is used as training information to reflect the empirical rules of medical professionals. Therefore, it is possible to obtain judgment results with higher accuracy that reflect the empirical rules of medical professionals, and to adjust the position even more efficiently.

[0120] Furthermore, if, as a result of adjusting the treatment table 210 based on the judgment result output from the trained judgment model, the treatment table 210 is manually adjusted by a medical professional in S108 and S109 of Figure 4, the processor 111 may acquire information indicating the amount of movement of the treatment table 210 and use this information to further train the trained judgment model.

[0121] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and critical site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training critical site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0122] 7. High-resolution processing Figure 6 is a conceptual diagram showing an example of a simulation image generation process according to one embodiment of the present disclosure. Specifically, Figure 6 shows an example of a high-resolution processing performed on a simulation image. This processing is mainly performed by the processor 111 of the processing unit 100 reading and executing a program stored in the memory 112.

[0123] Here, Figure 7A shows an example of a simulation image according to one embodiment of the present disclosure. Figure 7B shows an example of an X-ray image according to one embodiment of the present disclosure. Specifically, Figure 7A shows an example of a simulation image output in S109 of Figure 4 or used as a virtual X-ray image in Figure 5B. Figure 7B shows an example of an X-ray image output in S109 of Figure 4 or used in Figure 5A. As is clear from Figures 7A and 7B, since the simulation image is generated by reconstructing from a CT image, its resolution may be inferior to the X-ray image acquired during the positioning stage. Therefore, it is possible to acquire a high-resolution simulation image to adjust the position more efficiently.

[0124] As shown in Figure 6, processor 111 executes a process related to generating a trained generative model for generating a high-resolution simulation image by increasing the resolution of the generated simulation image (S510). Here, we will explain the case in which the trained generative model is generated by deep learning using a deep neural network (DNN).

[0125] First, the processor 111 receives CT images from a CT scanner as training information to be used for deep learning, in which a part of a person (which may or may not include the subject) is included as the subject (S511). Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (for example, contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112.

[0126] The processor 111 then generates a simulation image (S512) by combining the CT values ​​with the body model reproduced based on the CT image in a direction along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a. The processor 111 stores the generated simulation image in memory 112, associating it with the CT image.

[0127] Next, the processor 111 acquires an X-ray image from the medical device 200 by positioning the person who was the subject of the CT image on the treatment table 210 (S513). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. This X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a judgment image in the memory 112, associating it with the simulation image.

[0128] Next, the processor 111 reads the simulation image acquired in S512 and the X-ray image received in S513 from memory 112 as a combination of a training simulation image (i.e., training image) and a training X-ray image (i.e., training judgment image) which is the correct label (S513). The processor 111 similarly reads the combination of a training simulation image (i.e., training image) and a training X-ray image (i.e., training judgment image) which is the correct label for multiple people. Then, the processor 111 uses the combinations of training simulation images (i.e., training images) and training X-ray images (i.e., training judgment images) acquired from multiple people as training information to train the model.

[0129] As described above, when the processor 111 acquires training information, it provides this training information to the learners that constitute the deep neural network (DNN), and using the training X-ray images as the ground truth, it repeats training while adjusting the parameters of each neuron that constitute the deep neural network (DNN) (e.g., number of layers, number of nodes in each layer, connection method of nodes between layers, activation function, error function, gradient descent algorithm, pooling region, kernel, weight coefficients, and weight matrix). Then, the processor 111 acquires a trained generative model (e.g., a deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained generative model as a program in memory 112. With this, the processor 111 finishes generating the trained generative model.

[0130] In the above, the trained generative model is generated using a deep neural network (DNN). Specifically, it is also possible to generate it using methods that utilize neural networks such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; methods that utilize gradient boosting decision trees (GBDT) such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0131] Furthermore, although the case in which the trained generative model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained generative model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0132] Next, the processor 111 reads the trained generative model generated in S210 from memory 112 and performs high-resolution processing of the simulation image (S520). Specifically, the processor 111 reads the simulation image acquired during the treatment planning stage from the patient management table and inputs the simulation image into the trained generative model.

[0133] The processor 111 obtains a high-resolution simulation image, which is a high-resolution version of the pre-trained generative model, as the generated result. Thus, a high-resolution processing is performed using the pre-trained generative model. The high-resolution simulation image thus generated is used for visual adjustment of the treatment table 210 by medical personnel and as training information for the pre-trained judgment model.

[0134] Although the high-resolution processing is performed by the processor 111, if the trained generative model is stored in the memory of another device such as a server, the processor 111 may send the simulated image to the server device and have it executed by the processor of that server device.

[0135] In this way, by generating high-resolution simulation images, it becomes possible to adjust the treatment table 210 by visual inspection by medical professionals more accurately and efficiently. Furthermore, as shown in Figure 5B, by using the high-resolution simulation images as virtual X-ray images to generate a trained judgment model, the accuracy of the trained judgment model can be improved, enabling more efficient error detection.

[0136] In this embodiment, we can provide a processing device, a processing program, a processing method, and a processing system that can efficiently adjust the relative position of a subject.

[0137] 8. Other Various images such as CT images, simulation images, and X-ray images may not be image data itself, but rather image data that has undergone various preprocessing steps such as high-resolution enhancement, region extraction, noise reduction, edge enhancement, image correction, and image transformation, including filtering such as bandpass filters (including high-pass and low-pass filters), averaging filters, Gaussian filters, Gabor filters, Canny filters, Sobel filters, Laplacian filters, median filters, and bilateral filters; vascular extraction using Hessian matrices; segmentation of specific regions (e.g., disease regions) using machine learning; trimming of segmented regions; de-haze processing; super-resolution processing; and combinations thereof.

[0138] Figure 4 illustrates the case where the position of the treatment table 210 is adjusted to correct for positional errors. However, the procedure is not limited to this; the irradiation direction and irradiation position of the treatment beam, which are set during the treatment planning stage, may also be adjusted according to the determined positional error.

[0139] Figure 4 illustrates the case where the relative position of the subject is adjusted by outputting the generated position adjustment information to the medical device 200. However, the process is not limited to this; the processor 111 of the processing unit 100 may also output the position adjustment information to any terminal device accessible to medical personnel via the communication interface 113, and display the position adjustment information via the display of the terminal device. Based on the displayed position adjustment information, medical personnel can also manually adjust the relative position of the subject.

[0140] The embodiments and variations of this disclosure are presented as examples only and are not intended to limit the scope of this disclosure. These embodiments and variations can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of this disclosure. These embodiments and their variations are included in the scope and spirit of the invention and in the scope of the invention and its equivalents as described in the claims.

[0141] The processes and procedures described in this disclosure can be implemented not only by those expressly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in this disclosure can be implemented by implementing logic corresponding to the processes on media such as integrated circuits, volatile memory, non-volatile memory, magnetic disks, and optical storage. Furthermore, the processes and procedures described in this disclosure can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0142] Even if the processes and procedures described in this disclosure are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and / or multiple modules. Similarly, even if the various types of information described in this disclosure are described as being stored in a single memory or storage unit, such information may be distributed and stored in multiple memories on a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described in this disclosure may be implemented by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0143] 1. Processing System 100 Processing Units 200 Medical equipment 200-1 Imaging device 200-2 Irradiation device

Claims

1. A processing unit comprising at least one processor, The at least one processor, During the treatment planning stage, a reference image is obtained that includes at least a portion of the subject's body parts. A judgment image is obtained in which the aforementioned body part of the subject is included as the subject. The acquired reference image and the judgment image are input to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, respectively. Based on the judgment result showing the error obtained from the trained judgment model, position adjustment information is generated to adjust the relative position of the subject. A processing unit configured to perform a process for that purpose.

2. The processing apparatus according to claim 1, wherein the trained judgment model is generated based on training information which is a combination of a training reference image in which at least a part of a person is the subject and a training judgment image in which the part is the subject.

3. The processing apparatus according to claim 2, wherein the learning judgment image is an X-ray image in which the body part of the person is included as the subject.

4. The processing apparatus according to claim 2, wherein the learning judgment image is an image generated by applying an arbitrary positional error to the learning reference image.

5. The processing apparatus according to claim 2, wherein the learning judgment image is an image generated by applying an arbitrary positional error to a simulation image generated based on the learning reference image.

6. The processing apparatus according to claim 1, wherein the trained judgment model is generated based on training information which is a combination of a training CT image obtained from a CT image in which at least a part of a person is included as the subject, a training judgment image in which at least a part of the person is included as the subject, and training movement amount information indicating the result of a medical professional adjusting the position of the person.

7. The apparatus according to claim 1, wherein the trained judgment model is generated based on learning information which is a combination of learning information which is a combination of learning treatment plan information including setting information for a treatment beam used in treatment, learning disease location information indicating the location of a disease to be treated, and learning important site location information indicating the location information of important sites, a learning CT image obtained from a CT image in which at least a part of a person is the subject, and a learning judgment image in which the at least part of the person is the subject.

8. The apparatus according to claim 7, wherein the at least one processor is configured to perform a process for obtaining a determination result indicating the error from the trained determination model by further inputting at least one of the following to the trained determination model: treatment plan information including setting information for a treatment beam used to treat the subject; disease location information indicating the location of a disease to be treated; and important site location information indicating the location information of an important site.

9. The apparatus according to claim 1, wherein the at least one processor is configured to perform a process for increasing the resolution of a simulation image generated based on the reference image acquired in the treatment planning stage.

10. The processing apparatus according to claim 9, wherein the high-resolution enhancement is performed using a trained generative model generated based on training information which is a combination of a training image generated based on a CT image in which at least a part of a person is included as the subject, and a determination image which is a higher-resolution image than the training image and includes the part as the subject.

11. The apparatus according to claim 1, wherein the reference image is a CT image and the determination image is an X-ray image.

12. By being executed by at least one processor, During the treatment planning stage, a reference image is obtained that includes at least a portion of the subject's body parts. A judgment image is obtained in which the aforementioned body part of the subject is included as the subject. The acquired reference image and the judgment image are input to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, respectively. Based on the judgment result showing the error obtained from the trained judgment model, position adjustment information is generated to adjust the relative position of the subject. A processing program that causes the aforementioned at least one processor to function in this manner.

13. A processing method that is performed by at least one processor, The treatment planning stage involves obtaining reference images that include at least a portion of the subject's body parts, The steps include: obtaining a judgment image in which the aforementioned body part of the subject is included as the subject; The steps include inputting the acquired reference image and the judgment image, respectively, into a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, The steps include generating position adjustment information for adjusting the relative position of the subject based on the judgment result showing the error obtained from the trained judgment model, A processing method that includes this.

14. A processing system comprising at least one processor, The at least one processor, During the treatment planning stage, a reference image is obtained that includes at least a portion of the subject's body parts. A judgment image is obtained in which the aforementioned body part of the subject is included as the subject. The acquired reference image and the judgment image are input to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, respectively. Based on the judgment result showing the error obtained from the trained judgment model, position adjustment information is generated to adjust the relative position of the subject. A processing system configured to perform the necessary operations.

Images