Processing device, processing program, processing method, and processing system

Abstract

[Problem] To efficiently adjust the relative position of a patient. [Solution] In a treatment planning stage, a reference image is acquired, the reference image including, as an imaging object, at least a part of a patient. A determination image that includes, as an imaging object, the aforementioned part of the patient positioned on a treatment table for treatment is acquired. The acquired reference image and determination image are inputted into a trained determination model for determining the error in the position of the patient included in the reference image and the determination image. A process for generating control information for generating position adjustment information for adjusting the relative position of the patient on the basis of the determination result indicating the error acquired from the trained determination model is executed.

Description

Processing device, processing program, processing method, and processing system

[0001] The present disclosure relates to a processing device, a processing program, a processing method, and a processing system configured to perform processing related to controlling the relative position of a treatment subject.

[0002] Conventionally, in a radiation therapy device for detecting a disease position such as a tumor and irradiating a treatment beam to the disease position to treat the disease such as a tumor, it has been known that it is necessary to control a treatment table for positioning a patient. For example, Patent Document 1 describes "a treatment table on which a patient is placed during radiation therapy, comprising a top plate on which the patient is placed, a positioning mechanism for adjusting the position of the top plate, and a force sensor installed at the tip of the positioning mechanism, wherein the top plate has an opening directly above the force sensor, and the patient placed on the top plate is positioned while correcting the deflection of the positioning mechanism using the output of the force sensor."

[0003] Japanese Patent Application Laid-Open No. 2019-154547

[0004] Therefore, based on the above-described technology, an object of the present disclosure is to provide a processing device, a processing program, a processing method, and a processing system capable of efficiently adjusting the relative position of a subject according to various embodiments.

[0005] According to one aspect of the present disclosure, there is provided "a processing device including at least one processor, wherein the at least one processor acquires a reference image including at least a part of a subject as a subject in a treatment planning stage, acquires a determination image including the subject as a subject, inputs the acquired reference image and the determination image to a learned determination model for determining an error in the position of the subject included in the reference image and the determination image, and performs processing for generating position adjustment information for adjusting the relative position of the subject based on a determination result indicating the error acquired from the learned determination model."

[0006] According to one aspect of the present disclosure, a processing program is provided which, executed by at least one processor, acquires a reference image in which at least a portion of a subject is included as the subject during the treatment planning stage, acquires a judgment image in which the portion of the subject is included as the subject, inputs the acquired reference image and the judgment image, respectively, to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, and causes the at least one processor to function to generate position adjustment information for adjusting the relative position of the subject based on the judgment result indicating the error obtained from the trained judgment model.

[0007] According to one aspect of the present disclosure, a processing method is provided which is performed by at least one processor and includes the steps of: acquiring a reference image in which at least a part of a subject is included as the subject during the treatment planning stage; acquiring a determination image in which the part of the subject is included as the subject; inputting the acquired reference image and the determination image into a trained determination model for determining the error in the position of the subject included in the reference image and the determination image, respectively; and generating position adjustment information for adjusting the relative position of the subject based on a determination result indicating the error obtained from the trained determination model.

[0008] According to one aspect of the present disclosure, a processing system is provided comprising at least one processor, wherein the at least one processor is configured to acquire a reference image in which at least a part of a subject is included as the subject during the treatment planning stage, acquire a judgment image in which the part of the subject is included as the subject, input the acquired reference image and the judgment image, respectively, to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, and to perform a process for generating position adjustment information for adjusting the relative position of the subject based on a judgment result indicating the error obtained from the trained judgment model.

[0009] According to this disclosure, it is possible to provide a processing device, a processing program, a processing method, and a processing system that can efficiently adjust the relative position of a subject.

[0010] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved.

[0011] Figure 1A is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. Figure 1B is a diagram conceptually showing the configuration of a medical device 200 according to one embodiment of the present disclosure. Figure 2 is a block diagram showing the configuration of a processing device 100 according to one embodiment of the present disclosure. Figure 3A is a diagram conceptually showing an example of a subject management table stored in the processing device 100 according to one embodiment of the present disclosure. Figure 3B is a diagram conceptually showing an example of a CT image and a simulation image according to one embodiment of the present disclosure. Figure 4 is a diagram showing the processing flow executed in the processing device 100 according to one embodiment of the present disclosure. Figure 5A is a diagram conceptually showing an example of a position error calculation process according to one embodiment of the present disclosure. Figure 5B is a diagram conceptually showing an example of a position error calculation process according to one embodiment of the present disclosure. Figure 5C is a diagram conceptually showing an example of a position error calculation process according to one embodiment of the present disclosure. Figure 6 is a diagram conceptually showing an example of a simulation image generation process according to one embodiment of the present disclosure. Figure 7A is a diagram showing an example of a simulation image according to one embodiment of the present disclosure. Figure 7B is a diagram showing an example of an X-ray image according to one embodiment of the present disclosure.

[0012] 1. Configuration of Processing System 1 The Processing System 1 disclosed herein is a system used to adjust the relative position of a subject on a treatment table in a treatment performed by irradiating the subject with a treatment beam. In particular, the Processing System 1 acquires a reference image in which at least a part of the subject is included as the subject during the treatment planning stage. The Processing System 1 also acquires a judgment image in which the relevant part of the subject, positioned on the treatment table for treatment, is included as the subject. The Processing System 1 then inputs the acquired reference image and the judgment image into a trained judgment model, and obtains a judgment result from the trained judgment model that indicates the error in the position of the subject that may be included in the reference image and the judgment image. Based on the acquired judgment result, the Processing System 1 generates position adjustment information for adjusting the relative position of the subject.

[0013] More specifically, although not limited to the following, the processing system 1 acquires a CT image in the treatment planning stage that includes at least a portion of the subject's body as the subject. This CT image is accompanied by information indicating the diseased area identified by a medical professional or other person, or control information indicating the irradiation position and direction of the treatment beam. The processing system 1 also acquires an X-ray image in the positioning stage that includes the same body part of the subject who is positioned on the treatment table for treatment. The CT image acquired in the treatment planning stage and the X-ray image acquired in the positioning stage are then input into a trained judgment model for determining the error in the subject's position, and a judgment result indicating the error in the subject's position is obtained. Based on this judgment result, at least one of the subject's position and posture on the treatment table is adjusted to eliminate the error. In this way, the processing system 1 can accurately irradiate the subject's position, which has been adjusted on the treatment table, with a treatment beam according to the treatment plan information planned in the treatment planning stage. In other words, the processing system 1 can use the generated control information to irradiate the treatment beam to the target in a more accurate position and direction, thereby enabling treatment of that target.

[0014] The treatment beams used during the treatment phase primarily consist of radiation such as X-rays, proton beams, and heavy ion beams. During the treatment planning phase, the irradiation position and direction of the treatment beam are pre-set to ensure appropriate irradiation to the disease location identified through CT images, etc. In other words, during the treatment phase, it is extremely important that the treatment beam irradiates the pre-set irradiation location with the intensity (e.g., energy), irradiation direction, irradiation field, dose fractionation, and irradiation dose predetermined during the treatment planning phase. Conversely, during the positioning phase, which precedes the treatment phase, it is crucial to position the patient at the same location as the irradiation position of the treatment beam determined during the treatment planning phase. However, during the actual positioning phase, subtle changes in the patient's physical condition and posture over time since the treatment planning phase can cause errors between the disease location (or irradiation position) identified during the treatment planning phase and the actual disease location of the patient on the treatment table on the day of treatment. Therefore, it is necessary to correct these errors more efficiently and accurately during the positioning phase.

[0015] Generally, during the positioning stage, the similarity between the so-called DRR (Digital Reconstructed Radiograph) image generated from the CT image acquired during the treatment planning stage and the X-ray image acquired before treatment is calculated. This calculates the error between the disease location (or irradiation location) identified during the treatment planning stage and the disease location of the patient on the treatment table on the day of treatment. The patient's relative position (for example, the position of the treatment table) is then adjusted by the amount of the calculated error.

[0016] Then, an X-ray image is acquired again, and this time, a medical professional visually compares the DRR image with the newly acquired X-ray image and manually adjusts the position of the treatment table. This process of acquiring X-ray images again and adjusting the relative position of the patient (e.g., the position of the treatment table) is repeated. Once this adjustment is complete, during the treatment phase, the treatment beam is irradiated to the determined disease location (or irradiation location) in the direction set by the control information. Here, during the treatment planning phase, the disease location and the irradiation direction of the treatment beam are identified three-dimensionally in three-dimensional space using CT images taken in three dimensions. On the other hand, the judgment image (e.g., X-ray image) is represented in two dimensions. Therefore, the manual adjustments described above are performed based on the experience of medical professionals, taking into account the three-dimensional shape of the disease location and the direction of the treatment beam from the two-dimensional X-ray image. In addition, soft tissues such as internal organs may shift position or deform between the treatment planning phase and the treatment phase. Furthermore, even parts such as bones, which deform less, are affected by joint movements. Therefore, it is difficult to perfectly match the position between the treatment planning stage and the treatment stage. For these reasons, it is generally necessary for medical professionals to position the device while considering which areas to prioritize. Points to consider at this time include avoiding exposure to important healthy areas, prioritizing positional displacement and deformation in important areas (e.g., areas with disease) rather than giving much consideration to positional displacement or deformation in unimportant healthy areas (e.g., areas without the disease being treated), and positioning the device in a way that minimizes the impact of movements that occur during treatment, such as respiration, pulsation, and peristalsis.

[0017] Furthermore, DRR images, which are used for error calculation and visual comparison, are typically generated from CT images (reference images) and therefore have lower resolution than judgment images (e.g., X-ray images). This affects the accuracy and time required for similarity calculations and visual comparisons.

[0018] In this positioning stage, the processing system 1 inputs a reference image (e.g., a CT image) acquired in the treatment planning stage and a judgment image (e.g., an X-ray image) acquired on the treatment table before treatment into a trained judgment model. This allows the system to obtain a judgment result indicating the error between the disease location (or irradiation location) identified on each CT image in the treatment planning stage and the disease location of the patient on the treatment table on the day of treatment. Based on this judgment result, the processing system 1 generates position adjustment information to adjust the relative position of the patient. This enables efficient adjustment of the relative position of the treatment table. Furthermore, while conventional positioning was done while considering which locations medical professionals prioritized, using the trained judgment model allows for efficient position adjustment while taking these points into consideration.

[0019] Furthermore, the trained judgment model used in processing system 1 utilizes training information consisting of a combination of a reference image (e.g., a CT image) and a judgment image (e.g., an X-ray image). In addition, the amount of movement of the treatment table, which has been manually adjusted by a medical professional, can also be added as further training information. By using the trained judgment model trained with such training information, processing system 1 can reproduce the amount of movement adjusted by the medical professional's rules of thumb (e.g., deciding which position to prioritize when adjusting the position), enabling more efficient position adjustment.

[0020] Furthermore, processing system 1 performs high-resolution processing on the DRR image using a trained generative model. The DRR image that has undergone such high-resolution processing can be used for various purposes, such as being used as training information (virtual X-ray image) for generating a trained judgment model, being used as a DRR image for calculating similarity, or being used as a DRR image for comparison with X-ray images visually inspected by medical professionals. This allows for more accurate and rapid error adjustment and more efficient position adjustment.

[0021] Figure 1A is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. According to Figure 1A, the processing system 1 includes at least a medical device 200 which includes an imaging device 200-1 for photographing a person to be treated and an irradiation device 200-2 for irradiating the person with a treatment beam, and a processing device 100 for performing the process of generating the position adjustment information. These processing devices 100 and the medical device 200 are communicated with each other by at least one of a wireless communication network and a wired communication network.

[0022] Here, Figure 1B is a diagram conceptually showing the configuration of a medical device 200 according to one embodiment of the present disclosure. Specifically, Figure 1B is a diagram conceptually showing a radiation irradiation device, which is an example of the medical device 200 described in Figure 1A. According to Figure 1B, the medical device 200 is used to irradiate a patient lying on a treatment table 210, also called a treatment table, couch, or operating table, which is installed on the medical device 200, with a treatment beam. Therefore, the medical device 200 includes a head 200-2a of an irradiation device 200-2 that emits a treatment beam. The medical device 200 is capable of moving the gantry 200-2b on which the head 200-2a is installed two-dimensionally or three-dimensionally relative to the base 200-2c so that the irradiation direction of the treatment beam is predetermined. Note that the treatment table 210 does not need to be positioned with the patient lying down, and the patient may be positioned in various positions such as sitting or standing.

[0023] Furthermore, the medical device 200 includes, as an imaging device 200-1, an X-ray tube 200-1b for irradiating X-rays to photograph a subject, and a flat panel detector (FPD) 200-1a for imaging the X-rays irradiated from the X-ray tube 200-1b and transmitted through the subject. This captures a judgment image used to determine the error in the location of the disease in the subject.

[0024] In Figure 1B, an example of such a judgment image is an X-ray image obtained using the X-ray tube 200-1b and its flat panel detector 200-1a. However, it is not limited to X-ray images. The judgment image can be any image that can be used to determine the location of the disease, and in addition to X-ray images, for example, MR images, CT images, PET images, infrared images, 3D point cloud data, or any combination thereof can be suitably used.

[0025] Furthermore, although Figure 1B illustrates an imaging device 200-1 consisting of one set of X-ray tubes 200-1b and its flat panel detector 200-1a, a judgment image may be acquired using two sets of X-ray tubes and flat panel detectors with different X-ray irradiation directions. Alternatively, three or more sets of X-ray tubes and flat panel detectors with different X-ray irradiation directions may be installed, and a judgment image may be acquired by selecting the optimal set of one or two X-ray tubes and flat panel detectors according to the orientation of the subject, the location of the disease, the orientation of the gantry 200-2b, etc. In addition, although Figure 1B describes the use of a flat panel detector as the detector, it is also possible to use other detectors such as scintillators.

[0026] Returning to Figure 1A, in processing system 1, typically, when a judgment image captured by the imaging device 200-1 is transmitted to the processing device 100 via a communication network, the processing device 100 inputs the received judgment image and a reference image acquired during the treatment phase into a trained judgment model. The processing device 100 then obtains a judgment result from the trained judgment model that shows the error between the disease location in the judgment image and the disease location (irradiation location) in the reference image. Based on the acquired judgment result, the processing device 100 generates position adjustment information to adjust the relative position of the treatment table. Based on this position adjustment information, the relative position of the subject is adjusted. The processing device 100 also generates control information for irradiating the treatment beam to the position set during the treatment planning phase and transmits it to the medical device 200. Based on the received control information, the medical device 200 controls the treatment beam irradiated from the irradiation device 200-2. As a result, the treatment beam is irradiated to the subject, who is placed on the treatment table 210 and whose relative position has been adjusted, and the disease is treated.

[0027] In this disclosure, the processing system 1 is given as an example consisting only of the processing unit 100 and the medical device 200. However, in addition to these devices, it is also possible to combine various devices depending on the application and purpose, such as an electronic medical record device, a database device, a medical interview device, various diagnostic devices, a model generation device, and a treatment planning device. Specifically, as will be explained in detail below, the processing system 1 is divided into multiple stages, such as the treatment planning stage, the positioning stage, and the treatment stage, and each stage is carried out accordingly. For example, the learning process for generating the trained judgment model and trained generation model used in the positioning stage is performed by the server device and the model generation device. In addition, the processing related to the generation of the treatment plan in the treatment planning stage is performed by the treatment planning device. Furthermore, the processing related to positioning in the treatment stage and the processing related to the generation of control information for controlling the treatment beam are performed by the processing unit 100. By distributing the processing across multiple devices in this way, it becomes possible to, for example, perform treatment planning and learning for other patients in parallel, even while actually performing treatment by irradiating with a treatment beam.

[0028] Furthermore, in this disclosure, "subjects" can refer to any person who may be eligible for treatment, and are not limited to those with specific attributes. Therefore, such subjects may include any person, such as patients, test subjects, diagnostic subjects, and healthy individuals. In addition, the term "person" is sometimes used in this disclosure, and this may refer to subjects or to subjects plus other people.

[0029] Furthermore, in this disclosure, “disease” can be any disease that is treatable with a therapeutic beam, and is not limited to any specific disease. Examples of such diseases include a wide range of conditions, such as brain tumors, head and neck cancers, esophageal cancers, lung cancers, breast cancers, hepatobiliary and pancreatic cancers, rectal cancers, uterine cancers, prostate cancers, skin cancers, malignant lymphomas, or combinations thereof, benign tumors, leukemias, tuberculous lymphadenitis, skin diseases, gastric ulcers, keloids, hemangiomas, eye diseases, heart diseases, arteriovenous malformations, or other gynecological diseases.

[0030] Furthermore, in this disclosure, "the relative position of the subject" refers to the relative positional relationship between the subject on the treatment table and the irradiation device 200-2. Therefore, when position adjustment information is generated to adjust the relative position of the subject, the position of the treatment table or the subject located on the treatment table may be adjusted according to this information, but the position of the irradiation device 200-2 or the irradiation position of the treatment beam emitted from the irradiation device 200-2 may also be adjusted, or both may be adjusted in combination (for example, adjusting the translational deviation by adjusting the position of the treatment table or the subject located on the treatment table and adjusting the rotational deviation by adjusting the position of the irradiation device 200-2 or the irradiation position of the treatment beam emitted from the irradiation device 200-2, or vice versa).

[0031] Furthermore, in this disclosure, "control information" may refer to any information used to control the treatment beam. Examples of such control information include, but are not limited to, the following: information to instruct at least one of turning the treatment beam on or off; information to instruct at least one of the irradiation direction and intensity of the treatment beam; information indicating the disease location determined based on the assessment image; and combinations of the above information. In the following, unless otherwise specified, we will describe the case in which information to instruct at least one of turning the treatment beam on or off is used as control information.

[0032] 2. Diagram 2 of the configuration of the processing unit 100 is a block diagram showing the configuration of the processing unit 100 according to one embodiment of the present disclosure. According to Figure 2, the processing unit 100 includes a processor 111, a memory 112, and a communication interface 113. Each of these components is electrically connected to each other via control lines and data lines. The processing unit 100 does not need to have all of the components shown in Figure 2; it is possible to omit some components or to add other components. For example, it is possible to use an external memory connected in a communication manner as memory, or a database device, a server device, a terminal device, etc., together with the processing unit 100. It is also possible to distribute and execute some processing with other devices. In other words, the processing unit 100 is not limited to a single device, but also includes cases where it is distributed across multiple devices depending on the handling of information and the processing load.

[0033] Such a processing device 100 can utilize various forms of devices, such as a server device installed on the cloud, an on-premise server device installed in a specific facility or equipment, a terminal device used by medical personnel at a medical institution or other facility that operates the medical device 200, or a control device connected to the medical device 200 via a communication network.

[0034] The processor 111 functions as a control unit that controls other components of the processing system 1 based on a processing program stored in the memory 112. Specifically, the processor 111 executes processing related to generating control information for the treatment beam based on the processing program. The processor 111 executes, in particular, the following based on the processing program stored in the memory 112: "a process to acquire a reference image in which at least a part of the subject is included as the subject during the treatment planning stage," "a process to acquire a judgment image in which a part of the subject is included as the subject," "a process to input the acquired reference image and judgment image into a trained judgment model for determining the error in the position of the subject included in the reference image and judgment image," and "a process to generate position adjustment information for adjusting the relative position of the subject based on the judgment result showing the error obtained from the trained judgment model." The processor 111 is mainly composed of one or more CPUs, but a GPU or FPGA may be combined as appropriate.

[0035] Furthermore, the processes related to the generation of the trained judgment model and the trained generation model do not necessarily have to be executed by the processing unit 100, but may be executed by other devices other than the processing unit 100.

[0036] Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 112 stores instruction commands for various control of the processing system 1 according to this embodiment as programs. Specifically, Memory 112 stores programs for the processor 111 to execute, such as "a process to acquire a reference image in which at least a part of the subject is included as the subject during the treatment planning stage," "a process to acquire a judgment image in which a part of the subject is included as the subject," "a process to input the acquired reference image and judgment image into a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image," and "a process to generate position adjustment information for adjusting the relative position of the subject based on the judgment result showing the error obtained from the trained judgment model." In addition to these programs, Memory 112 also stores information for the subject management table and the subject management table. Memory 112 also stores the trained judgment model and the trained generation model as programs.

[0037] Furthermore, the memory 112 may also include a database device for storing information from the subject management table and the target person management table, via a communication network. In addition, the programs for the trained judgment model and the trained generation model do not necessarily have to be stored in the memory 112, but may be stored in the memory of another device other than the processing unit 100.

[0038] The communication interface 113 functions as a communication unit for sending and receiving judgment images and control information between the medical device 200 and other devices via a wired or wireless network. Examples of the communication interface 113 include various types of connectors for wired communication such as USB, SCSI, and wired LAN, wireless communication transceivers such as wireless LAN, Bluetooth®, LTE, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0039] 3. Other Components Although the specific configuration of the medical device 200 is not shown in the diagram, it includes a processor that functions as a control unit, a memory that functions as a storage unit, a communication interface that functions as a communication unit, etc. Each of these elements is electrically connected to one another via control lines and data lines. The medical device 200 does not need to have all of these components; it is possible to omit some of them or to add other components.

[0040] 4. Various Information Used in Processing System 1 Figure 3A is a conceptual diagram showing a subject management table stored in a processing device 100 according to one embodiment of this disclosure. According to Figure 3A, the subject management table stores CT images, treatment plan information, simulation images, disease location information, and X-ray images, respectively, associated with subject ID information.

[0041] "Patient ID information" is information used to identify each patient receiving treatment, and is unique to each patient. Patient ID information is newly generated, for example, each time a patient registers for treatment at a medical institution or each time they are registered as a patient for treatment using a therapeutic beam.

[0042] A "CT image" is an image obtained during the treatment planning stage by continuously taking cross-sectional images in the thickness direction along the opposite axis (for example, from the head to the legs) at a fixed spatial interval (e.g., slice interval = 5 mm) over a predetermined area (one or more parts or the whole body) of the human body. A CT image is an image that includes at least a part of the subject as the subject and is used as one of the reference images. Such CT images are used to calculate errors, generate simulation images, and generate treatment planning information and disease location information.

[0043] Note that disease position information or irradiation position information of a treatment beam can be given to each CT image by a medical worker or a learned specific model. Therefore, the CT image may be the image itself taken by the CT device, or the image to which the disease position information or irradiation position information is input. Also, the CT image may be image data that is the image itself taken by the CT device, or processed image data after image processing is performed on the image data. Further, the CT image may be either a still image or a moving image, and its image format may be any, such as a black-and-white image or a color image, and it may be not limited to a two-dimensional image but a three-dimensional or four-dimensional image.

[0044] Also, the reference image may be any image that can be used to determine the position error of the subject (including the subject itself, a specific part of the subject, or a part of the subject) together with, for example, a determination image using an X-ray image. Therefore, in addition to CT images, MRI images, PET images, or combinations thereof can also be suitably used as the reference image.

[0045] "Treatment plan information" is information indicating a radiation treatment plan generated based on, for example, a CT image obtained by taking a picture of a subject with a CT device prior to treatment. Such treatment plan information includes, for example, various setting information such as the intensity (e.g., energy amount) of the treatment beam, irradiation direction, irradiation field, dose fractionation, and irradiation dose. Among these pieces of information, the irradiation field indicates the irradiation position of the treatment beam and specifies the position in the three-dimensional coordinate space in a virtual body model reproduced from the CT image of the subject (including both a specific point and the range including the point). The irradiation position is specified, for example, by a medical worker such as a doctor specifying the disease position in each CT image.

[0046] The "simulation image" is an image that pseudo-reproduces a two-dimensional X-ray image by combining the CT values of each CT image along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a for a plurality of CT images continuously taken in the direction along the body axis direction. The simulation image typically uses a DRR image (Digital Reconstructed Radiograph image), but other simulation X-ray images generated by Monte Carlo simulation or the like based on CT images can also be used. Such a simulation image can also be generated, for example, using a trained machine learning model that can output a simulation image by inputting each CT image.

[0047] Here, FIG. 3B is a diagram conceptually showing an example of a CT image and a simulation image according to an embodiment of the present disclosure. Specifically, FIG. 3B is a diagram showing the relationship between the CT image stored in the subject management table of FIG. 3A and the simulation image generated from the CT image.

[0048] The CT image is typically obtained by taking cross-sections in the thickness direction in the direction from the head to the feet of the body at spatially constant intervals (for example, slice interval = 5 mm) over a predetermined range (one or more parts or the whole body) in a human body. In FIG. 3B, a plurality of CT images including at least CT images Bm-7 to CT image Bm and CT images Bn-7 to CT image Bn are taken at spatially constant intervals (for example, slice interval = 5 mm) across one or more parts.

[0049] As described above, the multiple CT images obtained in this way were taken at regular intervals in the direction from the head to the legs. Therefore, by superimposing each CT image in this direction and combining CT images at the same coordinate positions, it is possible to virtually form a three-dimensional body model. The simulation image is a pseudo-X-ray image obtained by virtually transmitting the pseudo-body model from a predetermined direction. That is, according to Figure 3B, simulation image Cm is generated as a pseudo-transmissive X-ray image of the body model reconstructed from CT image Bm-7 to CT image Bm, and simulation image Cn is generated as a pseudo-X-ray image of the body model reconstructed from CT image Bn-7 to CT image Bn.

[0050] The CT images and simulation images obtained in this manner are typically used as the CT images and simulation images shown in Figure 3A.

[0051] In addition to images generated from CT images using the method exemplified above, simulation images can also be generated from CT images using a pre-trained generative model, images drawn using operator input, simulated two-dimensional X-ray images, or pseudo-X-ray images generated from CT images using Monte Carlo simulation. Furthermore, image processing such as contrast adjustment, noise addition, and edge enhancement may be randomly applied to the generated simulation images within a predetermined range, and multiple simulation images with different contrast, noise levels, and edge characteristics may be used.

[0052] "Disease location information" is information indicating the location of a disease contained in the diagnostic image. Such disease location information is information indicating the location where a disease exists, identified by a medical professional such as a physician while referring to the CT image, and is information indicating coordinates in three-dimensional space. Specifically, it is generated by using information indicating the location of the disease where the disease exists for each CT image acquired as described above (for example, contour information such as CTV (clinical target volume), PTV (planned specimen volume), VOI (Volume of Interest), ROI (Region of Interest)). Such contour information can be generated by projecting contour information attached to the CT image in the same direction along the projection line used when generating the simulation image. Note that disease location information does not need to be identified by a medical professional as described above; it may also be identified by inputting the CT image into a trained identification model, for example.

[0053] An "X-ray image" is an image taken during the positioning stage so as to include the part of the subject that is positioned on the treatment table 210 for treatment, and is used as one of the judgment images. Such an X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a with the flat panel detector 200-1a. In other words, an X-ray image is an image that visualizes the internal structure of the subject by utilizing the different X-ray absorption rates of each tissue, such as organs and bones, inside the subject.

[0054] The judgment image can be any image that can be used to determine the error in the subject's position in comparison to a reference image, such as a CT image. Therefore, in addition to X-ray images, MR images, CT images, PET images, or any combination thereof can be suitably used as the judgment image.

[0055] Although not specifically illustrated in Figure 3A, it is also possible to add various types of information, such as medical interview information, electronic medical record information, and subject attribute information, to the subject management table in association with the subject ID information.

[0056] 5. Processing Flow Executed by Processing Unit 100 Treatment methods using therapeutic beams (e.g., radiotherapy) are important treatment methods alongside surgery and chemotherapy for tumors such as brain tumors, head and neck cancers, esophageal cancers, lung cancers, breast cancers, hepatobiliary and pancreatic cancers, rectal cancers, uterine cancers, prostate cancers, skin cancers, malignant lymphomas, or combinations thereof. However, these treatment methods require minimizing the dose irradiated to normal cells surrounding tumor cells in order to cure the tumor or alleviate symptoms. Therefore, it is necessary to create an appropriate and individualized treatment plan for each case, including the intensity of the therapeutic beam, irradiation direction, irradiation field, dose fractionation, irradiation dose, and chemotherapy to be used in combination. For this reason, these treatment methods are generally divided into a treatment planning stage, which is carried out in advance before the day of treatment; a positioning stage, which corrects the difference in the position and posture of the subject during CT imaging in the treatment planning stage and the position and posture of the subject during X-ray imaging in the treatment stage; and a treatment stage, in which the treatment is carried out.

[0057] Figure 4 is a diagram showing the processing flow performed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 4 is a diagram showing the processing flow performed by the processor 111 of the processing device 100 from the treatment planning stage to the treatment stage. This processing flow is mainly performed by the processor 111 of the processing device 100 reading and executing a program stored in the memory 112.

[0058] In Figure 4, the steps of each process are shown in chronological order, but the steps do not need to be performed consecutively. For example, after each step of the treatment planning stage is completed, the steps of the positioning stage and the treatment stage may be performed after an arbitrary period of time.

[0059] (A) Treatment planning stage According to Figure 4, in the treatment planning stage, the processor 111 receives CT images taken by the CT scanner, which include at least a portion of the subject as the subject, along with the subject ID information, from the CT scanner or PACS (Medical Image Management System) via the communication interface 113, thereby acquiring the CT images (S101).

[0060] As explained in Figure 3A, the processor 111 accepts input operations from medical professionals for each stored CT image via an arbitrary input interface, and inputs information indicating the location of the disease where the disease exists (for example, contour information such as CTV, PTV, VOI, or ROI) or information indicating the irradiation position of the treatment beam. The processor 111 then stores each CT image into which the information indicating the disease location has been input as reference information, as a reference image, and associates it with the patient ID information in the patient management table.

[0061] Furthermore, the processor 111 stacks each CT image along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a to reproduce a body model containing three-dimensional contour information and generate a simulation image. The processor 111 then stores treatment plan information set by a medical professional such as a doctor, based on the generated body model and simulation image, in the patient management table, associating it with the patient ID information (S102). As explained in Figure 3A, this treatment plan information includes various information such as the intensity of the treatment beam (e.g., energy amount), irradiation direction, irradiation field indicating the irradiation position of the treatment beam, dose division, and irradiation dose. The processor 111 also stores the generated simulation image as a reference image in the patient management table, associating it with the patient ID information. Although not specifically shown, the processor 111 may also store the acquired CT images and the body model (image data) in three-dimensional coordinate space reproduced based on the CT images, associating them with the patient ID information.

[0062] Each step in the treatment planning stage involves various considerations, such as creating a fixation device to secure the patient to the treatment table 210 and developing a treatment plan. Therefore, these steps are carried out in advance, rather than on the day of treatment.

[0063] It should be noted that, conventionally, systems using markers (e.g., tumor markers), such as the technology described in Japanese Patent Publication No. 2000-167072, have been known. Therefore, it is possible to use the technology using markers in conjunction with processing system 1. This makes it possible to switch to processing using markers if, for some reason, it is difficult to detect the location of the disease in processing system 1, which does not use markers. However, in the case of technology using markers, the marker images will be included in the CT images taken in advance to implant markers in the subject's body and in the simulation images reproduced from those CT images. If such images are used as first training medical images, the presence and location of the disease will be judged based on the presence or absence of markers, which is undesirable as training images. Therefore, in such cases, it is possible to remove the marker images from the CT images or simulation images by image processing.

[0064] (B) Positioning Stage The positioning stage is performed immediately before or on the day the treatment is performed. In this positioning stage, the procedure begins with positioning the subject on the treatment table 210 installed on the medical device 200, with the subject's movement restricted using restraints or the like. Subsequently, when an X-ray image is taken by the imaging device 200-1 of the medical device 200, the processor 111 of the processing device 100 receives the X-ray image from the medical device 200 along with the subject ID information via the communication interface 113 (S103). The processor 111 stores the received X-ray image as a determination image in the subject management table, associating it with the subject ID information.

[0065] Next, the processor 111 reads the CT image stored as a reference image in S102 and the X-ray image stored as a judgment image in S103, and calculates the error in the position of the subject included in both images (S104). Specifically, the processor 111 inputs both images into a trained judgment model in order to determine the error. The processor 111 then obtains a judgment result from the trained judgment model indicating the error in the position of the subject as an output. Details of this process by the trained judgment model will be explained in Figures 5A to 5C, etc.

[0066] Furthermore, in calculating positional errors, it is possible for medical professionals to specify in advance which parts of the body should be judged as miscalculated. Specifically, the processor 111 accepts input from any terminal device available to medical professionals and obtains information specifying any part of the body to be focused on in determining the error. This specified information may be any information indicating coordinates, the name of the body part, the outline of the body part, etc. The processor 111 then calculates the positional error at the specified body part identified by the acquired specified information. This makes it possible for medical professionals to specify in advance, for example, important parts such as disease locations and vital organs, parts that receive a large amount of radiation from the treatment beam, or parts that are easily projected in X-ray images such as bones, and to perform positioning that reflects the points to focus on in calculating the error.

[0067] When the processor 111 obtains a judgment result, it stores the judgment result information in the memory 112 and outputs the judgment result information to any terminal device accessible to medical personnel via the communication interface 113 (S105).

[0068] The judgment result may be any information indicating the amount of positional discrepancy between the subject's body in the CT image stored as the reference image and the subject's body in the X-ray image stored as the judgment image. This position may refer to the entire body of the subject, any part of the subject, or the location of the subject's disease. In other words, the judgment result or the judgment result information indicating the judgment result may include, but is not limited to, the following: • Information indicating that there is a positional error between the two images • Information indicating the amount of positional error between the two images

[0069] Next, the processor 111 determines whether the position error is within an acceptable range. This acceptable range is a pre-set range. If the error falls within this range, the processor 111 determines that no adjustment of the treatment table 210's position is necessary and proceeds to S110. On the other hand, if the error is not within the range, the processor 111 generates position adjustment information to adjust the relative position of the treatment table 210 based on the acquired determination result (S107), and outputs this position adjustment information to the medical device 200 via the communication interface 113 (S108). When the medical device 200 receives this position adjustment information via the communication interface, it adjusts the relative position of the patient by adjusting at least one of the settings, such as the position of the treatment table, the position of the irradiation device 200-2, and the irradiation direction of the treatment beam, based on the position adjustment information to correct the determined error. Then, the processor 111 returns to S103.

[0070] As described above, once the relative position of the subject is adjusted, the processor 111 again executes the process of acquiring the X-ray image in S103, the process of calculating the position error in S104, the process of outputting the judgment result in S105, and the process of determining whether the error is within an acceptable range in S106. If the processor 111 does not determine in S106 that the error is within an acceptable range, it executes the process of generating position adjustment information in S107 and the process of outputting position adjustment information in S108. As a result, the relative position of the subject is adjusted again based on the position adjustment information by adjusting at least one of the settings such as the position of the treatment table, the position of the irradiation device 200-2, and the irradiation direction of the treatment beam. In other words, the processor 111 repeats the processes in S103 to S108 and the adjustment of the relative position of the subject until the error in S106 is within an acceptable range.

[0071] The position adjustment information may include, but is not limited to, the following. Furthermore, the judgment result information may be output directly as position adjustment information. Examples include: • Information indicating the amount of adjustment to the position of the treatment table or the person on the treatment table to correct positional errors between the two images. • Information indicating the amount of adjustment to the position of the irradiation device 200-2 to correct positional errors between the two images. • Information indicating the amount of adjustment to settings such as the irradiation direction of the treatment beam to correct positional errors between the two images. • Any combination of information indicating the amount of adjustment to the position of the treatment table or the person on the treatment table, information indicating the amount of adjustment to the position of the irradiation device 200-2, and information indicating the amount of adjustment to settings such as the irradiation direction of the treatment beam to correct positional errors between the two images.

[0072] Furthermore, although not particularly essential, it is possible to add a step of visual adjustment by a medical professional during the positioning stage. Specifically, after it is determined that the error is within an acceptable range, when an X-ray image is taken again by the imaging device 200-1 of the medical device 200, the processor 111 of the processing unit 100 receives the X-ray image from the medical device 200 along with the patient ID information via the communication interface 113 (S109). The processor 111 associates the received X-ray image with the patient ID information and stores it in the patient management table as a judgment image for visual inspection. The processor 111 also reads the simulation image stored in association with the patient ID information during the treatment planning stage and outputs the X-ray image acquired for visual inspection and the simulation image to a terminal device accessible to a medical professional via the communication interface 113 (S110).

[0073] Here, the simulation image output in S110 is generated from the CT image as described above, and therefore has lower resolution than the X-ray image. This may affect the accuracy of visual confirmation by medical professionals. Therefore, the processor 111 can input the simulation image into a trained generation model and perform a high-resolution processing on the simulation image. Details of this processing will be explained in Figure 6 and other figures.

[0074] Medical personnel refer to the X-ray image and the simulation image (or the high-resolution simulation image) to check for any errors in the patient's position. If an error is found, the processor 111 accepts input from the medical personnel via an optional input interface and generates position adjustment information to adjust the position of the treatment table 210 or the patient on the treatment table 210, and outputs this position adjustment information to the medical device 200 via the communication interface 113. Upon receiving the position adjustment information via the communication interface, the medical device 200 adjusts the position of the treatment table based on the position adjustment information to correct the determined error.

[0075] The processor 111 then repeats the above processes in S109 and S110 until the medical professional accepts the error. With this, the processor 111 completes the processing in the positioning stage.

[0076] By performing this processing at the positioning stage, the coordinate space in which the X-ray image is formed and the coordinate space in which the irradiation position of the treatment beam is set correspond precisely to each other, making it possible to accurately irradiate the treatment beam to the position set at the treatment planning stage. Furthermore, as will be explained in detail in Figures 5A to 5C, a trained judgment model or a trained generation model is used to calculate errors and adjust the position at the positioning stage. Therefore, it becomes possible to perform the processing at the positioning stage more efficiently.

[0077] (C) Treatment Stage During the treatment stage, the processor 111 performs a control information generation process to generate control information for the treatment beam (S111), and transmits the generated control information to the medical device 200 via the communication interface 113 (S112). Specifically, the processor 111 reads treatment plan information from the patient management table based on the patient ID information. Then, based on the treatment plan information, the processor 111 generates control information to control the irradiation device 200-2 of the medical device 200 in order to irradiate the treatment beam to the patient's position adjusted in the positioning stage, using information such as the intensity (e.g., energy amount), irradiation direction, irradiation field, dose division, and irradiation dose of the treatment beam. The processor 111 transmits the generated control information to the irradiation device 200-2 of the medical device 200 via the communication interface 113.

[0078] When the medical device 200 receives control information, it operates the irradiation device 200-2 based on the control information and performs the process of irradiating the subject with a therapeutic beam. In this way, during the treatment phase, steps S110 to S111 in the processing unit 100 and the irradiation process in the medical device 200 are repeatedly executed according to the treatment plan information. With this, the processor 111 terminates the processing in the treatment phase.

[0079] 6. Processing related to error calculation in the positioning stage Figures 5A to 5C conceptually show an example of position error calculation processing according to one embodiment of the present disclosure. Specifically, Figures 5A to 5C show the details of the process for generating a trained judgment model used in the position error calculation processing and the error calculation processing using the generated trained judgment model. In particular, Figure 5A shows the process when an X-ray image is used as training information, Figure 5B shows the process when a virtual X-ray image is used as training information, and Figure 5C shows the process when the amount of movement of the treatment table manually adjusted by a medical professional is used as training information. Each process is mainly performed by the processor 111 of the processing unit 100 reading and executing a program stored in the memory 112.

[0080] (A) Processing when X-ray images are used as training information. According to Figure 5A, the processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as an object in the reference image and the judgment image (S210). Here, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0081] First, the processor 111 receives CT images from a CT scanner that include a part of a person (which may or may not include the subject) as learning information for deep learning (S211). Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (for example, contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112. It is also possible to use training simulation images (for example, training DRR images) generated from the CT images as training CT images.

[0082] Next, the processor 111 acquires an X-ray image from the medical device 200 by positioning the person who was the subject of the CT image on the treatment table 210 (S212). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. The X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a determination image in the memory 112, associating it with the reference image.

[0083] Next, the processor 111 reads the CT image acquired in S211 and the X-ray image received in S213 from the memory 112 as a combination of a training CT image (i.e., a training reference image) and a training X-ray image (i.e., a training judgment image) (S213). The processor 111 similarly prepares combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) for multiple people, and acquires positional error information between the two images prepared for training as the correct label. Then, the processor 111 reads the combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) acquired from multiple people, and the positional error information (correct label) between the two images, as training information.

[0084] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and location information of critical areas, which are generated based on the CT images acquired in S211, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., which areas will receive how much radiation). For example, a slight positional deviation in an area with low radiation exposure may have little effect, but even a slight deviation in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be accurately aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation with the treatment beam is undesirable (e.g., non-disease areas), and positional deviations in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0085] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learners that constitute the deep neural network (DNN) and repeats the learning process while adjusting the parameters of each neuron that constitute the deep neural network (DNN) (for example, the number of layers, the number of nodes in each layer, the connection method of nodes between layers, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix). Then, the processor 111 acquires a trained decision model (for example, the deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in the memory 112. With this, the processor 111 finishes generating the trained decision model.

[0086] In the above, the trained decision model is generated using a deep neural network (DNN). Specifically, it can also be generated using neural network methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; gradient boosting decision tree (GBDT) methods such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0087] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0088] Next, the processor 111 reads the trained judgment model generated in S210 from the memory 112 and performs position error determination processing (S220). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the location of the disease, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0089] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0090] Thus, by using a pre-trained classification model in determining positional errors, it becomes possible to make determinations more efficiently compared to methods such as calculating similarity.

[0091] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0092] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and important site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training important site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0093] (B) Processing when a virtual X-ray image is used as training information According to Figure 5B, the processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as a subject in the reference image and the judgment image (S310). Here, similar to Figure 5A, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0094] First, the processor 111 receives CT images from a CT scanner that include a part of a person (which may or may not include the subject) as learning information for deep learning (S311). Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (for example, contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112. It is also possible to use training simulation images (for example, training DRR images) generated from the CT images as training CT images.

[0095] Here, in deep learning, it is necessary to prepare a large amount of training information for learning. In Figure 5A, X-ray images taken of a person sitting on the treatment table 210 were used as training information, but there are many time constraints in efficiently gathering people with diseases and acquiring this information. Therefore, the processor 111 generates a simulation image based on the CT image acquired in S311 as a virtual X-ray image (S312). Specifically, the processor 111 generates a DRR image as a virtual X-ray image by combining CT values ​​in a direction along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a for the body model reproduced based on the CT image. In addition to this, the processor 111 generates multiple virtual X-ray images by, for example, applying an arbitrary position error to the CT image or the DRR image generated from the CT image using the following method: Method 1: The processor 111 acquires a simulation image generated by combining CT values ​​in a direction that is randomly shifted from the above direction. Method 2: The processor 111 generates an image in which the disease location identified in the CT image is randomly shifted, and acquires a simulation image based on that image. Method 3: The processor 111 generates an image in which the disease location identified in the DRR image is randomly shifted, and acquires that image as a simulation image.

[0096] In this way, the processor 111 generates multiple simulated images as virtual X-ray images, each with a pseudo-arbitrary position error applied to a set of CT images. The processor 111 stores the generated virtual X-ray images in memory 112 as judgment images, associating them with the reference images.

[0097] Next, the processor 111 reads the CT images acquired in S311 and each virtual X-ray image generated in S312 from the memory 112 as a combination of a training CT image (i.e., a training reference image) and a training virtual X-ray image (i.e., a training judgment image) (S313). The processor 111 similarly prepares combinations of training CT images (i.e., training reference images) and training virtual X-ray images (i.e., training judgment images) for multiple people, and acquires position error information between the two images prepared for training (for example, information indicating the amount of an arbitrary position error given in S312) as the correct label. Then, the processor 111 reads the combinations of training CT images (i.e., training reference images) and training X-ray images (i.e., training judgment images) acquired from multiple people, and the position error information (correct label) between the two images as training information.

[0098] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and location information of critical areas, which are generated based on the CT images acquired in S311, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., which areas will receive how much radiation). For example, a slight misalignment in an area with low radiation exposure may have little effect, but even a slight misalignment in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be accurately aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation with the treatment beam is undesirable (e.g., non-disease areas), and misalignment in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0099] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learner constituting the deep neural network (DNN), similar to Figure 5A, and repeats the learning process while adjusting the parameters of each neuron constituting the deep neural network (DNN) (for example, the number of layers, the number of nodes in each layer, the connection method of nodes between layers, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix). Then, the processor 111 acquires a trained decision model (for example, the deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in the memory 112. With this, the processor 111 finishes generating the trained decision model.

[0100] In the above, the trained decision model is generated using a deep neural network (DNN). Specifically, it can also be generated using neural network methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; gradient boosting decision tree (GBDT) methods such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0101] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0102] Next, the processor 111 reads the trained judgment model generated in S310 from the memory 112 and performs position error determination processing (S320). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the location of the disease, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0103] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0104] Thus, by using a pre-trained classification model in determining positional errors, it becomes possible to determine them more efficiently compared to methods such as calculating similarity. Furthermore, in generating the pre-trained classification model, instead of using X-ray images as training information, a virtual X-ray image with an arbitrary positional error is used, allowing for more efficient collection of training information and even more efficient determination of positional errors.

[0105] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0106] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and important site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training important site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0107] (C) Processing when the amount of movement of the treatment table manually adjusted by a medical professional is used as learning information According to Figure 5C, the processor 111 performs processing related to the generation of a trained judgment model for determining the error in the position of the subject included as a subject in the reference image and the judgment image (S410). Here, as with Figures 5A and 5B, we will explain the case in which the trained judgment model is generated by deep learning using a deep neural network (DNN).

[0108] First, the processor 111 receives CT images from a CT scanner that include a part of a person (which may or may not include the subject) as learning information for deep learning (S411). Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (for example, contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112. It is also possible to use training simulation images (for example, training DRR images) generated from the CT images as training CT images.

[0109] Next, the processor 111 acquires an X-ray image from the medical device 200 by positioning the person who was the subject of the CT image on the treatment table 210 (S412). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. The X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a determination image in the memory 112, associating it with the reference image.

[0110] As explained in S108 and S109 of Figure 4, in actual treatment, in addition to adjusting the treatment table 210 based on the calculated error, medical professionals may manually adjust the position of the treatment table 210 based on empirical rules while visually comparing the simulation image and the X-ray image. Therefore, the processor 111 acquires information indicating the amount of movement of the treatment table 210 that has been manually adjusted by the medical professional as the correct label (S413). The processor 111 stores the acquired information indicating the amount of movement of the treatment table 210 in memory 112, associating it with the reference image.

[0111] Next, the processor 111 reads the CT image acquired in S411, the X-ray image generated in S412, and the information indicating the amount of movement of the treatment table 210 acquired in S413 from the memory 112 as a combination of a learning CT image (i.e., a learning reference image), a learning X-ray image (i.e., a learning judgment image), and learning movement amount information (S414). The processor 111 similarly prepares combinations of learning CT images (i.e., learning reference images), learning X-ray images (i.e., learning judgment images), and learning movement amount information for multiple people. The processor 111 then uses the combinations of learning CT images (i.e., learning reference images), learning X-ray images (i.e., learning judgment images), and learning movement information acquired from multiple people as learning information.

[0112] Furthermore, the learning information can include at least one of the following: treatment plan information, disease location information, and location information of critical areas, which are generated based on the CT images acquired in S411, as learning treatment plan information, learning disease location information, or learning critical area location information. Including this treatment plan information enables more accurate error determination. Specifically, the treatment plan information includes information such as the irradiation position, direction, and intensity of the treatment beam (i.e., which areas will receive how much radiation). For example, a slight positional deviation in an area with low radiation exposure may have little effect, but even a slight deviation in an area with high radiation exposure can become a significant problem. Disease location information includes information indicating the location of the disease, and the treatment beam must be accurately aimed at the disease location. Critical area location information includes information indicating the location of areas where irradiation with the treatment beam is undesirable (e.g., non-disease areas), and positional deviations in these areas will have a significant impact. Therefore, this information serves as an important indicator of which locations should be prioritized for alignment. In other words, by using this information as training data, it becomes possible to generate a judgment model that takes into account which positions should be given priority when performing alignment.

[0113] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learner constituting the deep neural network (DNN), similar to Figure 5A, and repeats the learning process while adjusting the parameters of each neuron constituting the deep neural network (DNN) (for example, the number of layers, the number of nodes in each layer, the connection method of nodes between layers, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix). Then, the processor 111 acquires a trained decision model (for example, the deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained decision model as a program in the memory 112. With this, the processor 111 finishes generating the trained decision model.

[0114] In the above, the trained decision model is generated using a deep neural network (DNN). Specifically, it can also be generated using neural network methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; gradient boosting decision tree (GBDT) methods such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0115] Furthermore, although the case in which the trained decision model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained decision model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0116] Next, the processor 111 reads the trained judgment model generated in S310 from the memory 112 and performs position error determination processing (S420). Specifically, in the treatment planning stage, the processor 111 reads a CT image (i.e., a reference image) with information indicating the location of the disease, such as contour information, from the patient management table. Also, in the positioning stage, the processor 111 reads an X-ray image (i.e., a judgment image) obtained by photographing the part of the patient located on the treatment table 210 from the patient management table. Then, the processor 111 inputs the read CT image (i.e., a reference image) and the X-ray image (i.e., a judgment image) into the trained judgment model.

[0117] The processor 111 obtains information as a judgment result from the trained judgment model that indicates the error in the position of the subject (e.g., disease location) included as a subject in each image. Thus, error calculation processing is performed using the trained judgment model.

[0118] Although the position error determination process is performed by the processor 111, if a trained determination model is stored in the memory of another device such as a server device, the processor 111 may send the reference image and determination image to the server device and have the process executed by the processor of that server device.

[0119] Thus, by using a pre-trained judgment model in determining positional errors, it becomes possible to make judgments more efficiently compared to methods such as calculating similarity. Furthermore, in generating the pre-trained judgment model, information indicating the amount of manual movement of the treatment table 210 is used as training information to reflect the empirical rules of medical professionals. Therefore, it is possible to obtain judgment results with higher accuracy that reflect the empirical rules of medical professionals, and to adjust the position even more efficiently.

[0120] Furthermore, if, as a result of adjusting the treatment table 210 based on the judgment result output from the trained judgment model, the treatment table 210 is manually adjusted by a medical professional in S108 and S109 of Figure 4, the processor 111 may acquire information indicating the amount of movement of the treatment table 210 and use this information to further train the trained judgment model.

[0121] Furthermore, in addition to CT and X-ray images, the processor 111 can also input at least one of the treatment plan information, disease location information, and important site location information read from the patient management table into the trained judgment model. If the trained judgment model was generated based on the training treatment plan information, training disease location information, and training important site location information, inputting this information makes it possible to obtain information indicating the error in the location that should be emphasized as output.

[0122] 7. High-resolution processing Figure 6 is a conceptual diagram showing an example of simulation image generation processing according to one embodiment of the present disclosure. Specifically, Figure 6 is a diagram showing an example of high-resolution processing performed on a simulation image. This processing is mainly performed by the processor 111 of the processing unit 100 reading and executing a program stored in the memory 112.

[0123] Here, Figure 7A shows an example of a simulation image according to one embodiment of the present disclosure. Figure 7B also shows an example of an X-ray image according to one embodiment of the present disclosure. Specifically, Figure 7A shows an example of a simulation image that is output in S109 of Figure 4 or used as a virtual X-ray image in Figure 5B. Figure 7B also shows an example of an X-ray image that is output in S109 of Figure 4 or used in Figure 5A. As is clear from Figures 7A and 7B, since the simulation image is generated by reconstructing from a CT image, its resolution may be inferior to the X-ray image acquired during the positioning stage. Therefore, it is possible to acquire a high-resolution simulation image to adjust the position more efficiently.

[0124] As shown in Figure 6, the processor 111 executes a process related to generating a trained generative model for generating a high-resolution simulation image by increasing the resolution of the generated simulation image (S510). Here, we will explain the case in which the trained generative model is generated by deep learning using a deep neural network (DNN).

[0125] First, the processor 111 receives CT images from a CT scanner as learning information to be used for deep learning, in which a part of a person (which may or may not include the subject) is included as the subject (S511). Preferably, the CT images taken during the treatment planning stage are used. Each CT image is accompanied by information indicating the location of a disease (for example, contour information such as CTV, PTV, VOI, ROI) which has been identified by a medical professional or a trained specific model by referring to each CT image. The processor 111 stores the CT images accompanied by the information indicating the location of the disease as reference images in the memory 112.

[0126] The processor 111 then generates a simulation image (S512) by combining the CT values ​​with the body model reproduced based on the CT image in a direction along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a. The processor 111 stores the generated simulation image in memory 112, associating it with the CT image.

[0127] Next, the processor 111 receives an X-ray image from the medical device 200, which is taken with the person who was the subject of the CT image positioned on the treatment table 210 (S513). This X-ray image is generated by detecting the X-rays irradiated from the X-ray tube 200-1b along the projection line connecting the X-ray tube 200-1b and the flat panel detector 200-1a using the flat panel detector 200-1a. This X-ray image similarly includes the body parts of the person that were included as the subject in the CT image. The processor 111 stores the acquired X-ray image as a judgment image in the memory 112, associating it with the simulation image.

[0128] Next, the processor 111 reads the simulation image acquired in S512 and the X-ray image received in S513 from memory 112 as a combination of a training simulation image (i.e., training image) and a training X-ray image (i.e., training judgment image) which is the correct label (S513). The processor 111 similarly reads the combination of a training simulation image (i.e., training image) and a training X-ray image (i.e., training judgment image) which is the correct label for multiple people. Then, the processor 111 uses the combinations of training simulation images (i.e., training images) and training X-ray images (i.e., training judgment images) acquired from multiple people as training information to train the model.

[0129] As described above, when the processor 111 acquires the learning information, it provides this learning information to the learner constituting the deep neural network (DNN), and using the training X-ray images as the ground truth, it repeats the learning process while adjusting the parameters of each neuron constituting the deep neural network (DNN) (for example, the number of layers, the number of nodes in each layer, the connection method of nodes between layers, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix). Then, the processor 111 acquires a trained generative model (for example, the deep neural network (DNN) and its parameters). The processor 111 stores the acquired trained generative model as a program in the memory 112. With this, the processor 111 finishes generating the trained generative model.

[0130] In the above, the trained generative model is generated using a deep neural network (DNN). Specifically, it is also possible to generate it using neural network methods such as convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; gradient boosting decision tree (GBDT) methods such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and machine learning methods such as ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0131] Furthermore, although the case in which the trained generative model is generated by the processor 111 of the processing unit 100 has been described, it may also be generated using the processor 111 of another device, such as a model generation device. Also, although the generated trained generative model is stored in the memory 112 of the processing unit 100, it may also be stored in the memory of another device, such as a server device.

[0132] Next, the processor 111 reads the trained generative model generated in S210 from the memory 112 and performs high-resolution processing of the simulation image (S520). Specifically, the processor 111 reads the simulation image acquired during the treatment planning stage from the patient management table and inputs the simulation image into the trained generative model.

[0133] The processor 111 obtains a high-resolution simulation image, which has been refined from the trained generative model, as the generated result. Thus, high-resolution processing is performed using the trained generative model. The high-resolution simulation image thus generated is used for the adjustment of the treatment table 210 by medical personnel through visual inspection, and as training information for the trained judgment model.

[0134] Although the high-resolution processing is performed by the processor 111, if a trained generative model is stored in the memory of another device such as a server, the processor 111 may send the simulated image to the server device and have it executed by the processor of that server device.

[0135] In this way, by generating high-resolution simulation images, it becomes possible to adjust the treatment table 210 by visual inspection by medical professionals more accurately and efficiently. Furthermore, as shown in Figure 5B, by using the high-resolution simulation images as virtual X-ray images to generate a trained judgment model, the accuracy of the trained judgment model can be improved, enabling more efficient error detection.

[0136] In this embodiment, we can provide a processing device, a processing program, a processing method, and a processing system that can efficiently adjust the relative position of a subject.

[0137] 8. Others Various images such as CT images, simulation images, and X-ray images may not be image data itself, but may be image data that has undergone various preprocessing steps such as high-resolution enhancement, region extraction, noise reduction, edge enhancement, image correction, and image transformation, using filtering processes such as bandpass filters including high-pass and low-pass filters, averaging filters, Gaussian filters, Gabor filters, Canny filters, Sobel filters, Laplacian filters, median filters, and bilateral filters, vascular extraction using Hessian matrices, segmentation of specific regions (e.g., disease regions) using machine learning, trimming of segmented regions, de-haze processing, super-resolution processing, and combinations thereof.

[0138] Figure 4 illustrates the case where the position of the treatment table 210 is adjusted to correct for positional errors. However, the procedure is not limited to this; the irradiation direction and irradiation position of the treatment beam, which are set during the treatment planning stage, may also be adjusted according to the determined positional errors.

[0139] Figure 4 illustrates the case where the relative position of the subject is adjusted by outputting the generated position adjustment information to the medical device 200. However, the process is not limited to this; the processor 111 of the processing unit 100 may also output the position adjustment information to any terminal device accessible to medical personnel via the communication interface 113, and display the position adjustment information via the display of the terminal device. Based on the displayed position adjustment information, medical personnel can also manually adjust the relative position of the subject.

[0140] The embodiments and variations of this disclosure are presented as examples only and are not intended to limit the scope of this disclosure. These embodiments and variations can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of this disclosure. These embodiments and their variations are included in the scope and spirit of the invention and in the scope of the invention and its equivalents as described in the claims.

[0141] The processes and procedures described in this disclosure can be implemented not only by those expressly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in this disclosure can be implemented by implementing logic corresponding to the processes on media such as integrated circuits, volatile memory, non-volatile memory, magnetic disks, and optical storage. Furthermore, the processes and procedures described in this disclosure can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0142] Even if the processes and procedures described in this disclosure are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and / or multiple modules. Similarly, even if the various types of information described in this disclosure are described as being stored in a single memory or storage unit, such information may be distributed and stored in multiple memories on a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described in this disclosure may be implemented by integrating them into fewer components or by decomposing them into more components.

[0143] 1 Processing System 100 Processing Unit 200 Medical Device 200-1 Imaging Device 200-2 Irradiation Device

Claims

1. A processing device comprising at least one processor, wherein the at least one processor is configured to perform the following processes: acquire a reference image in which at least a part of a subject is included as the subject during the treatment planning stage; acquire a determination image in which the part of the subject is included as the subject; input the acquired reference image and the determination image, respectively, into a trained determination model for determining the error in the position of the subject included in the reference image and the determination image; and generate position adjustment information for adjusting the relative position of the subject based on the determination result indicating the error obtained from the trained determination model.

2. The processing apparatus according to claim 1, wherein the trained judgment model is generated based on training information which is a combination of a training reference image in which at least a part of a person is the subject and a training judgment image in which the part is the subject.

3. The processing apparatus according to claim 2, wherein the learning judgment image is an X-ray image in which the subject is the part of the person.

4. The processing apparatus according to claim 2, wherein the learning judgment image is an image generated by applying an arbitrary positional error to the learning reference image.

5. The processing apparatus according to claim 2, wherein the learning judgment image is an image generated by applying an arbitrary positional error to a simulation image generated based on the learning reference image.

6. The processing apparatus according to claim 1, wherein the trained judgment model is generated based on training information which is a combination of a training CT image obtained from a CT image in which at least a part of a person is included as the subject, a training judgment image in which at least a part of the person is included as the subject, and training movement amount information indicating the result of a medical professional adjusting the position of the person.

7. The apparatus according to claim 1, wherein the trained judgment model is generated based on training information which is a combination of training information which is a combination of training information which is a training CT image obtained from a CT image in which at least a part of a person is included as the subject and training judgment image which includes the at least part of the person as the subject.

8. The apparatus according to claim 7, wherein the at least one processor is configured to perform a process for obtaining a determination result indicating the error from the trained determination model by further inputting at least one of the following to the trained determination model: treatment plan information including setting information for a treatment beam used to treat the subject; disease location information indicating the location of a disease to be treated; and important site location information indicating the location information of an important site.

9. The apparatus according to claim 1, wherein at least one processor is configured to perform a process for increasing the resolution of a simulation image generated based on the reference image acquired in the treatment planning stage.

10. The processing apparatus according to claim 9, wherein the high-resolution enhancement is performed using a trained generative model generated based on training information which is a combination of a training image generated based on a CT image in which at least a part of a person is included as the subject, and a determination image which is a higher-resolution image than the training image and includes the part as the subject.

11. The apparatus according to claim 1, wherein the reference image is a CT image and the determination image is an X-ray image.

12. A processing program that causes at least one processor to function such as to acquire a reference image in which at least a portion of a subject is included as the subject during the treatment planning stage, acquire a judgment image in which the portion of the subject is included as the subject, input the acquired reference image and the judgment image, respectively, to a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, and generate position adjustment information for adjusting the relative position of the subject based on the judgment result indicating the error obtained from the trained judgment model.

13. A processing method performed by at least one processor, comprising: acquiring a reference image in which at least a portion of a subject is included as the subject during the treatment planning stage; acquiring a judgment image in which the portion of the subject is included as the subject; inputting the acquired reference image and the judgment image into a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image, respectively; and generating position adjustment information for adjusting the relative position of the subject based on a judgment result indicating the error obtained from the trained judgment model.

14. A processing system comprising at least one processor, wherein the at least one processor is configured to perform the following processes: acquire a reference image in which at least a part of a subject is included as the subject during the treatment planning stage; acquire a judgment image in which the part of the subject is included as the subject; input the acquired reference image and the judgment image, respectively, into a trained judgment model for determining the error in the position of the subject included in the reference image and the judgment image; and generate position adjustment information for adjusting the relative position of the subject based on the judgment result indicating the error obtained from the trained judgment model.

Images