Patient motion tracking system configured for automatic ROI generation

Abstract

To relate to a patient motion tracking system for automatic generation of a region of interest on a 3D surface of a patient positioned in a radiotherapy treatment room.SOLUTION: More particularly, the disclosure relates to an assistive approach of a motion tracking system, by which a region of interest (ROI) is automatically generated on a generated 3D surface of a patient. Furthermore, a method for automatically generating the ROI on the 3D surface of the patient is described. In particular, all the embodiments refer to systems incorporating methods for automatic ROI generation in a radiotherapy treatment setup.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a patient motion tracking system for automatic generation of a region of interest on a 3D surface of a patient positioned in a radiation treatment room. Specifically, the present disclosure relates to an approach to assist the motion tracking system in automatically generating a region of interest (ROI) on the generated 3D surface of the patient. Furthermore, a method for automatically generating an ROI on the 3D surface of the patient is described. In particular, all embodiments refer to a system incorporating the method for automatic ROI generation in a radiation treatment setup. [Background technology]

[0002] Selecting a region of interest (ROI) is a critical step in radiation therapy, especially in surface-guided radiotherapy (SGRT). The ROI is a region of the patient's anatomy that typically contains the target area for radiation. This ROI is often used to monitor patient registration and patient motion tracking during patient registration and treatment, respectively. Therefore, the ROI should be clinically meaningful and preferably be appropriately located relative to the patient's tumor, i.e., centered around the target area to be treated by radiation therapy. In the radiation setup, the patient is typically immobilized by restraining devices such as a head restraint, abdominal restraint, or similar device that holds the patient in place on a couch in the treatment room. For motion tracking to be fast and accurate, the ROI should preferably exclude such restraint devices and thus include only the patient's target area. It should also be of an appropriate size to ensure reliable surface registration and an adequate frame rate. Therefore, the performance of a patient motion tracking system depends on how well the ROI is defined on the patient. The ROI is typically drawn on the patient surface in imaging software by the physician treating the patient.

[0003] In existing systems, a user manually denotes an area intended to cover a region of interest, for example, with a brush stroke or a rectangular selection on a screen image of the patient. This manual process of denoting (i.e., creating) an ROI for each patient is time-consuming and prolongs treatment time for each patient undergoing radiation therapy. Furthermore, physicians must have some knowledge and skill to draw the optimal ROI for each anatomical location. Different anatomical locations may require different ROI creation, and therefore ROIs and physicians must undergo some training to know what ROI to use for each anatomical location. Therefore, there is a need to provide a simple solution that addresses at least some of the above problems and enables an optimized process for identifying and marking ROIs for each patient undergoing radiation therapy. Summary of the Invention

[0004] Accordingly, a simple patient motion tracking system for automatic generation of a region of interest (ROI) on a 3D surface of a patient is disclosed. More specifically, this disclosure describes a system and method for automatic generation of an ROI on a 3D surface in imaging software based on minimal user input. The patient motion tracking system described in various examples herein is generally configured to track at least a portion of a patient during radiation treatment. In particular, the portion of the patient tracked by the system is established as a region of interest covering a target area (i.e., cancerous tissue area) of the patient. Accurate estimation and generation of the ROI is critical to the accuracy of the motion tracking system, which is why this application particularly focuses on a system that enables automatic generation of a region of interest on a generated 3D surface of a patient positioned in a radiation treatment room.

[0005] Accordingly, in one embodiment, the system includes a memory storing region-of-interest (ROI) description data and a 3D surface generation processor configured to utilize an input surface and generate a 3D surface from the input surface. The 3D surface preferably includes at least a target area of ​​the input surface, where the target area is defined as a portion of the patient surface where cancerous tissue is concentrated. The system further includes an ROI generation processor configured to utilize the stored ROI description data and the 3D surface to output an ROI-labeled 3D surface to a display unit and a motion tracking module, where the ROI-labeled 3D surface is utilized by the motion tracking module to track patient motion during patient positioning and / or treatment in a radiation therapy room. It should be understood that an "ROI-labeled 3D surface" is a 3D surface of a patient to which an ROI has been applied. Thus, the 3D surface is generated by the 3D surface generation processor and then processed by the system to output an ROI-labeled version of the generated 3D surface. Thus, by providing a patient motion tracking system incorporating an ROI generation processor, a system is provided that can automatically generate ROIs on the generated 3D surface. The automatically generated ROIs are used by the motion tracking module to ensure that the correct part of the patient is tracked during radiation treatment. In particular, the generated ROIs on the 3D surface are used by the motion tracking module to assess whether the patient is positioned on the machine couch within a set threshold that corresponds to where the beam of the radiation treatment machine is focused.

[0006] In addition to providing accurate motion tracking of the target area, the creation of an ROI on a 3D surface also allows the amount of data used for motion tracking to be limited to the ROI, which ensures fast processing of the motion tracking module.

[0007] In an embodiment, the system further comprises a 3D scanning reconstruction system that enables generation of the input surface. The 3D scanning reconstruction system is configured to be arranged in the radiation treatment room in any suitable manner, for example, the 3D scanning reconstruction system is capable of recording data of a patient lying on a couch in the treatment room and transmitting the data to the 3D surface generation module. The 3D scanning reconstruction system may be any recording system that records the input surface of the patient, and thus the 3D scanning reconstruction system may be provided as a Kinect system, a structured light system, a LIDAR laser scanning system, a time-of-flight system, a stereo camera system, a computed tomography system, or a magnetic resonance imaging system.

[0008] It should be noted that the input surface may, in an embodiment, be a CT scan data set and is input to the 3D surface generation processor. In such a case, the CT scan data acquired from the patient during the CT scan can be used to generate a 3D surface on which the ROI is automatically drawn by the system.

[0009] In a preferred embodiment, the input surface is configured as a series of 2D image frames of at least the target area of ​​the patient, and the 3D surface generation processor is configured to generate the 3D surface from the 2D image frames. Thus, in a preferred embodiment, the 3D scanning and reconstruction system is configured as one or more cameras (e.g., a pair of cameras) configured to be positioned in the treatment room and having a field of view covering at least a portion of the patient. The 2D images recorded by the cameras are, in one embodiment, input to the 3D generation processor. The 3D generation processor then generates the 3D surface using the input 2D image frames. Thus, in more detail, the system may further comprise one or more cameras configured to be positioned in the radiation treatment room and to acquire a series of 2D image frames of at least the target area of ​​the patient.

[0010] Alternatively, the input surfaces to the 3D generation processor may also be structured as, for example, CT scan data, as described above. Thus, in such an embodiment, the 3D generation processor may be provided with input surfaces structured in RTSTRUCT DICOM format, i.e., written to disk as a list of 2D contours, which the 3D generation processor then uses to obtain the 3D surface.

[0011] The system may be configured such that the stored ROI description data includes one or more reference surfaces, each having an annotated reference ROI applied thereto, so that the ROI generation processor can automatically generate a representative ROI to be overlaid on the patient's 3D surface. Thus, one or more reference surfaces corresponding to target areas of the 3D surface may be stored in memory, along with corresponding annotated reference ROIs to be applied thereto. In this manner, one or more representative data of the target areas of the 3D surface generated are stored in the system's memory and used by the ROI generator to automatically generate an ROI that fits the 3D surface of the input surface. Various possibilities for generating the reference surface and reference ROI stored in memory are explained in detail in the detailed description of the figures. Therefore, it should be noted that the terms "reference surface" and "reference ROI" also cover data stored in the system's memory, which will hereinafter also be referred to as "template surface" and "template ROI."

[0012] In embodiments, the annotated reference ROI is based on the identification of one or more landmarks applied to each reference surface, where the landmarks represent uniquely identifiable portions of the reference surface. By providing a set of landmarks, the annotated ROI represents a unique set of features, which should preferably be compared to the 3D surface input to the ROI generation processor. This will become more apparent in the detailed description. It will become apparent below that these reference landmarks may represent a set of landmarks applied to a "training set" of reference surfaces and reference ROIs.

[0013] Additionally, in one embodiment, a set of input landmarks may be annotated onto a 3D surface that is input to the ROI generation processor. These input landmarks may be annotated onto the 3D surface, for example, by a physician, clinician, or similar person treating a patient in a treatment room. Thus, in an embodiment, the 3D surface also includes input landmark information, which may be used in the ROI generation processor, as will become clear in the detailed description. Thus, this disclosure distinguishes between reference landmarks and input landmarks, which are utilized in a "training database" and as input to the ROI generation processor, respectively.

[0014] The system described herein can be used for various target ROIs, such as ROIs generated for treatment of a patient's abdomen, head and neck, chest, etc. No matter which part of the patient's body is to be treated with radiation therapy, an ROI must be generated for the target area on the patient's 3D surface. Thus, for example, when performing stereotactic radiosurgery (SRS), it is important that the landmarks in the annotated ROI represent a representative set of data for the patient's head. Thus, in such an embodiment, the input and reference landmarks may be set as points representing the patient's left and right eyes, chin, and nose. For other parts of the patient to be treated for cancer, the input and reference landmarks annotating the ROI are selected to represent uniquely identifiable points on the target area.

[0015] As described above, it is important that the ROI generated by the ROI generation processor adequately covers the target area where the cancerous tissue is located. Therefore, the system enables a clinician, doctor, or the like to correct and adjust the automatically generated ROI on the 3D surface to ensure that the ROI adequately covers the target area. Therefore, to facilitate such adjustment, the ROI-labeled 3D surface is configured to be input to a display unit, and the display unit is configured to allow a user to adjust the region of interest via a control input to the ROI generation processor. When the user provides such control input, adjustment of at least the boundary of the ROI label on the ROI-labeled 3D surface is enabled.

[0016] As mentioned above, since the performance of a patient motion tracking system depends on how well a region of interest (ROI) is defined on the patient's 3D surface, it is also relevant to evaluate the quality of the ROI output from the ROI generation processor. That is, the ROI should preferably contain a sufficient amount of data for accurate motion tracking of the ROI. Thus, in an embodiment, the system may be further configured to evaluate the quality of the ROI by estimating the amount of data in the ROI-labeled 3D surface. Thus, in an embodiment, the ROI-labeled 3D surface is loaded into a quality module of the system, which is configured to estimate one or more geometric measurements of the 3D data in the ROI-labeled 3D surface and compare the estimated geometric measurements with one or more set thresholds.

[0017] In one example, the ROI labeled surface may have a number of data points that should be set below a set threshold for sufficient quality.

[0018] Geometric measurements should be considered as points / triangles on the ROI-labeled 3D surface, angles, curvature, size, etc. of the ROI-labeled 3D surface. Thus, appropriate metrics such as the shape of the ROI, data structure, etc. are evaluated. In one embodiment, statistical measures of surface curvature may be estimated to assess the quality of the ROI. In another alternative, the range of normals of the ROI may be estimated and compared to a set threshold.

[0019] Accordingly, the system may thus be configured with a quality module that reads the ROI-labeled 3D surface, and that is configured to estimate the amount of 3D points in the ROI-labeled 3D surface and compare the estimated amount of 3D points with a set threshold. In addition to estimating the amount of points, the above alternatives can also be used.

[0020] Regardless of the method for "assessing" the quality of the ROI, the system may be configured to assess the quality of the ROI and output an error message or the like to a user of the system indicating that the ROI automatically generated by the ROI generation processor and / or alternatively the adjusted (clinician adjusted) ROI does not contain a sufficient amount of data to be used for motion tracking and / or that the ROI contains too much data for motion tracking. Desirably, the ROI may not contain data, surfaces, curvatures, etc., that exceed a set threshold, as this may increase processing time and, therefore, motion tracking functionality.

[0021] It should be noted that the ROI generation processor detailed above is configured with a memory storing region of interest (ROI) descriptive data. The stored ROI descriptive data may include various descriptive information of the surface and ROI used for generating the ROI on the 3D surface. Various descriptive ROI data are available in connection with the patient motion tracking system described herein and are described in further detail in the following description of the figures.

[0022] Embodiments of the present disclosure can be best understood from the following detailed description considered in conjunction with the accompanying drawings. The figures are schematic and simplified for clarity, showing details merely to aid in understanding the claims, while other details are omitted. The same reference numerals are used throughout for the same or corresponding parts. Individual figures of each embodiment may be combined with any or all figures of the other embodiments, respectively. These and other embodiments, features and / or technical advantages will be apparent from and elucidated with reference to the examples set forth hereinafter. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 shows a schematic diagram of a patient motion tracking system according to a first example for automatic generation of ROIs on a 3D surface. [Figure 2]FIG. 10 shows a schematic diagram of a patient motion tracking system according to a second example for automatic generation of ROIs on a 3D surface. [Figure 3] FIG. 1 shows a schematic diagram of a patient motion tracking system according to a second example for the automatic generation of ROIs on a 3D surface, particularly in a stereotactic radiosurgery setup. [Figure 4] FIG. 10 shows a schematic diagram of a patient motion tracking system according to a third example for automatic generation of ROIs on a 3D surface. [Figure 5] FIG. 10 shows a schematic diagram of a patient motion tracking system according to a fourth example for automatic generation of ROIs on a 3D surface. [Figure 6] FIG. 10 shows a schematic diagram of a patient motion tracking system according to a fifth example for automatic generation of ROIs on a 3D surface. DETAILED DESCRIPTION OF THE INVENTION

[0024] The detailed description set forth below in conjunction with the accompanying drawings is intended as a description of various configurations of a patient motion tracking system. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, as will be apparent to those skilled in the art, these concepts may be practiced without such specific details. Some aspects of the apparatus and methods are described in terms of various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). Depending on the application, design constraints, or other reasons, the elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0025] To set the stage, current approaches for drawing ROIs on the 3D surface of a patient in a radiation therapy setup will first be described. Accordingly, current approaches involve a radiation therapy clinician, physician, or other trained individual drawing an ROI on the patient's CT scan. The drawn ROI is then used to position the patient in the treatment room. Once the patient is in the treatment room, a reference capture of the patient is taken by a patient tracking monitoring system, and the ROI drawn on the CT scan is transferred to the reference capture. The ROI on the reference capture is then used for patient monitoring. Thus, the ROI is drawn during the planning phase of the treatment process and is not directly correlated to the patient's actual position in the treatment room. It should be noted that the ROI may optionally be edited on the reference capture prior to using the ROI for patient monitoring. Thus, current methods known within the art utilize a manual approach in which a clinician, physician, or the like manually draws relevant regions of interest on the target area of ​​the CT scan. This manual approach is precisely what the present disclosure seeks to avoid, in order to provide a more accurate automated approach for ROI generation while simultaneously optimizing the time used per patient undergoing radiation therapy. With such existing methods, a risk of error is introduced into the motion tracking module, for example, when using ROIs drawn on a CT scan rather than actual 3D surfaces generated in the treatment room. This is because it is not certain that the patient will be positioned on the couch in the treatment room in exactly the same position as when the CT scan was acquired. Thus, the ROI may have changed, which affects the accuracy of patient motion tracking taking into account the target area.

[0026] Therefore, with reference to FIG. 1 , the present disclosure seeks to solve these and other problems of current systems by providing a patient motion tracking system 1 configured for automatic generation of a region of interest on a 3D surface of a patient positioned in a radiation treatment room (or, alternatively, the patient may be positioned, for example, in a CT scan room (not shown in further detail)). The patient motion tracking system 1 includes a memory 2 storing region of interest (ROI) description data 3. The system further includes a 3D surface generation processor 4 configured to utilize an input surface 5 and generate a 3D surface from the input surface 5, the 3D surface comprising at least a target area of ​​the input surface. The 3D surface is input to an ROI generation processor 6. The ROI generation processor 6 is configured to utilize the stored ROI description data 3 and the 3D surface to output an ROI-labeled 3D surface 7 to a display 8 and a motion tracking module 9. In this manner, the ROI-labeled 3D surface 7 is used by the motion tracking module 9 to track the patient's motion during patient positioning and / or treatment in the treatment room.

[0027] 1 shows that the memory 2 may form part of the ROI generation processor 6. It should be noted that the memory need not necessarily be part of this specific part of the system 1, but may be held anywhere in the patient motion tracking system.

[0028] 1, the system 1 comprises a 3D scanning reconstruction system 10 configured to be placed in a radiation treatment room and configured to generate an input surface 5. As explained in the overview section, the 3D scanning reconstruction system can be, for example, a camera such as a stereoscopic camera, which may be a system such as a Kinect setup or any other suitable system that generates a data stream from which a 3D surface can be generated.

[0029] In an embodiment of the system, not shown in further detail, the input surface 5 may be configured as a series of 2D image frames of at least the target area of ​​the patient. Thus, the input surface 5 may be generated in any suitable manner as "images" or similar "reconstruction data" of the patient, which are input to a 3D surface generation processor that generates a 3D surface from the 2D image frames or other suitable reconstruction data. In a preferred embodiment, the input surface is a set of 2D images captured from one or more cameras, such as stereoscopic cameras, mounted in the treatment room.

[0030] In an alternative embodiment, the input surface may be configured as CT scan data of the patient, as described above, the scan data being acquired during the pre-planning phase of the radiation treatment. When using CT scan data as the input surface, the 3D generation processor is configured to generate a 3D surface from the CT scan data. The CT scan data, in this embodiment, does not include the previously generated ROI and is configured as raw CT scan data.

[0031] In a further alternative, the input surface may consist of data received from a LIDAR sensor in the form of a point cloud onto which a 3D surface can be fitted.

[0032] In a preferred embodiment utilizing one or more cameras mounted in a treatment room, the cameras are configured to acquire a series of 2D image frames of at least a target area of ​​the patient. Furthermore, these 2D cameras are configured to capture a 2D image stream of the patient lying on a couch continuously during treatment, following automatic generation of an ROI on the 3D surface. The 2D image stream is input to a motion tracking module and utilized in conjunction with the ROI-labeled 3D surface to track any possible patient motion by comparing the 3D surface generated based on the continuous receipt of the 2D image stream by the motion tracking module with the ROI-labeled 3D surface.

[0033] More specifically, the stored ROI description data includes one or more reference surfaces, each having an annotated reference ROI applied to it. Details of the reference surfaces and the reference ROIs will become clearer when describing more specific embodiments relating to the ROI description data below. Furthermore, the annotated reference ROIs are based on the identification of one or more landmarks applied to each of the reference surfaces, where the landmarks represent uniquely identifiable portions of the reference surfaces. It should be noted that two different sets of landmarks may be used in the following exemplary embodiments. In some embodiments, landmarks are simply applied to the reference ROI stored in memory as ROI description data. However, in other embodiments, in addition to the reference landmarks, a set of input landmarks may also be applied to the 3D surface input to the ROI generation processor.

[0034] As represented schematically in Figure 1, the ROI-labeled 3D surface 7 is configured to be input to a display unit 8. The display unit 8 is then configured to allow a user to adjust the region of interest via control input to the ROI generation processor 6. The control input from the user thus makes use of adjustment of at least the boundaries of the ROI labels of the ROI-labeled 3D surface 7.

[0035] As further shown in FIG. 1 , the system 1 may further include an ROI quality module 11. The ROI quality module 11 is configured to take the ROI-labeled 3D surface 7 as input and evaluate the quality of ROI labels forming part of the ROI-labeled 3D surface 7. More specifically, the ROI-labeled 3D surface 7 is read into the ROI quality module 11, which then estimates geometric measurements of the 3D data in the ROI-labeled 3D surface 7, such as size or curvature, and compares the estimated geometric measurements with one or more set thresholds. The magnitude of the threshold may be set based on the amount of data contained within the ROI labels of the ROI-labeled surface, i.e., the size of the ROI-labeled surface. Alternatively, the magnitude of the threshold may be based on a percentage index between the amount of reference data required for the ROI-labeled 3D surface to be used for motion tracking and the actual amount of data in the generated ROI-labeled 3D surface. In either case, it is important that the set threshold reflects that the ROI-labeled 3D surface is within the maximum value of the ROI; exceeding the maximum value would result in too significant an impact on the frame rate. More specifically, in the context of the ROI quality module, in an embodiment, an ROI is checked to ensure that it has fewer than 10,000 triangles (alternate points, curvatures, normals, etc.) to be considered an acceptable ROI. The reason for selecting a set threshold below the maximum value for the ROI-labeled 3D surface is that an ROI-labeled 3D surface that is too large is unlikely to be clinically relevant; such a large surface would also slow down the software and interfere with patient monitoring. Therefore, the threshold for the quality of the ROI-labeled 3D surface should be set to balance the advantages and disadvantages of providing the motion tracking system with a sufficient amount of data to accurately monitor and track the patient, while at the same time being within the data limits that do not slow down the processing performed by the motion tracking module software. Alternatively, a geometric measure, such as surface curvature, can be compared to the set threshold to ensure that the selected ROI contains sufficient geometric detail for the performance of the motion tracking system.

[0036] When the ROI-labeled 3D surface is evaluated by the ROI quality module, the ROI quality module is configured to output an evaluation to the user. The evaluation may be configured as one of two or more results. In one scenario, the ROI quality module outputs an indicator to the user that the ROI-labeled 3D surface is appropriate for motion tracking, or in another scenario, the ROI quality module outputs an indicator to the user that the ROI-labeled 3D surface should be adjusted to generate a more appropriate ROI-labeled 3D surface for motion tracking. Thus, the ROI quality module is configured to provide feedback (represented as arrow 12) to the display module when the ROI-labeled 3D surface is approved for further motion tracking. It should be noted that, in an embodiment, the ROI-labeled 3D surface is input to the motion tracking module 9 only when the ROI-labeled 3D surface is “approved,” i.e., evaluated by the quality module 11 as appropriate for motion tracking. This ensures that the ROI-labeled 3D surface contains the exact amount of data required for accurate motion tracking.

[0037] Referring now to FIG. 2, an example embodiment of the ROI generation processor 106 is shown in more detail. In this embodiment, the ROI generation processor 106 is configured to take as input a 3D surface of a patient lying on a couch in a treatment room, as described above in general terms. The 3D surface is generated from the input surface 105, as described above. The input surface 105 is input to the 3D surface generation processor 104 of the system 1. Furthermore, the input surface 105 is acquired by one of the selected 3D reconstruction systems 110, such as one or more cameras. Naturally, therefore, some features of the system described in the embodiment of FIG. 1 are similar to those of the embodiment in FIG. 2. Therefore, only the main differences will be described in detail. The generated 3D surface is represented in FIG. 2 as surface 105a, which is input from the 3D surface generation module 104 to the ROI generation processor 106 (input path represented by arrow 106a).

[0038] 2, the ROI generation processor 106 comprises a memory 102 configured with ROI description data 103a, 103b. The ROI description data 103a, 103b are more particularly configured as a template surface 103a and a template ROI 103b, which are configured to be input to the ROI generation processor 106. Thus, the memory 102 can form part of the ROI generation processor 106 and / or be a separate unit in the system. Furthermore, the template surface 103a and the template ROI 103b can be configured as a reference surface and a reference ROI in this embodiment.

[0039] The ROI generation processor 106 is configured to align and warp the template ROI 103b and template surface 103a with the 3D surface 105a to generate at least a warped ROI 113, and then transfer the warped ROI 113 to the 3D surface 105a (indicated by arrow 115).

[0040] 2, the system may also include a training module 120 configured to generate and output to memory 102 a template surface 103 a and a template ROI 103 b. Thus, the training module 120 may form part of the system, but may also be utilized as a separate component and, as such, be remote from the system. In either case, the training module 120 has stored template ROI 103 b and template surface 103 a, and is used to generate a representative dataset of the ROI that describes what the ROI will look like on an example target surface (i.e., a 3D surface).

[0041] Thus, in an embodiment, the system may include a training module 120 as described, having two or more reference target surfaces 121a, 121b, 121c, and 121d, each having an annotated reference ROI 122a, 122b, 122c, and 122d applied thereto, as shown in FIG. 2 . The training module is configured to align the two or more reference target surfaces 121a, 121b, 121c, and 121d and then calculate an average of the aligned reference target surfaces to generate the template surface 103a. Furthermore, the training module is also configured to calculate an average of the annotated reference ROIs 122a, 122b, 122c, and 122d to generate the template ROI 103b. The generated template ROI 103b and template surface 103a are then stored in the system's memory and subsequently used by the ROI generation processor 106 to generate the ROI-labeled 3D surface 107, as described above. The generation of the ROI-labeled 3D surface 107 is as described above. Also as described in relation to Fig. 1, this ROI-labeled 3D surface 107 is output to the display module 108 and / or the motion tracking module 109, which then uses the ROI-labeled 3D surface 107 for tracking the patient's motion in the radiation treatment setup. Moreover, as described in the general embodiment of Fig. 1, the system in this embodiment also has a similar quality module 111, which returns feedback 112 and evaluation of the generated ROIs to the user via the display module 108.

[0042] Referring now to FIG. 3, a further example of a system substantially similar to that described in connection with FIG. 2 is shown. The embodiment shown in FIG. 3 represents a system configuration in a stereotactic radiosurgery treatment situation. In this situation, treatment is focused on a treatment region of the brain. In such a system, a patient is initially positioned on a couch in a treatment room. A head mask or similar head restraint is then utilized to immobilize the patient. In such treatment, the region of interest to be tracked by the motion tracking module is preferably centered around the patient's face. Thus, in the illustrated example, the data used for 3D surface generation consists of a 3D reconstruction image (also represented as input surface 205) of the patient lying on the couch, such as a camera (represented by 10) image and / or CT scan data, from which the 3D surface generator 204 is configured to generate a 3D surface 205a of the patient's face lying on the couch. This 3D surface 205a is input to ROI generation processor 206, which utilizes template surface 203a and template ROI 203b stored in memory 202 to generate an ROI-labeled 3D surface 207 of the patient in the same manner as described in connection with Figure 2. Template ROI 203b and template surface 203a are generated in the same manner as described above in connection with training module 120 of Figure 2. Thus, in the embodiment shown in Figure 3, the same features of training module 120 in Figure 2 have been given their reference numbers increased by 100.

[0043] 2 and 3, it should be noted that the template ROI 203b may be generated from the annotation of reference landmarks onto the surfaces of multiple different reference surfaces 221a, 221b, 221c, 221e, 221f in the training module 220. Thus, the reference surface 221a in the training module 220 may have annotated landmarks that define the ROIs 222a, 222f, 222d, 222c of each of the reference surfaces. In this manner, the template ROI 203b in the template surface 203a similarly has reference landmarks based on the averaging and alignment 223 of the reference surfaces and reference ROIs in the training module.

[0044] In an embodiment, the reference landmarks may be utilized along with a set of input landmarks 224a, 224b, 224c, 224d that define markers on the patient's 3D surface 205a. Accordingly, the system may prompt the user with the option of displaying one or more of the input landmarks 224a, 224b, 224c, 224d on the 3D surface 205a. The 3D surface 205a is then used by the ROI generation processor 206 to align and warp at least the template ROI 203b to the 3D surface 205a, thereby generating the ROI-labeled 3D surface 207.

[0045] In the specific example of the SRS setup just described, the reference landmarks and input landmarks may be configured as points representing the patient's left and right eyes, chin, and nose, as represented by black dots 224a, 224b, 224c, 224 in FIG. 3.

[0046] Furthermore, the ROI-labeled 3D surface 207 is input to a display module 208 and to a motion tracking module 209, as described above. Moreover, even though not shown, the embodiment of Figure 3 should also be considered to have the quality module described in connection with Figure 1.

[0047] It should be noted that another embodiment of the system is depicted in FIG. 4, and the general features described in relation to FIG. 1 apply. Therefore, only the main differences of the system will be described in further detail. The embodiment of FIG. 4 represents a patient tracking system according to FIG. 1, in which the ROI generation module 306 is configured with a trained ROI model 325. Thus, in this embodiment, the ROI description data is configured as the ROI model 325. The ROI model 325 is trained in a machine learning processor, i.e., the training module 326, before being stored in the memory 302 of the ROI generation processor 306 or alternatively in another unit or module of the system. Thus, the system may be configured with the training module itself, or may be configured to store a pre-trained ROI model. In the latter case, the pre-trained ROI model may be generated in another system and then maintained in the patient motion tracking system.

[0048] More specifically, the ROI model 325 is trained based on one or more reference surfaces 321 a, 321 b, 321 c, and 321 d, to which annotated reference ROIs 322 a, 322 b, 322 c, and 322 d are respectively applied. Thus, it should be noted that in all embodiments described herein, the reference surfaces and reference ROIs are utilized as training data for generating ROI models and / or ROI templates and surfaces, which are used as inputs to the memory of the ROI generation processor. It should be noted that the training module may be configured as a neural network.

[0049] More specifically, the reference surfaces in the training module 326 are configured, in the illustrated embodiment, as depth map and normal map surface representations of the input surfaces. That is, in the embodiment depicted in FIG. 4, the reference surfaces 321a, 321b, 321c, and 321d may be configured, for example, as depth map 327a and normal map 327b of the surfaces used for training the neural network. While only one depth map 327a and one normal map 327b are shown for illustrative purposes, it should be noted that the training procedure may involve utilizing multiple reference surfaces, including a depth map and a normal map for each reference surface. Thus, in one embodiment, the model may be trained in the training module 326 based on the data just described. However, other suitable data for training may also be contemplated. The data used for training should be any suitable data that provides a representative way of describing the patient's target surface and the ROI of interest of a given body part of the patient (e.g., head, abdomen, arm, leg, etc.).

[0050] In the embodiment of Figure 4, the ROI generation processor is configured, similar to that described above, to use the 3D surface 305a in this case as input to the ROI model 325 (configured as a trained neural network) and to output the ROI-labeled 3D surface 307 to the display 308 and / or the motion tracking module 309 and / or to a quality module (not shown) as described above.

[0051] More specifically, the trained model 325 is stored in and utilized by the ROI generation processor 306 to generate the ROI-labeled 3D surface 307. This is done by inputting the 3D surface 305a into the ROI model 325, which is configured to classify the vertices of the 3D surface 305a as being inside or outside the region of interest defined by the trained model according to the model depth map and normal map representations. It should thus be understood that the trained model includes a feature representation of the region of interest and a feature representation of the surface related to such region of interest. In this manner, the 3D surfaces input to the model can be classified based on their feature representations, and based on the classification, the model is configured to output an appropriate ROI label for the 3D surface input to the model.

[0052] In the embodiment of FIG. 4 , the trained model 325 may be a machine learning model configured as a conventional neural network, where convolution is applied to the 3D surface by defining a local 2D coordinate system for each point on the 3D surface and locally resampling the surface in alignment with the convolution kernel. The output of the model is a classification of points as being inside or outside the region of interest defined by the trained model. In a more detailed embodiment, the convolution may be applied to a mesh of the 3D surface by defining a local 2D coordinate system for one or more vertices on the 3D surface. In this way, the output of the model is a classification of vertices as being inside or outside the region of interest defined by the trained model. In further detail, the mesh may be configured as a point cloud, where the points of the point cloud are used to define a face, e.g., a triangle or a rectangle. It should be noted that this description is just one suitable method of creating a model using data. Other suitable approaches should be considered within the scope of this disclosure.

[0053] Furthermore, in another example embodiment according to FIG. 4, a feature vector may be computed for each point of the reference surface as a description of their local neighborhood in terms of geometry or texture, and these features are used as input to a machine learning model to classify each point as being inside or outside a region of interest defined by the model.

[0054] In yet another embodiment, a motion tracking system is configured to utilize a combination of the features of the above systems to automatically generate a ROI-labeled 3D surface. Such an embodiment is illustrated in FIG. 5, which shows how the trained model 425 described in connection with FIG. 4 can be used, for example, to generate a set of input landmarks to be applied to the 3D surface 405a. Thus, instead of a clinician annotating the input landmarks 424a, 424b, 424c, and 424d as described in connection with FIGS. 2 and 3, the trained landmark model 425 can be used to identify relevant input landmarks 424a, 424b, 424c, and 424d in the 3D surface 405a. The input landmarks 424a, 424b, 424c, and 424d are used by the ROI generation processor 406 as described in connection with FIGS. 2 and 3.

[0055] Thus, in more detail, in the embodiment of FIG. 5, the ROI generation processor 406 includes a landmark generation model 425, which is trained, for example, by a machine learning processor, before being stored in the ROI generation processor 406. The landmark generation model 425 is trained, for example, based on one or more reference surfaces to which annotated landmarks are respectively applied. In this embodiment, the annotated reference ROIs used in accordance with the description of FIGS. 2 and 3 are represented by 422a, 422b, 422c, and 422d. Thus, the main difference between the embodiment of FIG. 4 and the embodiment of FIGS. 2 and 3 is only that the input landmarks are found automatically by use of the trained model. The training module 420 corresponds to the description of the training modules 120 and 220 of FIGS. 2 and 3, and reference is made to those descriptions.

[0056] Thus, in this embodiment, landmark generation model 425 is configured to output representations of input landmarks 424a, 424b, 424c, and 424d on 3D surface 405a, thereby generating a landmark-labeled 3D surface. Landmark-labeled 3D surface 405a is then utilized by ROI generation processor 406, along with template surface 403a and template ROI 403b, to align and warp template ROI 403b and template surface 403a with landmark-labeled 3D surface 405a to generate aligned and warped ROI 413. Aligned and warped ROI 413 is then transferred to 3D surface 405a to output ROI-labeled 3D surface 407. Template ROI 403b and template surface 403a may be generated by training module 420 as described in connection with FIGS. 2 and 3, and details thereof in FIG. 5 can be found in the descriptions of FIGS. 2 and 3. Furthermore, a 3D surface 405a is generated from the input surface 405 in a 3D surface generator 404, also as described in the above embodiment.

[0057] 5 results in a fully automated ROI generation system, achieved without any user input, by using trained landmarks to identify landmarks in the 3D surface required for the ROI generation processor to identify an appropriate ROI for motion tracking by the motion tracking module 409 from stored ROI templates. Furthermore, in accordance with all embodiments described herein, this ROI-labeled 3D surface can be evaluated by the user (via the display 408) and / or automatically by the system by utilizing the ROI quality module, as described above.

[0058] In a further embodiment, depicted in Fig. 6, the ROI generation processor 506 is configured to utilize another method for automatically generating ROI-labeled 3D surfaces for an input 3D surface. Thus, as depicted in Fig. 6, the system comprises a memory 502 having a set of reference surfaces 521a, 521b, 521c to which annotated reference ROIs 522a, 522b, 522c, respectively, are applied, corresponding to the above embodiment. The reference surfaces and reference ROIs represent a set of ROI description atlases stored in the memory 502 and used as input to the ROI generation processor 506.

[0059] Thus, when the patient's 3D surface 505a is input to the ROI generation processor 506, the 3D surface image 505a is input to a registration and similarity index module 531. Module 531 also takes as input a set of reference ROIs and reference surfaces from memory 502. Registration and similarity index module 531 is configured to perform registration and similarity index between the 3D surface 505a and the reference ROIs and reference surfaces, to output a ranking 532 of the reference surfaces 521a_r, 521b_r, 521c_r and their corresponding reference ROIs 522a_r, 522b_r, 522c_r with respect to the 3D surface 505a input to the ROI generation processor 506. From the ranking 532, the ROI generation processor 506 is configured to select the N closest reference surfaces and their corresponding reference ROIs to the 3D surface 505a.

[0060] In the illustrated example, the features used in the memory atlases to calculate the similarity index with the input surface 505a are depicted as being the reference ROI and the reference surface. Note that additional features associated with the patient's target region, such as gender, height, weight, age, physical condition descriptors, etc., may be used as descriptive features for ranking and selecting the atlases. One or more of such features may be utilized alone or in combination with the example shown in FIG. 6.

[0061] Once the N closest reference surfaces, each with its own ROI, have been identified 528, the ROI generation processor 506 is configured to perform registration 529 (i.e., non-rigid registration) between the N closest reference surface instances 528 and the 3D surface 505a. The registered atlas ROIs are then used by an ROI fusion module 530 of the ROI generation processor 506 to generate an ROI-labeled 3D surface 507, which is output to the display module 508 and the motion tracking module 509. More specifically, there is a similarity between the template approach (i.e., FIGS. 2 and 3) and the atlas approach (FIG. 6) in that the top N selected atlases can be considered as templates. Each of these templates defines an ROI for the input surface, and the ROI fusion module combines the multiple ROIs into a single ROI, e.g., by majority voting.

[0062] The embodiment described in connection with Figure 6 is another approach to automatically generating ROI labels for the 3D surface of a patient in this manner. In the example of Figure 6, the procedure and system are described in an SRS setup, but may be utilized with any suitable radiation treatment in which target areas are designated around other parts of the patient's body. An important feature of all embodiments described herein is that the "training data," either stored directly in memory or trained remotely from the system, is, as such, based on an appropriate representation of the portion of the patient to be treated by the radiation treatment.

[0063] It should be noted that in the embodiments described herein, processors and modules are described. The processors and / or modules may be configured as one or more computer-readable media. Thus, functions may be stored or encoded on a tangible computer-readable medium as one or more instructions or code. A computer-readable medium includes a computer storage medium adapted to store a computer program having program code that, when executed on a processing system, causes the processing system to perform at least a portion (e.g., most or all) of the steps of the methods described herein.

[0064] By way of example, and without limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium usable to carry or store desired program code in the form of instructions or data structures and accessible by a computer. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs; disks typically reproduce data magnetically, while discs reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media. In addition to being stored on tangible media, computer programs can also be transmitted over transmission media such as wired or wireless links or networks, e.g., the Internet, and loaded into a data processing system for execution at a location different from the location of the tangible media.

[0065] Further, the data processing system and / or generation module may have a processor adapted to execute a computer program that causes the processor to perform at least some (e.g., most or all) of the steps of the system configurations described in this specification and claims.

[0066] It is contemplated that structural features of the devices described in the detailed description and / or claims may be combined with steps of configuring the system when appropriately substituted by corresponding processes.

[0067] As used, the singular forms "a," "an," and "the" are intended to include the plural (i.e., to mean "at least one") unless expressly stated otherwise. It will be further understood that the words "includes," "comprises," "including," and / or "comprising," when used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will also be understood that when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, but unless expressly stated otherwise, intervening elements may also be present. Furthermore, as used herein, "connected" or "coupled" may include being wirelessly connected or coupled. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the precise order set forth herein, unless expressly stated otherwise.

[0068] It should be understood that references throughout this specification to features included as "one embodiment" or "embodiment" or "aspect" or "may" mean that the particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Furthermore, particular features, structures, or characteristics may be combined as appropriate in one or more embodiments of the present disclosure. The above description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to those aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects.

[0069] The claims are not limited to the embodiments shown herein but are to be accorded the full scope consistent with the language of the claims. In the claims, reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather means "one or more." Unless specifically stated otherwise, the term "some" refers to one or more.

[0070] Accordingly, coverage should be determined with reference to the claims that follow. [Explanation of symbols]

[0071] 1. Patient Motion Tracking System 2,102,202,302,402,502 memory 3 Region of Interest (ROI) description data 4,104,204,404,504 3D Surface Generation Processor 5,105,205,405 Input Surface 6,106,206,306,406,506 ROI Generation Processor 7,107,207,307,407,507 3D surfaces with ROI labels 8,108,208,308,408,508 Display 9,109,209,309,409,509 Motion Tracking Module 10,110 3D scanning reconstruction system 11,111 ROI Quality Modules 103a, 203a, 403a Template surface 103b, 203b, 403b Template ROI 105a,205a,305a,405a,505a 3D surface 113,213,413 Warped ROI 120,220,326,420 Training Module 121,221,321,421,521 Reference Surface 122,222,322,422,522 Annotated Reference ROIs 224,424 input landmarks 325 ROI Model 327a Depth Map 327b Normal Map 425 Landmark Generation Model

Claims

1. 1. A patient motion tracking system for automatic generation of a region of interest on a 3D surface of a patient, comprising: a memory storing region of interest (ROI) description data; a 3D surface generation processor configured to obtain a 3D surface including at least the target area; an ROI generation processor configured to utilize the stored ROI description data and the 3D surface to output an ROI-labeled 3D surface to a motion tracking module; and The ROI-labeled 3D surface is utilized by the motion tracking module to track the patient's motion during patient positioning and / or treatment in the treatment room. Patient motion tracking system.

2. a 3D scanning and reconstruction system configured to be positioned in a radiation treatment room and configured to generate an input surface; the 3D surface is generated from the input surface; The patient motion tracking system of claim 1 .

3. the input surface is constructed as a series of 2D image frames of at least the target area of ​​the patient; the 3D surface generation processor is configured to generate the 3D surface from the 2D image frames; The patient motion tracking system of claim 2 .

4. and one or more cameras configured to be positioned in the radiation treatment room and to acquire the series of 2D image frames of at least the target area of ​​the patient. The patient motion tracking system of claim 3 .

5. the stored ROI description data comprises one or more reference surfaces, each reference surface having an annotated reference ROI applied thereto; A patient motion tracking system according to any one of claims 1 to 4.

6. the annotated reference ROI is based on the identification of one or more landmarks applied to each of the reference surfaces, the landmarks representing uniquely identifiable portions of the reference surfaces; The patient motion tracking system of claim 5 .

7. the ROI-labeled 3D surface is configured to be input to a display; the display is configured to allow a user to adjust the region of interest via control inputs to the ROI generation processor; the control input utilizes adjustment of at least a boundary of a ROI label of the ROI-labeled 3D surface; A patient motion tracking system according to any one of claims 1 to 6.

8. The ROI-labeled 3D surface is loaded into a quality module of the system; the quality module is configured to estimate one or more geometric measures of the 3D data in the ROI-labeled 3D surface and compare the estimated geometric measures to one or more set thresholds. A patient motion tracking system according to any one of claims 1 to 7.

9. the ROI description data includes a template surface and a template ROI; the template surface and the template ROI are input to the ROI generation processor; the ROI generation processor is configured to align and warp the template ROI and the template surface with the 3D surface to generate at least a warped ROI, and subsequently transfer the warped ROI to the 3D surface. The patient motion tracking system of claim 1 .

10. a training module configured to generate and output the template surface and the template ROI to the memory; 10. The patient motion tracking system of claim 9.

11. the training module has two or more reference target surfaces, each having an annotated reference ROI applied thereto; The training module includes: aligning the two or more reference target surfaces and then calculating an average of the aligned reference target surfaces to generate the template surface; Calculate the mean of the annotated reference ROIs to generate the template ROI It is configured as follows: The patient motion tracking system of claim 10.

12. the ROI description data is configured as an ROI model that is trained by a machine learning processor before being stored in the memory, the ROI model being trained based on the one or more reference surfaces each having an annotated reference ROI applied thereto; The patient motion tracking system of claim 5 .

13. the ROI generation processor is configured to use the 3D surface as input to the ROI model and to output the ROI-labeled 3D surface to the motion tracking module.

13. The patient motion tracking system of claim 12.

14. the reference surface is constructed as a depth map and normal map representation of the reference surface; the ROI model is configured to utilize the 3D surface and classify vertices in the depth map and normal map representations as being inside or outside the region of interest defined by the trained model.

14. A patient motion tracking system according to claim 12 or 13.

15. the ROI generation processor further comprises a landmark generation model, the landmark generation model being trained by a machine learning processor before being stored in the ROI generation processor, the landmark generation model being trained based on the one or more reference surfaces to which annotated landmarks have been applied, respectively; The patient motion tracking system of claim 5 .

16. A radiation therapy system comprising: a 3D scanning and reconstruction system configured to be positioned in a radiation therapy room and configured to generate an input surface from which the 3D surface is generated; the stored ROI description data comprises one or more reference surfaces, each having an annotated reference ROI applied thereto; the ROI generation processor further comprises a landmark generation model, the landmark generation model being trained by a machine learning processor prior to being stored in the ROI generation processor, the landmark generation model being trained based on the one or more reference surfaces each having annotated landmarks applied thereto; The landmark generation model outputs a representation of landmarks on the input surface, thereby generating a landmark-labeled 3D surface, the landmark-labeled 3D surface comprising: aligning and warping a template ROI and template surface with the landmark-labeled 3D surface to generate an aligned and warped ROI, and then transferring the aligned and warped ROI to the 3D surface to output the ROI-labeled 3D surface; Utilized by the ROI generation processor together with the template surface and the template ROI; The patient motion tracking system of claim 1 .

Images