VR-based SGRT positioning verification method and related equipment

Abstract

The invention discloses a VR-based SGRT positioning verification method and related equipment, and relates to the technical field of radiotherapy, and the method comprises the steps: obtaining the treatment plan related data of a patient; performing three-dimensional visualization reconstruction by using the treatment plan related data, and constructing a three-dimensional model of the patient; acquiring real-time body surface data of the patient through the SGRT; performing rigid body registration and deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to a deformation registration result; using the reconstructed three-dimensional deformation field to deform the image and the treatment plan related data into the constructed three-dimensional positioning coordinate system, and displaying the image and the treatment plan related data through VR to verify the SGRT positioning; vR and SGRT are combined, through rigid body registration and deformation registration strategies, the precision of positioning verification can be enhanced, real-time interaction feedback can be provided, and treatment errors are effectively reduced.

Description

[0001] This application is a divisional application of the Chinese application with application number CN202411493476.3, application date October 24, 2024, and invention name “VR-based SGRT positioning verification method and related equipment”. Technical Field

[0002] The present invention relates to the field of radiotherapy technology, and in particular to a VR-based SGRT positioning verification method and related equipment. Background Art

[0003] In radiation therapy (RT), the accuracy of patient positioning is extremely important, especially in high-dose irradiation situations. Any positioning error may cause damage to normal tissue or miss the tumor target area. To ensure positioning accuracy, traditional methods rely on image-guided radiation therapy (IGRT), such as CT or X-rays, which use real-time imaging to confirm whether the patient's internal structure is consistent with the treatment plan. However, the disadvantage of IGRT is that it requires repeated imaging, resulting in additional exposure of the patient to radiation, and the operation process is complicated.

[0004] As a novel technology, surface-guided radiation therapy (SGRT) uses optical methods for contactless positioning, avoiding the increased radiation dose associated with traditional imaging techniques. SGRT uses multiple cameras to capture the patient's three-dimensional surface shape and position in real time, comparing it with the reference 3D image surface used in the treatment plan. Rigid or elastic registration algorithms are used to correct the patient's positioning, enabling the SGRT system to continuously monitor changes in the patient's body surface without radiation exposure. This makes it particularly suitable for dynamic or frequently adjusted treatments.

[0005] The SGRT system relies on a combination of multiple cameras and projectors to generate and register real-time 3D surface point cloud data of the patient. After acquiring the patient's 3D surface, the system aligns it with the planned reference image, calculates deviations, and adjusts the positioning in a timely manner. This non-ionizing nature allows SGRT to be used repeatedly throughout the radiotherapy process without increasing the patient's radiation exposure. The SGRT system has been widely used in clinical settings such as breast cancer, abdominal, head and neck, intracranial tumors, and pediatric tumors.

[0006] Although SGRT can monitor the patient's surface position in real time, it cannot directly provide detailed information about internal organs or tumors, especially when deep or surface movement is not synchronized with internal tissues; when the patient's surface changes, the collected surface contour cannot be accurately aligned with the surface contour in the imported plan, and SGRT surface information alone cannot effectively verify the positioning results; surface contour data has the greatest variation in positioning, and the information that can be used for fusion and registration is also the least, and there are problems in the registration process, especially when two inconsistent contours are rigidly aligned and measured, it is impossible to obtain a global optimal solution; SGRT visualization does not include plan-related information such as 3D imaging, outlining, dose, etc., and cannot provide an intuitive reference for the accuracy of radiotherapy execution. Patent CN111870825B describes a precise field-by-field radiotherapy positioning method based on a virtual intelligent medical platform. This method uses markers and gold standard registration (VR to patient registration, irradiation field to accelerator registration) as a registration method, requiring the addition of additional markers. Patent CN111214764B describes a radiotherapy positioning verification method and device based on a virtual intelligent medical platform. This method uses multiple gold standard identifications to determine the isocenter and uses isocenter registration for registration, requiring the addition of additional markers. Furthermore, this isocenter registration method cannot calculate bed movement angles. Patent CN116370848B describes a radiotherapy positioning method and system that uses surface rigid body registration only. However, surface registration cannot describe tissue deformation in the body and therefore cannot accurately register. The movement of the target volume and organs at risk within the human body often differs from the specific direction and movement of the surface. Therefore, surface displacement cannot be directly applied to the target volume or isocenter. Surface displacement does not truly reflect the change in the isocenter. Summary of the Invention

[0007] Based on the above problems, the purpose of the present invention is to provide a VR-based SGRT positioning verification method and related equipment, combining VR with SGRT, and through a rigid body registration plus deformation registration strategy, not only can the accuracy of positioning verification be enhanced, but also real-time interactive feedback can be provided, which is beneficial to ensuring the efficiency and effectiveness of radiotherapy.

[0008] The purpose of the present invention is achieved by the following technical solutions:

[0009] In a first aspect, the present invention proposes a VR-based SGRT positioning verification method, the method comprising:

[0010] Acquiring treatment plan-related data for the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position;

[0011] Performing three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient;

[0012] Acquire the patient's real-time body surface data through SGRT;

[0013] Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0014] Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0015] Using the reconstructed three-dimensional deformation field, the image and treatment plan-related data are deformed into the constructed three-dimensional positioning coordinate system and displayed through VR to verify the SGRT positioning.

[0016] Preferably, before the deformation registration, the method further comprises:

[0017] According to the rigid body registration result, the first displacement is obtained;

[0018] The bed moving operation is performed through the first displacement.

[0019] Preferably, the method further comprises:

[0020] According to the deformation registration result, the second displacement is obtained;

[0021] The bed moving operation is performed through the second displacement.

[0022] Preferably, a point cloud-based rigid body registration method is used to perform rigid body registration, wherein the loss function is to minimize the distance from the point to the surface.

[0023] Preferably, the loss function is:

[0024]

[0025] Among them, Loss is the loss function, p i is the patient’s real-time surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i is the target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points.

[0026] Preferably, reconstructing the three-dimensional deformation field includes:

[0027] Collect historical data from multiple patients and establish a machine learning model; the historical data includes body surface contour data before and after treatment, internal organ imaging data, and surface contour deformation during patient positioning;

[0028] Based on the real-time surface contour deformation of the patient obtained by the deformation registration, the overall deformation of the internal organs is obtained through a machine learning model.

[0029] Preferably, the displaying by VR to verify the SGRT positioning includes:

[0030] Import the patient's three-dimensional model into the VR system to build a virtual environment;

[0031] The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in VR glasses.

[0032] Preferably, the real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying the result in VR glasses includes:

[0033] Presenting the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan through the VR device;

[0034] The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

[0035] Preferably, after the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in VR glasses, the method further comprises:

[0036] Based on the degree of coincidence, the positioning is adjusted to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0037] In a second aspect, the present invention provides a VR-based SGRT positioning verification device, the device comprising:

[0038] A treatment plan acquisition module is used to obtain treatment plan-related data of the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment center position;

[0039] A model building module, configured to perform three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient;

[0040] A real-time data acquisition module is used to obtain the patient's real-time body surface data through SGRT;

[0041] A rigid body registration module, configured to perform rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0042] a deformation registration module, configured to perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and to reconstruct a three-dimensional deformation field according to the deformation registration result;

[0043] The verification module is used to use the reconstructed 3D deformation field to deform the image and treatment plan-related data into the constructed 3D positioning coordinate system and display it through VR to verify the SGRT positioning;

[0044] Before deformable registration, the apparatus further performs the following steps:

[0045] According to the rigid body registration result, the first displacement is obtained;

[0046] The bed moving operation is performed through the first displacement.

[0047] In a third aspect, the present invention provides a treatment system, comprising:

[0048] Radiotherapy devices, used to deliver radiation therapy to patients;

[0049] The positioning verification device of the present invention is used to verify the position of a patient.

[0050] In a fourth aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the aforementioned method of the present invention or the functions of the apparatus of the present invention when executing the computer program.

[0051] In a fifth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the steps of the aforementioned method of the present invention.

[0052] Compared with the existing technology, the beneficial effects of the present invention include at least: by fusing more three-dimensional information for positioning verification, it can provide richer convergence basis or better optimization starting point when positioning fails, thereby effectively avoiding positioning failure. Although surface contour data is important, it varies greatly and the information used for fusion and registration is limited. This method, by combining VR technology, introduces more three-dimensional anatomical information, making the registration process more accurate and reliable. Because the movement of the target area and organs at risk inside the human body is often inconsistent with the surface movement, traditional methods are difficult to accurately reflect the changes in the isocenter. This method introduces three-dimensional information through VR technology, which can observe and adjust the patient's position and the change process of the isocenter in real time, thereby significantly improving the positioning accuracy and ensuring the accuracy of treatment; SGRT can usually only provide a comparison of rigid body changes on the surface, while changes in tissues, organs and targets in the body often involve more deformation. With the help of VR technology and deformation field, this method can realistically simulate the anatomical changes in the target area caused by the positioning process, providing doctors with more comprehensive treatment information and ensuring accurate and reliable radiotherapy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1Schematic diagram of a VR-based SGRT positioning verification method according to an embodiment of the present invention;

[0054] Figure 2 1 is a flow chart of a VR-based SGRT positioning verification method according to an embodiment of the present invention;

[0055] Figure 3 Schematic diagram of a VR-based SGRT positioning verification device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the example embodiments to those skilled in the art. Identical reference numerals in the drawings represent identical or similar structures, and thus repeated descriptions thereof will be omitted.

[0057] The words expressing positions and directions described in the present invention are all explained with reference to the accompanying drawings as examples, but can be modified as needed, and all such modifications are within the scope of protection of the present invention.

[0058] Reference Figure 1 The present invention provides a VR-based SGRT positioning verification method, the method comprising:

[0059] Obtaining treatment plan-related data for the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, treatment isocenter location, and prescribed dose;

[0060] Performing three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient;

[0061] Acquiring real-time body surface data of the patient through SGRT; the real-time body surface data includes the real-time body surface contour, position and three-dimensional surface shape of the patient;

[0062] Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0063] Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0064] Using the reconstructed 3D deformation field, images and treatment planning data are deformed into the constructed 3D positioning coordinate system and displayed through VR to verify the SGRT positioning. The image-related data may include CT (computed tomography) or MRI (magnetic resonance imaging) images of the patient.

[0065] The working principle of the above technical solution is as follows: First, relevant data is obtained from the patient's treatment plan. This data is the basis of radiotherapy, including the patient's surface contour data (i.e., the shape and position of the patient's body surface), delineation data (such as the location and shape of the tumor and critical organs), dose data (i.e., the prescribed dose), and the treatment isocenter position (i.e., the center point of the radiotherapy treatment). The surface contour data is used as the fusion reference for the SGRT real-time measurement data, and the isocenter position is used to evaluate the actual need for the conversion of actual bed movement parameters. In addition, the image, target volume or organ at risk delineation, and the prescribed dose are used for three-dimensional fusion display in VR.

[0066] Using the acquired treatment plan data, computer graphics technology is used to perform 3D visualization reconstruction, creating a 3D model that closely resembles the patient's actual surface and internal anatomy. This model forms the basis for subsequent registration and verification.

[0067] During the positioning process, SGRT acquires real-time patient surface data. The SGRT multi-camera array rapidly captures the patient's surface features, while the projector projects specific light patterns. These light patterns interact with the patient's body surface, which are then captured by the cameras and converted into high-precision 3D point cloud data. This process ensures that every subtle change in the patient's body surface is captured and recorded in real time. Based on this, SGRT generates key data such as the patient's surface contour, position, and 3D surface shape in real time; these data reflect the patient's actual condition during the positioning process.

[0068] The real-time acquired body surface data is rigidly registered with the surface contour data in the 3D model. Rigid body registration refers to the process of aligning corresponding points in two or more images through rigid body transformation (i.e., rotation and translation). A rigid body is an idealized physical model whose volume and shape do not change after being subjected to external forces. Therefore, in rigid body registration, the distance and direction relationship between the points in the image remain unchanged after being transformed. The main purpose of rigid body registration is to align the real-time acquired body surface data (such as using optical surface scanning to scan the patient's position) with the surface contour data in the 3D model. The purpose of this step is to preliminarily align the real-time data and model data, eliminate large position shifts, and provide a basis for subsequent deformation registration.

[0069] Based on rigid body registration, deformable registration is performed. This takes into account deformation of the patient's body surface due to factors such as respiration, thereby more accurately reflecting actual changes in the patient's body surface. Through deformable registration of the body surface point cloud, a 3D surface mesh is calculated. Following the principle of diffeomorphism, a 3D deformation field is reconstructed—a vector field describing the deformation of the patient's body surface from the model state to the real-time state.

[0070] Using the reconstructed three-dimensional deformation field, the deformed patient images and treatment plan-related data are mapped into a constructed three-dimensional positioning coordinate system; this coordinate system is the reference system used to determine the patient's position and posture during radiotherapy; the overall deformation of the patient's internal organs in the three-dimensional model (such as organ deformation in CT and MRI scans) is inferred from the surface deformation field.

[0071] Finally, VR technology is used for display. VR technology provides an immersive environment, allowing doctors to intuitively visualize the patient's body surface, changes in treatment plan data, and the relationship between them. This allows doctors to verify SGRT positioning and ensure the accuracy of radiotherapy.

[0072] The effects of the above technical solution are:

[0073] By combining SGRT technology and three-dimensional visualization reconstruction, the present invention can acquire the patient's surface data in real time and accurately align it with a pre-constructed three-dimensional model; the present invention does not require the addition of additional markers for alignment; the rigid body to deformation method more accurately reflects the deformation of organs at risk (OAR) and target areas; through the dual alignment mechanism of rigid body alignment and deformation alignment, the accuracy of radiotherapy is significantly improved, ensuring that the treatment plan accurately matches the patient's actual condition.

[0074] Utilizing VR (Virtual Reality) technology for display, this invention provides doctors with an intuitive and immersive verification environment. Doctors can clearly see the patient's body deformation, treatment plan, and the relationship between them, making it easier for them to make accurate decisions. This helps doctors promptly identify and correct potential deviations during radiotherapy, improving treatment effectiveness. The invention can construct a personalized 3D model based on the patient's actual body surface data and internal anatomical structure, and perform precise registration and verification, helping doctors develop more personalized radiotherapy plans for patients to better meet their treatment needs.

[0075] In summary, the present invention significantly improves the accuracy, efficiency, and safety of radiotherapy by combining multiple advanced technologies, providing doctors with more intuitive and accurate decision support, while also bringing better treatment effects and experience to patients.

[0076] In some embodiments, before deformable registration, the method further includes:

[0077] According to the rigid body registration result, the first displacement is obtained;

[0078] Through the first displacement, a bed moving operation is performed, that is, the bed carrying the patient is moved by the distance of the first displacement.

[0079] The working principle and effects of the above technical solution are as follows:

[0080] The rigid body registration algorithm calculates the displacement difference between the real-time surface data and the 3D model, known as the first displacement. This displacement difference reflects the deviation of the patient from the ideal position during setup. The primary purpose of the bed movement operation is to adjust the patient's position based on the first displacement obtained from the rigid body registration results, so that it matches the ideal position in the radiotherapy plan—the intended position for treatment implementation. After obtaining the first displacement, the position of the treatment table is automatically or manually adjusted based on this information. This adjustment process typically involves translation along the X, Y, and Z axes, and possibly rotation, to ensure that the patient's surface data is fully consistent with the radiotherapy plan requirements. After the bed movement operation is completed, real-time surface data acquisition and registration verification are typically performed again to ensure the accuracy and effectiveness of the bed movement operation. This helps ensure precision and safety during radiotherapy. Performing rigid body registration and bed movement before deformable registration makes the subsequent deformable registration process more accurate and efficient. This also provides more reliable data support for radiotherapy plan formulation and adjustment, helping to optimize the entire radiotherapy process.

[0081] In some embodiments, the method further comprises:

[0082] According to the deformation registration result, the second displacement is obtained;

[0083] Through the second displacement, a bed moving operation is performed, that is, the bed carrying the patient is moved by the distance of the second displacement.

[0084] The principle and effect of the above technical solution are: during the positioning process, the patient's body surface morphology may undergo slight changes due to factors such as breathing and muscle tension; deformation registration can monitor and accurately calculate these changes in real time.

[0085] By collecting real-time data about the patient's body surface and comparing it with the 3D model in the radiotherapy plan, the deformable registration algorithm calculates the morphological differences between the two, known as the deformable registration result. This deformable registration result reflects the actual changes in the patient's body surface morphology during radiotherapy and serves as an important basis for subsequent bed transfer operations.

[0086] Based on the deformable registration results, the displacement difference between the real-time body surface morphology and the ideal morphology in the radiotherapy plan can be calculated. This displacement difference is called the second displacement.

[0087] The second displacement is a vector containing multiple dimensions (such as the X-axis, Y-axis, Z-axis, and possible rotation angles), which describes the position and direction of the patient that needs to be adjusted during radiotherapy. Based on the information of the second displacement, the position of the radiotherapy bed can be adjusted automatically or manually to correct the patient's body surface morphology deviation during radiotherapy. The bed moving operation usually includes translation along the X-axis, Y-axis, and Z-axis, as well as possible rotation adjustment to ensure that the patient's body surface morphology matches the requirements of the radiotherapy plan. During the bed moving operation, it is necessary to closely monitor the changes in the patient's body surface morphology and make real-time adjustments as needed to ensure the accuracy and safety of radiotherapy. After the bed moving operation is completed, real-time surface data acquisition and deformation registration verification are usually performed again to ensure the accuracy and effectiveness of the bed moving operation. If it is found that displacement deviation still exists, the bed moving operation can be performed again based on the new deformation registration results until the patient's body surface morphology is completely consistent with the requirements of the radiotherapy plan.

[0088] In summary, the principle of using the second displacement obtained from deformation registration results and using this second displacement to perform bed movement is a complex and precise process. It relies on advanced deformation registration technology and precise bed movement equipment to ensure the accuracy and safety of radiotherapy.

[0089] In some embodiments, the VR-based SGRT positioning verification method uses a point cloud-based rigid body registration method to perform rigid body registration, and the loss function is defined as minimizing the point-to-plane distance (Point-to-PlaneDistance):

[0090]

[0091] Among them, Loss is the loss function, p i is the patient’s real-time surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i is the target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points; for each pair of corresponding points p i and q i , calculate the distance between them and project this distance onto the normal vector n i Then, the sum of the squares of the distances of all points is calculated and this value is minimized by optimizing R and t.

[0092] The working principle of the above technical solution is as follows: SGRT collects real-time point cloud data of the patient's body surface. This point cloud data contains the geometric shape and position information of the patient's body surface. Surface contour point clouds are extracted from the patient's CT or other medical imaging data. This data represents the patient's ideal body surface morphology for radiotherapy planning. Point cloud-based rigid body registration methods match the real-time surface data with the contour data. This typically involves finding a correspondence between the two sets of point clouds, that is, determining a one-to-one correspondence between surface points and contour points.

[0093] In order to evaluate the accuracy of registration, the aforementioned loss function is defined, which is to minimize the point-to-plane distance (Point-to-PlaneDistance). This loss function calculates the loss of each pair of corresponding points (p i and q i ) and project this distance to the target point q i Normal vector n at i Then, the square of the projection distance of all points is summed to get the total loss value. The loss function is minimized by optimizing the rotation matrix R and the translation vector t, which usually involves an iterative algorithm such as the Iterative Closest Point (ICP) algorithm or its variants to gradually adjust R and t until the best registration result is found.

[0094] The effects of the above technical solution are: by minimizing the point-to-plane distance, the spatial relationship between real-time surface data and skin outline data can be more accurately reflected; compared with the traditional point-to-point registration method, point-to-plane registration can consider more geometric information, thereby obtaining more accurate registration results; the point-to-plane registration method has better robustness to noise and abnormal points. In practical applications, both real-time surface data and skin outline data may be interfered with by noise, and this method can reduce the impact of these noises on the registration results. In addition, since the information of the normal vector is taken into account, even if there is partial missing or deformation of the surface data, a reasonable registration result can be obtained through the constraint of the normal vector. Compared with other complex registration methods, this method is more computationally efficient and suitable for application scenarios with high real-time requirements. In the field of radiotherapy, this method can ensure that the patient's surface morphology matches the requirements in the radiotherapy plan, thereby improving the accuracy of radiotherapy.

[0095] In some embodiments, deformation registration is performed through the RegFormer network; the overall deformation of the internal organ is obtained through surface contour deformation, and a three-dimensional deformation field is reconstructed.

[0096] RegFormer is a Transformer-based network. Through the self-attention mechanism, the features of each point interact with the features of all other points to generate a global context-aware feature vector. This is very beneficial for deformation registration, especially when different regions of the point cloud have different degrees of deformation, and can better capture the overall change trend of the point cloud.

[0097] After the deformation field of the point cloud is derived, the finite element method (FEM) or elastic deformation models are used to describe the deformation of the entire volume. These models can infer the internal deformation of the volume based on the deformation field on the point cloud. In other words, the overall deformation of internal organs (such as organ deformation in CT and MRI scans) can be inferred from the external surface deformation field captured by the point cloud.

[0098] The working principle and effect of the above technical solution are as follows: input the patient's current real-time surface data (or point cloud data) and the reference data in the radiotherapy plan (such as organ contours or point clouds obtained by CT or MRI scans). The input data is preprocessed, such as denoising and standardization, to ensure that the data quality meets the network input requirements. The encoder part of the RegFormer network is used to extract features from the input point cloud data. The features of each point will interact with the features of all other points to generate a global context-aware feature vector. The decoder part of the RegFormer network is used to predict the deformation field of the point cloud based on the extracted feature vector. The deformation field describes the displacement vector of each point from its original position to its deformed position. The deformation field output by the RegFormer network is discrete, that is, only deformation information is given for the input point cloud data.

[0099] To obtain deformation information for the entire volume, mathematical tools such as the finite element method (FEM) or elastic deformation models are required. These models can infer the deformation within the volume based on the deformation field on the point cloud. The volume is divided into multiple finite elements, and the deformation within each element can be calculated from the deformation field on the point cloud using interpolation methods. Assuming the volume is an elastic body, the deformation field within the volume is obtained by solving partial differential equations based on the principles of elasticity.

[0100] Because the RegFormer network processes point cloud data and the SGRT collects patient surface information, the initial information obtained is surface contour deformation. Using the finite element method or elastic deformation models, the overall deformation of internal organs can be inferred from this surface contour deformation information. The reconstructed 3D deformation field is visualized to provide the physician with an intuitive understanding of the patient's deformation. Based on the reconstructed deformation field, the radiotherapy plan is adjusted accordingly to ensure the accuracy and safety of radiotherapy.

[0101] In summary, the process of using the RegFormer network for deformable registration and reconstructing the three-dimensional deformation field through surface contour deformation is a process that combines deep learning, mathematical physics models, and medical applications. This method can accurately capture the patient's deformation information and provide strong support for the formulation and adjustment of radiotherapy plans.

[0102] In some embodiments, obtaining the overall deformation of the internal organ by deforming the surface contour and reconstructing the three-dimensional deformation field includes:

[0103] Collect historical data from multiple patients to establish a machine learning model; the historical data includes body surface contour data before and after treatment, internal organ imaging data, and surface contour deformation during patient positioning;

[0104] Based on the real-time surface contour deformation of the current patient, the overall deformation of the internal organs is obtained through a machine learning model.

[0105] The working principle of the above technical solution is:

[0106] Collect a large number of patients' pre- and post-treatment body contour data (such as point cloud data obtained through the SGRT system), internal organ imaging data (such as CT or MRI scans), and corresponding treatment plan data; clean, align, and standardize the collected data to ensure data quality and consistency; this includes removing noise, filling missing values, calibrating data at different time points, and scaling the data to an appropriate range; extract key features from the body contour data and internal organ imaging data. These features should be able to reflect the relationship between the deformation of the body surface and internal organs. Features can include the shape, size, curvature, volume, position, etc. of the contour. Based on the characteristics of the data and the complexity of the problem, select a machine learning model (such as random forest, support vector machine, neural network, etc.). Use the preprocessed historical data to train the model to learn the mapping relationship between the deformation of the body contour and the overall deformation of the internal organs.

[0107] During the positioning process, the real-time surface contour deformation data of the current patient is continuously collected through the SGRT system or other surface scanning technologies. The real-time surface contour deformation data is input into the trained machine learning model, and the model will output the predicted overall deformation of the internal organs. Using the predicted internal organ deformation, combined with medical image registration technology, the deformation is mapped into three-dimensional space to construct a three-dimensional deformation field; the predicted deformation needs to be aligned with the reference image (such as the CT or MRI image before treatment). Use cross-validation or other validation methods to evaluate the performance of the model to ensure that the model has generalization capabilities. Collect more real-time data and feedback for continuous optimization and iteration of the model.

[0108] The benefits of this technical solution are: This method can predict internal organ deformation in real time through surface deformation, obtaining an accurate three-dimensional deformation field and improving positioning accuracy. Accurate deformation prediction helps reduce the need for repeated patient movement and positioning due to inaccurate positioning, improving treatment comfort and patient experience. Accurate deformation prediction reduces repeated adjustments and verification time during the positioning process, thereby optimizing the pre-treatment preparation process and improving overall treatment efficiency.

[0109] In some embodiments, the verifying the SGRT positioning by displaying it through VR includes:

[0110] Import the patient's three-dimensional model into the VR system to build a virtual environment;

[0111] The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in VR glasses.

[0112] In some embodiments, the real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying the result in VR glasses includes:

[0113] Presenting the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan through the VR device;

[0114] The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

[0115] In some embodiments, after the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed on VR glasses, the method further includes:

[0116] Based on the degree of coincidence, the positioning is adjusted to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0117] The principle and effect of the above technical solution are as follows: First, a 3D model is generated from the patient's CT data, contouring, and planning data. The 3D model is then imported into a VR system to create a virtual environment that closely resembles the patient's anatomy. The SGRT system acquires the patient's position information in real time. This information is then fused with the 3D model in the virtual environment in real time. This means that the patient's actual position and the 3D model in the virtual environment are aligned in real time. The fused scene is then displayed to the physician through VR glasses. The physician, wearing the VR glasses, can observe a three-dimensional scene that combines both the patient's actual position and the virtual 3D model. Through the VR glasses, the physician can visually observe the degree of alignment between the patient's positioning and the planning CT scan, particularly the isocenter position. The physician can clearly see whether the patient's current position aligns with the target position in the radiotherapy plan, namely the treatment isocenter. The VR glasses also display the 3D model constructed from the imaging, contouring, and planning data, as well as relevant information in the patient's positioning coordinate system, in real time. If any mismatch or deviation is observed, the physician can immediately make adjustments to ensure that the patient's positioning meets the requirements of the radiotherapy plan and aligns with the treatment isocenter. Position verification using VR technology can greatly reduce human errors and machine errors and improve the accuracy of radiotherapy; VR technology enables doctors to observe and analyze the patient's positioning more quickly and intuitively, thereby improving work efficiency.

[0118] An embodiment of the present invention provides a VR-based SGRT positioning verification device, the device comprising:

[0119] A treatment plan acquisition module is used to obtain the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, treatment center position and prescription dose, etc.;

[0120] A model building module, configured to perform three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient;

[0121] A real-time data acquisition module is used to acquire the patient's real-time body surface data through SGRT; the real-time body surface data includes the patient's real-time body surface contour, position and three-dimensional surface shape;

[0122] A rigid body registration module, configured to perform rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0123] a deformation registration module, configured to perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and to reconstruct a three-dimensional deformation field according to the deformation registration result;

[0124] The verification module is used to use the reconstructed three-dimensional deformation field to deform the image and treatment plan related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning.

[0125] In some embodiments, the apparatus further comprises:

[0126] The first displacement module is used to obtain a first displacement according to the rigid body registration result; and perform a bed moving operation through the first displacement.

[0127] In some embodiments, the apparatus further comprises:

[0128] The first displacement module is used to obtain a second displacement according to the deformation registration result; and perform a bed moving operation through the second displacement.

[0129] In some embodiments, the rigid body registration module performs rigid body registration using a point cloud-based rigid body registration method, wherein the loss function is to minimize the distance from the point to the surface:

[0130]

[0131] Among them, Loss is the loss function, p i is the patient’s real-time surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i is the target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points; for each pair of corresponding points p i and q i , calculate the distance between them and project this distance onto the normal vector n i Then, the sum of the squares of the distances of all points is calculated and this value is minimized by optimizing R and t.

[0132] In some embodiments, the deformation registration module performs deformation registration through a RegFormer network, obtains the overall deformation of the internal organ through surface contour deformation, and reconstructs a three-dimensional deformation field.

[0133] In some embodiments, the verification module includes:

[0134] Import unit, used to import the patient's three-dimensional model into the VR system to build a virtual environment;

[0135] The fusion display unit is used to fuse the patient's position information with the three-dimensional model in the virtual environment in real time and display it in VR glasses.

[0136] In some embodiments, the fusion display unit includes:

[0137] a first display unit, configured to present, through the VR device, the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan;

[0138] The second display unit is used to display the three-dimensional model constructed by the image, outline data and planning data in real time through the VR device.

[0139] In some embodiments, after the display in the VR glasses, the device further includes:

[0140] The adjustment module is used to perform positioning adjustment based on the overlap to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0141] The working principle and effect of the above technical solution are the same as those in the embodiment of the method of the present invention, and will not be described in detail here.

[0142] An embodiment of the present invention provides a treatment system, comprising:

[0143] Radiotherapy devices, used to deliver radiation therapy to patients;

[0144] The positioning verification device described in the embodiment of the present invention is used to verify the position of a patient.

[0145] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of any one of the methods described in the embodiments of the present invention or the functions of the device described in the embodiments of the present invention.

[0146] An embodiment of the present invention also provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed, the steps of the method in the embodiment of the present invention are implemented. Its specific implementation method is consistent with the implementation method and the technical effect achieved in the above-mentioned method embodiment, and some contents will not be repeated here.

[0147] In the present invention, a readable storage medium can be any tangible medium that contains or stores a program that can be used by an instruction execution system, device, or component or used in combination with it. A program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0148] A computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, or any suitable combination thereof. The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user computing device, partially on an associated device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0149] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limiting the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the invention without departing from the principles and purpose of the present invention. All such changes shall fall within the scope of protection of the claims of the present invention.

Claims

1. The VR-based SGRT positioning verification method is characterized by: The method comprises: Acquiring treatment plan-related data for the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position; Performing three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient; Acquire the patient's real-time body surface data through SGRT; Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model; Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result; Using the reconstructed 3D deformation field, the image and treatment plan data are deformed into the constructed 3D positioning coordinate system and displayed through VR to verify the SGRT positioning. Before the deformable registration, the method further includes: According to the rigid body registration result, the first displacement is obtained; The bed moving operation is performed through the first displacement.

2. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The method further comprises: According to the deformation registration result, the second displacement is obtained; The bed moving operation is performed through the second displacement.

3. The VR-based SGRT positioning verification method according to claim 1, characterized in that: Rigid body registration is performed using a point cloud-based rigid body registration method, where the loss function is to minimize the distance from the point to the surface.

4. The VR-based SGRT positioning verification method according to claim 3, characterized in that: The loss function is: Among them, Loss is the loss function, p i is the patient’s real-time surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i is the target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points.

5. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The reconstructing of the three-dimensional deformation field includes: Collect historical data from multiple patients and establish a machine learning model; the historical data includes body surface contour data before and after treatment, internal organ imaging data, and surface contour deformation during patient positioning; Based on the real-time surface contour deformation of the patient obtained by the deformation registration, the overall deformation of the internal organs is obtained through a machine learning model.

6. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The verification of the SGRT positioning by displaying it through VR includes: Import the patient's three-dimensional model into the VR system to build a virtual environment; The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device.

7. The VR-based SGRT positioning verification method according to claim 6, characterized in that: The real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying the information on the VR glasses includes: Presenting the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan through the VR device; The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

8. The VR-based SGRT positioning verification method according to claim 7, characterized in that: After the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR glasses, the method further includes: Based on the degree of coincidence, the positioning is adjusted to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

9. The VR-based SGRT positioning verification device is characterized by: The device comprises: A treatment plan acquisition module is used to acquire the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, dose data, and treatment center position; A model building module, configured to perform three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient; A real-time data acquisition module is used to obtain the patient's real-time body surface data through SGRT; A rigid body registration module, configured to perform rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model; a deformation registration module, configured to perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and to reconstruct a three-dimensional deformation field according to the deformation registration result; The verification module is used to use the reconstructed 3D deformation field to deform the image and treatment plan-related data into the constructed 3D positioning coordinate system and display it through VR to verify the SGRT positioning; Before deformable registration, the apparatus further performs the following steps: According to the rigid body registration result, the first displacement is obtained; The bed moving operation is performed through the first displacement.

10. A treatment system, characterized in that The system comprises: Radiotherapy devices, used to deliver radiation therapy to patients; The positioning verification device described in claim 9 is used to verify the position of the patient.

11. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program.

12. A computer-readable storage medium, characterized in that The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the steps of the method according to any one of claims 1 to 8.

Images