VR-Based SGRT Setup Verification Method and Related Devices

Abstract

The present invention discloses a SGRT positioning verification method and related devices based on VR, which relates to the technical field of radiotherapy. The method includes obtaining treatment plan-related data of a patient; performing three-dimensional visualization reconstruction by using the treatment plan-related data to construct a three-dimensional model of the patient; obtaining real-time body surface data of the patient through SGRT; performing rigid body registration and deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result; using the reconstructed three-dimensional deformation field to deform the image and treatment plan-related data into the constructed three-dimensional positioning coordinate system, and displaying through VR to verify the SGRT positioning; combining VR and SGRT, through the rigid body registration plus deformation registration strategy, not only can the accuracy of positioning verification be enhanced, but also real-time interactive feedback can be provided, improving the efficiency and effect of radiotherapy.

Description

Technical Field

[0001] The present invention relates to the technical field of radiotherapy, and particularly to a VR-based SGRT positioning verification method and related devices. Background Art

[0002] In radiation therapy (RT), the accuracy of patient positioning is extremely important. Especially in the case of high-dose irradiation, any positioning error may lead to damage to normal tissues or missed irradiation of the tumor target area. To ensure positioning accuracy, traditional methods rely on image-guided radiation therapy (IGRT), such as CT or X-rays, which confirm whether the patient's internal structure is consistent with the treatment plan through real-time imaging. However, the disadvantage of IGRT is that it requires repeated imaging, resulting in additional radiation exposure for patients and a complex operation process.

[0003] As a new technology, surface-guided radiation therapy (SGRT) performs non-contact positioning through optical means, avoiding the problem of increased radiation dose in traditional imaging technologies. SGRT uses multiple cameras to capture the three-dimensional surface shape and position of the patient in real time, compares it with the reference 3D image body surface in the treatment plan, and uses rigid or elastic registration algorithms to correct the patient's positioning, enabling the SGRT system to continuously monitor the surface changes of the patient without radiation, which is particularly suitable for dynamic or frequently adjusted treatments.

[0004] The SGRT system relies on a combination of multiple cameras and projectors to generate and register the real-time three-dimensional surface point cloud data of the patient; after obtaining the three-dimensional surface of the patient, the system registers it with the reference image in the plan, calculates the deviation, and adjusts the positioning in a timely manner. This non-ionizing property allows SGRT to be repeatedly used throughout the radiotherapy process without increasing the patient's radiation exposure. The SGRT system has been widely applied in clinical scenarios such as breast cancer, abdomen, head and neck, intracranial tumors, and pediatric tumors.

[0005] Although SGRT can monitor the patient's body surface position in real time, it cannot directly provide detailed information about internal organs or tumors, especially in the case of deep parts or when the body surface movement is asynchronous with internal tissues; when changes occur on the patient's surface and the collected surface contour cannot be accurately registered with the body surface contour in the imported plan, SGRT's surface information alone cannot effectively verify the positioning result; the surface contour data changes the most during positioning, and the information available for fusion registration is also the least. There are problems in the registration process, especially when rigid body registration metrics are performed on two inconsistent contours, and a global optimal solution cannot be obtained; SGRT visualization does not contain plan-related information such as 3D images, delineation, dose, etc., and cannot provide an intuitive reference for the accuracy of radiotherapy execution. The existing technologies of radiotherapy precise field-by-field positioning methods based on virtual intelligent medical platforms have the problem of using fiducial registration (VR registration with the patient, irradiation field registration with the accelerator) as the registration method by posting fiducial points and require adding additional markers; the existing technologies of a radiotherapy positioning verification method and device based on a virtual intelligent medical platform have the problem of obtaining the isocenter position by identifying fiducials multiple times and using the isocenter registration method for registration, which requires adding additional markers, and the isocenter registration method cannot calculate the couch angle; the existing technologies of radiotherapy positioning methods and systems only perform registration through surface rigid bodies, and surface registration cannot describe the deformation of internal tissues and cannot perform accurate registration. The movements of internal target areas and critical organs in the human body are often inconsistent with the movement directions and displacements of the surface, so the surface displacement cannot be directly applied to the target area or the isocenter. The surface displacement does not truly reflect the transformation of the isocenter. Summary of the Invention

[0006] Based on the above problems, the purpose of the present invention is to provide a VR-based SGRT positioning verification method and related devices. By combining VR and SGRT and adopting a rigid body registration plus deformation registration strategy, not only can the accuracy of positioning verification be enhanced, but also real-time interactive feedback can be provided, which is beneficial to ensuring the efficiency and effect of radiotherapy.

[0007] The purpose of the present invention is achieved by adopting the following technical solutions:

[0008] In the first aspect, the present invention proposes a VR-based SGRT positioning verification method, and the method includes:

[0009] Obtain the treatment plan-related data of the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position;

[0010] Use the treatment plan-related data for three-dimensional visualization reconstruction to construct a three-dimensional model of the patient;

[0011] Obtain the real-time body surface data of the patient through SGRT;

[0012] Perform rigid registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0013] Perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstruct a three-dimensional deformation field according to the deformation registration result;

[0014] Utilize the reconstructed three-dimensional deformation field to deform the image and treatment plan-related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning.

[0015] Preferably, before the deformation registration, the method further includes:

[0016] Obtain a first displacement according to the rigid registration result;

[0017] Perform a bed moving operation through the first displacement.

[0018] Preferably, the method further includes:

[0019] Obtain a second displacement according to the deformation registration result;

[0020] Perform a bed moving operation through the second displacement.

[0021] Preferably, use a rigid registration method based on point cloud for rigid registration.

[0022] Preferably, the rigid registration includes calculating a loss function, and the loss function is:

[0023]

[0024] where Loss is the loss function, p i is a real-time body surface data point of the patient, q i is the point in the skin contour point cloud of the patient's delineated data corresponding to p i , n i is the normal vector at the target point q i , R is the rotation matrix, t is the translation vector; N is the number of samples of the data points.

[0025] Preferably, perform deformation registration through the RegFormer network, and obtain the overall deformation of the internal organs through surface contour deformation to reconstruct a three-dimensional deformation field.

[0026] Preferably, the display through VR to verify the SGRT positioning includes:

[0027] Import the three-dimensional model of the patient into the VR device to construct a virtual environment;

[0028] The patient position information is fused with a three-dimensional model in a virtual environment in real time and displayed in a VR device.

[0029] Preferably, the fusion of the patient position information with the three-dimensional model in the virtual environment and display in the VR device includes:

[0030] Through the VR device, present the patient's current position, the target position in the radiotherapy plan, and the coincidence degree between the patient's current position and the target position in the radiotherapy plan;

[0031] Through the VR device, display in real time the three-dimensional model constructed from the images, contour data, and plan data.

[0032] Preferably, after the fusion of the patient position information with the three-dimensional model in the virtual environment and display in the VR device, the method further includes:

[0033] According to the coincidence degree, perform positioning adjustment to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0034] In a second aspect, the present invention provides a VR-based SGRT positioning verification device, and the device includes:

[0035] A treatment plan acquisition module for acquiring data related to the patient's treatment plan; the data related to the treatment plan includes the patient's surface contour data, contour data, dose data, and treatment isocenter position;

[0036] A model construction module for performing three-dimensional visualization reconstruction using the data related to the treatment plan to construct a three-dimensional model of the patient;

[0037] A real-time data acquisition module for acquiring the patient's real-time body surface data through SGRT;

[0038] A rigid body registration module for performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0039] A deformation registration module for performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0040] A verification module for using the reconstructed three-dimensional deformation field to deform the images and data related to the treatment plan into the constructed three-dimensional positioning coordinate system and displaying through VR to verify the SGRT positioning.

[0041] In a third aspect, the present invention provides a treatment system, and the system includes:

[0042] Radiotherapy devices, used to deliver radiation therapy to patients;

[0043] The positioning verification device of the present invention is used to verify the position of a patient.

[0044] In a fourth aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the aforementioned method of the present invention or the functions of the apparatus of the present invention when executing the computer program.

[0045] In a fifth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the steps of the aforementioned method of the present invention.

[0046] Compared with the existing technology, the beneficial effects of the present invention include at least: by fusing more three-dimensional information for positioning verification, it can provide richer convergence basis or better optimization starting point when positioning fails, thereby effectively avoiding positioning failure. Although surface contour data is important, it varies greatly and the information used for fusion and registration is limited. This method, by combining VR technology, introduces more three-dimensional anatomical information, making the registration process more accurate and reliable. Because the movement of the target area and organs at risk inside the human body is often inconsistent with the surface movement, traditional methods are difficult to accurately reflect the changes in the isocenter. This method introduces three-dimensional information through VR technology, which can observe and adjust the patient's position and the change process of the isocenter in real time, thereby significantly improving the positioning accuracy and ensuring the accuracy of treatment; SGRT can usually only provide a comparison of rigid body changes on the surface, while changes in tissues, organs and targets in the body often involve more deformation. With the help of VR technology and deformation field, this method can realistically simulate the anatomical changes in the target area caused by the positioning process, providing doctors with more comprehensive treatment information and ensuring accurate and reliable radiotherapy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of a VR-based SGRT positioning verification method according to an embodiment of the present invention;

[0048] Figure 2 1 is a flow chart of the application of the VR-based SGRT positioning verification method to radiotherapy according to an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of a VR-based SGRT positioning verification device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. Like reference numerals in the figures denote like or similar structures, and thus their repetitive description will be omitted.

[0051] In the present invention, the words describing the expression positions and directions are all illustrated by taking the accompanying drawings as examples, but can be changed according to needs, and all the changes made are included in the protection scope of the present invention.

[0052] Refer to Figure 1 : The present invention provides a SGRT positioning verification method based on VR, and the method includes:

[0053] Obtain treatment plan-related data of a patient; the treatment plan-related data includes the patient's surface contour data, delineation data, treatment isocenter position, and prescription dose, etc.;

[0054] Use the treatment plan-related data to perform three-dimensional visualization reconstruction to construct a three-dimensional model of the patient;

[0055] Obtain the real-time body surface data of the patient through SGRT; the real-time body surface data includes the real-time body surface contour, position, and three-dimensional surface shape of the patient;

[0056] Perform rigid registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0057] Perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model to obtain a deformation registration result, and reconstruct a three-dimensional deformation field according to the deformation registration result;

[0058] Use the reconstructed three-dimensional deformation field to deform the image and treatment plan-related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning. The image-related data may include the patient's CT (Computed Tomography) image or MRI (Magnetic Resonance Imaging) image.

[0059] Refer to Figure 2 As shown, after the positioning is verified, if the positioning verification is passed, radiotherapy is entered until the radiotherapy ends. If the positioning verification fails, return to the step of obtaining the real-time body surface data of the patient through SGRT above, that is, the step of real-time body surface data acquisition, and calculate the bed movement parameters.

[0060] The working principle of the above technical solution is as follows: First, relevant data are obtained from the patient's treatment plan. These data are the basis for radiotherapy, including the patient's surface contour data (i.e., the shape and position information of the patient's body surface), delineation data (such as the position and shape of tumors and critical organs), dose data (i.e., the prescribed dose), and the treatment isocenter position (i.e., the central point of radiotherapy). The surface contour data is used as the fusion benchmark for SGRT real-time measurement data, and the isocenter position is used to evaluate the conversion of actual bed movement parameters; in addition, data such as images, target area or organ-at-risk delineation, and prescribed dose will be used for the three-dimensional fusion display of VR.

[0061] Using the obtained treatment plan-related data, three-dimensional visualization reconstruction is carried out through computer graphics technology to construct a three-dimensional model highly similar to the patient's actual body surface and internal anatomical structure. This model is the basis for subsequent registration and verification.

[0062] During the positioning process, the patient's body surface data is obtained in real time through SGRT; the SGRT multi-camera array quickly captures the patient's body surface features, and the projector is responsible for projecting specific light patterns. After interacting with the patient's body surface, these light patterns are captured by the camera and converted into high-precision three-dimensional point cloud data. This process ensures that every subtle change in the patient's body surface can be captured and recorded by the system in real time. On this basis, SGRT can generate key data such as the patient's body surface contour, position, and three-dimensional surface shape in real time; they reflect the actual state of the patient during the positioning process.

[0063] The rigid registration is performed on the real-time obtained body surface data, that is, the real-time body surface data and the surface contour data in the three-dimensional model to obtain the rigid registration result; rigid registration refers to the process of aligning corresponding points in two or more images through rigid body transformation (i.e., rotation and translation); a rigid body is an idealized physical model, and its volume and shape do not change after being subjected to external forces. Therefore, in rigid registration, the distance and direction relationship between points in the image remain unchanged after transformation; the main purpose of rigid registration is to align the real-time obtained body surface data (such as using optical surface scanning to scan the patient's position) with the surface contour data in the three-dimensional model; the purpose of this step is to initially align the real-time data and the model data, eliminate large displacements, and provide a basis for subsequent deformation registration.

[0064] On the basis of rigid registration, deformation registration is carried out. Deformation registration takes into account the deformation of the patient's body surface due to factors such as breathing, so it can more accurately reflect the actual changes of the patient's body surface. Through the deformation registration of the body surface point cloud, a three-dimensional deformation mesh of the body surface can be calculated, and according to the principle of diffeomorphism, a three-dimensional deformation field can be reconstructed, that is, a vector field describing the deformation of the patient's body surface from the model state to the real-time state.

[0065] Using the reconstructed three-dimensional deformation field, map the deformed patient image and treatment plan-related data into the constructed three-dimensional positioning coordinate system; this coordinate system is the reference system used to determine the position and posture of the patient during the radiotherapy process; from the body surface deformation field, infer the overall deformation of the patient's internal organs in the three-dimensional model (such as the organ deformation in CT and MRI scans).

[0066] Finally, display through VR technology. VR technology can provide an immersive environment, enabling doctors to visually see the changes in the patient's body surface, the relevant data of the treatment plan, and the relationships between them. In this way, doctors can verify the SGRT positioning to ensure the accuracy of radiotherapy.

[0067] The effects of the above technical solutions are as follows:

[0068] By combining SGRT technology and three-dimensional visualization reconstruction, the present invention can obtain the patient's body surface data in real time and accurately register it with the pre-constructed three-dimensional model; the present invention does not require registration by adding additional markers; the rigid body to deformation method more accurately reflects the deformation of Organs at Risk (OAR) and the target area; through the dual registration mechanism of rigid body registration and deformation registration, the accuracy of radiotherapy is significantly improved, ensuring the precise matching of the treatment plan and the actual state of the patient.

[0069] Using VR (Virtual Reality) technology for display, the present invention provides an intuitive and immersive verification environment for doctors. Doctors can clearly see the body surface deformation of the patient, the treatment plan, and the relationships between them, making it easier to make accurate decisions; it helps doctors timely discover and correct possible deviations during the radiotherapy process, improving the treatment effect; the present invention can construct a personalized three-dimensional model based on the patient's actual body surface data and internal anatomical structure, and perform precise registration and verification, which helps doctors formulate a more personalized radiotherapy plan for patients and better meet the treatment needs of patients.

[0070] In summary, by combining multiple advanced technologies, the present invention significantly improves the accuracy, efficiency, and safety of radiotherapy, provides more intuitive and accurate decision support for doctors, and also brings better treatment effects and experiences for patients.

[0071] In some embodiments, before deformation registration, the method further includes:

[0072] Obtain the first displacement according to the rigid body registration result;

[0073] Through the first displacement, perform a bed movement operation, that is, move the bed carrying the patient a distance of the first displacement.

[0074] The working principle and effects of the above technical solution are as follows:

[0075] Through the rigid body registration algorithm, the displacement difference between the real-time body surface data and the three-dimensional model can be calculated, which is the first displacement; this displacement difference reflects the deviation between the patient's position during setup and the ideal position; the main purpose of the bed movement operation is to adjust the patient's position according to the first displacement obtained from the rigid body registration result, so that it matches the ideal position in the radiotherapy plan, that is, the expected position for implementing the treatment plan. After obtaining the first displacement, the position of the treatment bed is adjusted automatically or manually according to the first displacement information; this adjustment process usually includes translations along the X-axis, Y-axis, and Z-axis, and possible rotational adjustments to ensure that the patient's body surface data is exactly the same as the requirements in the radiotherapy plan. After the bed movement operation is completed, real-time body surface data collection and registration verification are usually performed again to ensure the accuracy and effectiveness of the bed movement operation. This helps to ensure the accuracy and safety during the radiotherapy process. Performing rigid body registration and bed movement operations before deformation registration can make the subsequent deformation registration process more accurate and efficient. At the same time, this also provides more reliable data support for the formulation and adjustment of the radiotherapy plan, helping to optimize the entire radiotherapy process.

[0076] In some embodiments, the method further includes:

[0077] Obtaining a second displacement according to the deformation registration result;

[0078] Performing a bed movement operation, that is, moving the bed carrying the patient by the distance of the second displacement through the second displacement.

[0079] The principle and effects of the above technical solution are as follows: During the setup process, the patient's body surface morphology may change slightly due to factors such as breathing and muscle tension; deformation registration can monitor and accurately calculate these changes in real time.

[0080] By collecting the real-time data of the patient's body surface and comparing it with the three-dimensional model in the radiotherapy plan, the deformation registration algorithm can calculate the morphological difference between the two, which is the deformation registration result. The deformation registration result reflects the actual change situation of the patient's body surface morphology during the radiotherapy process and is an important basis for the subsequent bed movement operation.

[0081] Based on the deformation registration result, the displacement difference between the real-time body surface morphology and the ideal morphology in the radiotherapy plan can be calculated, and this displacement difference is called the second displacement.

[0082] The second displacement is a vector containing multiple dimensions (such as the X-axis, Y-axis, Z-axis, and possibly the rotation angle), which describes the position and orientation that the patient needs to be adjusted during radiotherapy. According to the information of the second displacement, the position of the radiotherapy couch can be adjusted automatically or manually to correct the surface form deviation of the patient during radiotherapy. The couch movement operation usually includes translations along the X-axis, Y-axis, and Z-axis, and possibly rotational adjustments, to ensure that the patient's surface form matches the requirements in the radiotherapy plan. During the couch movement operation, it is necessary to closely monitor the changes in the patient's surface form and make real-time adjustments as needed to ensure the accuracy and safety of radiotherapy. After the couch movement operation is completed, real-time surface data acquisition and deformation registration verification are usually performed again to ensure the accuracy and effectiveness of the couch movement operation. If it is found that there is still a displacement deviation, a new couch movement operation can be performed according to the new deformation registration result until the patient's surface form completely matches the requirements in the radiotherapy plan.

[0083] In summary, the working principle of obtaining the second displacement based on the deformation registration result and performing the couch movement operation through this second displacement is a complex and precise process. It relies on advanced deformation registration technology and precise couch movement equipment to ensure the accuracy and safety of radiotherapy.

[0084] In some embodiments, the VR-based SGRT setup verification method uses a point cloud-based rigid body registration method for rigid body registration. The rigid body registration includes calculating a loss function, which is defined as minimizing the point-to-plane distance:

[0085]

[0086] where Loss is the loss function used to calculate the couch movement parameters, p i is the real-time surface data point of the patient, q i is the point in the skin contour point cloud of the patient's delineated data corresponding to p i , n i is the normal vector at the target point q i , R is the rotation matrix, t is the translation vector; N is the number of samples of the data points; for each pair of corresponding points p i and q i , calculate the distance between them and project this distance onto the normal vector n i , then sum the squares of the distances of all points, and minimize this value by optimizing R and t. The couch movement parameters can include the translation distances in the X, Y, and Z directions and / or the rotation angle.

[0087] The working principle of the above technical solution is as follows: SGRT is used to collect the patient's body surface point cloud data in real time. These point cloud data contain the geometric shape and position information of the patient's body surface. Surface contour point clouds are extracted from the patient's CT or other medical image data. These data represent the ideal body surface form of the patient in the radiotherapy plan. A point cloud-based rigid body registration method is used to match the real-time body surface data with the delineated data, which usually involves finding the corresponding relationship between two sets of point clouds, that is, determining that the body surface points and the contour points correspond one by one.

[0088] To evaluate the registration accuracy, the aforementioned loss function is defined, that is, minimizing the Point-to-Plane Distance. This loss function calculates the distance between each pair of corresponding points (p i and q i ) and projects this distance onto the normal vector n i at the target point q i . Then, the sum of the squared projection distances of all points is obtained to get the total loss value. By optimizing the rotation matrix R and the translation vector t, the loss function is minimized, which usually involves iterative algorithms such as the Iterative Closest Point (ICP) algorithm or its variants to gradually adjust R and t until the best registration result is found.

[0089] The effects of the above technical solution are as follows: By minimizing the point-to-plane distance, it can more accurately reflect the spatial relationship between the real-time body surface data and the skin delineated data; compared with the traditional point-to-point registration method, point-to-plane registration can consider more geometric information, thus obtaining a more accurate registration result; the point-to-plane registration method has better robustness to noise and outliers. In practical applications, both the real-time body surface data and the skin delineated data may be interfered by noise, and this method can reduce the influence of these noises on the registration result. In addition, due to considering the information of the normal vector, even if there are partial missing or deformed body surface data, a reasonable registration result can be obtained through the constraint of the normal vector. Compared with other complex registration methods, this method is more computationally efficient and is suitable for application scenarios with high real-time requirements. In the field of radiotherapy, this method can ensure that the patient's body surface form matches the requirements in the radiotherapy plan, thereby improving the accuracy of radiotherapy.

[0090] In some embodiments, deformation registration is performed through the RegFormer network; through surface contour deformation, the overall deformation of internal organs is obtained, and a three-dimensional deformation field is reconstructed.

[0091] RegFormer is a Transformer-based network. Through the self-attention mechanism, the features of each point interact with the features of all other points to generate a globally context-aware feature vector, which is very beneficial for deformation registration. Especially when there are different degrees of deformations in different regions of the point cloud, it can better capture the overall change trend of the point cloud.

[0092] After obtaining the deformation field of the point cloud, the finite element method (FEM) or elastic deformation model is used to describe the deformation of the entire volume for these discrete point cloud deformation fields. These models can calculate the deformation inside the volume based on the deformation field on the point cloud. That is, through the external surface deformation field captured by the point cloud, the overall deformation of the internal organ can be inferred (such as the organ deformation in CT or MRI scans).

[0093] The working principle and effects of the above technical solution are as follows: Input the current real-time body surface data (or point cloud data) of the patient, as well as the reference data in the radiotherapy plan (such as the organ contour or point cloud obtained from CT or MRI scans). Preprocess the input data, such as denoising, standardization, etc., to ensure that the data quality meets the requirements of network input. Through the encoder part of the RegFormer network, feature extraction is performed on the input point cloud data. The features of each point interact with the features of all other points to generate a globally context-aware feature vector. Using the decoder part of the RegFormer network, the deformation field of the point cloud is predicted based on the extracted feature vector. The deformation field describes the displacement vector of each point from the original position to the deformed position. The deformation field output by the RegFormer network is discrete, that is, the deformation information is only given for the input point cloud data.

[0094] To obtain the deformation information of the entire volume, mathematical tools such as the finite element method (FEM) or elastic deformation model are needed. These models can calculate the deformation inside the volume based on the deformation field on the point cloud. The volume is divided into multiple finite elements, and the deformation within each element can be calculated according to the deformation field on the point cloud by interpolation. Assuming the volume is an elastic body, according to the principles of elasticity mechanics, the deformation field inside the volume is obtained by solving partial differential equations.

[0095] Since the RegFormer network processes point cloud data and SGRT collects the body surface information of the patient, the deformation information of the surface contour is obtained first. Through the finite element method or elastic deformation model, the overall deformation of the internal organ can be inferred based on the deformation information of the surface contour. Visualize the reconstructed three-dimensional deformation field to facilitate doctors to intuitively understand the patient's deformation situation. According to the reconstructed deformation field, adjust the radiotherapy plan accordingly to ensure the accuracy and safety of radiotherapy.

[0096] In summary, the process of using the RegFormer network for deformation registration and reconstructing a three-dimensional deformation field through surface contour deformation is a process that combines deep learning, mathematical physics models, and medical applications; this method can accurately capture the deformation information of patients and provide strong support for the formulation and adjustment of radiotherapy plans.

[0097] In some embodiments, obtaining the overall deformation of internal organs through surface contour deformation and reconstructing a three-dimensional deformation field includes:

[0098] Collect historical data of multiple patients to establish a machine learning model; the historical data includes body surface contour data before and after treatment, internal organ imaging data; and surface contour deformation during patient positioning.

[0099] Obtain the overall deformation of internal organs through the machine learning model based on the real-time surface contour deformation of the current patient.

[0100] The working principle of the above technical solution is as follows:

[0101] Collect body surface contour data before and after treatment of a large number of patients (such as point cloud data obtained through the SGRT system), internal organ imaging data (such as CT or MRI scans), and corresponding treatment plan data; clean, align, and standardize the collected data to ensure the quality and consistency of the data; including removing noise, filling in missing values, calibrating data at different time points, and scaling the data to an appropriate range; extract key features from the body surface contour data and internal organ imaging data, and these features should be able to reflect the relationship between body surface and internal organ deformation. Features can include the shape, size, curvature, volume, position, etc. of the contour. Select a machine learning model (such as random forest, support vector machine, neural network, etc.) according to the characteristics of the data and the complexity of the problem. Use the preprocessed historical data to train the model and learn the mapping relationship between body surface contour deformation and the overall deformation of internal organs.

[0102] During the positioning process, continuously collect real-time surface contour deformation data of the current patient through the SGRT system or other body surface scanning technologies. Input the real-time surface contour deformation data into the trained machine learning model, and the model will output the predicted overall deformation of internal organs. Use the predicted deformation of internal organs, combined with medical image registration technology, to map the deformation into three-dimensional space and construct a three-dimensional deformation field; it is necessary to align the predicted deformation with a reference image (such as a CT or MRI image before treatment). Evaluate the performance of the model using cross-validation or other validation methods to ensure that the model has generalization ability. Collect more real-time data and feedback for continuous optimization and iteration of the model.

[0103] The effects of the above technical solutions are as follows: This method can predict the deformation of internal organs in real time through surface deformation, obtain an accurate three-dimensional deformation field, and improve the positioning accuracy; accurate deformation prediction helps to reduce the repeated movement and positioning requirements of the patient's body due to inaccurate positioning, and improve the treatment comfort and patient experience. Accurate deformation prediction can reduce the repeated adjustment and calibration time during the positioning process, thereby optimizing the pre-treatment preparation process and improving the overall treatment efficiency.

[0104] In some embodiments, the display through VR for verifying SGRT positioning includes:

[0105] Import the three-dimensional model of the patient into the VR device to construct a virtual environment;

[0106] Fuse the patient position information with the three-dimensional model in the virtual environment in real time and display it in the VR device.

[0107] In some embodiments, the fusing the patient position information with the three-dimensional model in the virtual environment in real time and displaying it in the VR device includes:

[0108] Through the VR device, present the patient's current position, the target position in the radiotherapy plan, and the coincidence degree between the patient's current position and the target position in the radiotherapy plan;

[0109] Through the VR device, display in real time the three-dimensional model constructed by imaging, contour data, and planning data. The VR device is integrated with software and hardware, and the hardware may include VR glasses.

[0110] In some embodiments, after fusing the patient position information with the three-dimensional model in the virtual environment in real time and displaying it in the VR device, the method further includes:

[0111] According to the coincidence degree, perform positioning adjustment to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0112] The principle and effects of the above technical solution are as follows: First, generate a three-dimensional model from the patient's CT data, contour data, and treatment plan data. Then, import the three-dimensional model into a VR device to construct a virtual environment highly similar to the patient's anatomical structure; obtain the patient's position information in real time through the SGRT system; and perform real-time fusion of the patient's position information with the three-dimensional model in the virtual environment. This means that the patient's actual position can be corresponding to the three-dimensional model in the virtual environment in real time; display the fused scene to the doctor through a VR device, such as a VR glasses; the doctor can wear the VR glasses and observe a stereoscopic scene that includes both the patient's actual position and the virtual three-dimensional model. Through the VR glasses, the doctor can intuitively observe the matching degree between the patient's positioning and the position of the planned CT, especially the isocenter position. The doctor can clearly see whether the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position. In the VR glasses, it is also possible to display in real time the three-dimensional model constructed from the imaging, contour data, and treatment plan data, as well as the relevant information in the patient's positioning coordinate system. If a mismatch or deviation is observed, the doctor can immediately make adjustments to ensure that the patient's positioning meets the requirements of the radiotherapy plan and is consistent with the treatment isocenter position. By using VR technology for positioning verification, it is possible to greatly reduce human errors and machine errors and improve the accuracy of radiotherapy; VR technology enables doctors to observe and analyze the patient's positioning situation more quickly and intuitively, thereby improving work efficiency.

[0113] An embodiment of the present invention provides a VR-based SGRT positioning verification device, which includes:

[0114] A treatment plan acquisition module, configured to acquire treatment plan-related data of a patient; the treatment plan-related data includes the patient's surface contour data, contour data, treatment isocenter position, prescription dose, etc.;

[0115] A model construction module, configured to perform three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient;

[0116] A real-time data acquisition module, configured to acquire the patient's real-time body surface data through SGRT; the real-time body surface data includes the patient's real-time body surface contour, position, and three-dimensional surface shape;

[0117] A rigid body registration module, configured to perform rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0118] A deformation registration module, configured to perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstruct a three-dimensional deformation field according to the deformation registration result;

[0119] A verification module, which is used to deform the image and treatment plan-related data into the constructed three-dimensional positioning coordinate system by using the reconstructed three-dimensional deformation field, and display it through VR to verify the SGRT positioning.

[0120] In some embodiments, the device further includes:

[0121] A first displacement module, which is used to obtain a first displacement according to the rigid registration result; and perform a bed movement operation through the first displacement.

[0122] In some embodiments, the device further includes:

[0123] A first displacement module, which is used to obtain a second displacement according to the deformation registration result; and perform a bed movement operation through the second displacement.

[0124] In some embodiments, the rigid registration module performs rigid registration by using a point cloud-based rigid registration method, wherein the loss function is to minimize the distance from point to plane:

[0125]

[0126] where Loss is the loss function, p i is the real-time body surface data point of the patient, q i is the point in the skin contour point cloud of the patient's delineated data corresponding to p i n i is the normal vector at the target point q i R is the rotation matrix, t is the translation vector; N is the number of samples of the data points; for each pair of corresponding points p i and q i , calculate the distance between them, and project this distance onto the normal vector n i , then sum the squares of the distances of all points, and minimize this value by optimizing R and t.

[0127] In some embodiments, the deformation registration module performs deformation registration through the RegFormer network, and obtains the overall deformation of the internal organs through surface contour deformation, and reconstructs a three-dimensional deformation field.

[0128] In some embodiments, the verification module includes:

[0129] An import unit, which is used to import the three-dimensional model of the patient into the VR device to construct a virtual environment;

[0130] A fusion display unit, which is used to fuse the patient position information with the three-dimensional model in the virtual environment in real time and display it in the VR device.

[0131] In some embodiments, the fusion display unit includes:

[0132] A first display unit for presenting, via the VR device, the current position of the patient, the target position in the radiotherapy plan, and the degree of coincidence between the current position of the patient and the target position in the radiotherapy plan;

[0133] A second display unit for displaying, via the VR device, a three-dimensional model constructed from images, contour data, and planning data in real time.

[0134] In some embodiments, after being displayed in the VR device, the apparatus further includes:

[0135] An adjustment module for performing positioning adjustment according to the degree of coincidence to ensure that the current position of the patient coincides with the target position in the radiotherapy plan, i.e., the treatment isocenter position.

[0136] The working principle and effects of the above technical solution are the same as those in the method embodiments of the present invention and will not be elaborated herein.

[0137] An embodiment of the present invention provides a treatment system, the system includes:

[0138] A radiotherapy device for performing radiotherapy on a patient;

[0139] The positioning verification device described in the embodiment of the present invention for verifying the position of the patient.

[0140] An embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of any one of the methods described in the embodiments of the present invention or the functions of the device described in the embodiments of the present invention.

[0141] An embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, and when the computer program is executed, it implements the steps of the method in the embodiment of the present invention. Its specific implementation manner is the same as the implementation manner and the achieved technical effects described in the above method embodiments, and some contents will not be elaborated.

[0142] In the present invention, a readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0143] A computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above. The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the C language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on an associated device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0144] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limitations of the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and purposes of the present invention, and all such changes should fall within the protection scope of the claims of the present invention.

Claims

1. The VR-based SGRT positioning verification method is characterized by: The method comprises: Acquiring treatment plan-related data for the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position; Performing three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient; Acquire the patient's real-time body surface data through SGRT; Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model; Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result; Using the reconstructed 3D deformation field, the image and treatment plan data are deformed into the constructed 3D positioning coordinate system and displayed through VR to verify the SGRT positioning. Among them, a point cloud-based rigid body registration method is used to perform rigid body registration, where the loss function is to minimize the distance from the point to the surface; Deformable registration is performed through the RegFormer network. By deforming the surface contour, the overall deformation of the internal organs is obtained and the three-dimensional deformation field is reconstructed. Before the deformable registration, the method further includes: According to the rigid body registration result, the first displacement is obtained; The bed moving operation is performed through the first displacement.

2. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The method further comprises: According to the deformation registration result, the second displacement is obtained; The bed moving operation is performed through the second displacement.

3. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The loss function is: Among them, Loss is the loss function, p i is the patient’s real-time surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i is the target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points.

4. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The verification of the SGRT positioning by displaying it through VR includes: Import the patient's three-dimensional model into the VR system to build a virtual environment; The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device.

5. The VR-based SGRT positioning verification method according to claim 4, characterized in that: The real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying the information on the VR glasses includes: Presenting the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan through the VR device; The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

6. The VR-based SGRT positioning verification method according to claim 5, characterized in that: After the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR glasses, the method further includes: Based on the degree of coincidence, the positioning is adjusted to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

7. The VR-based SGRT positioning verification device is characterized by: The device comprises: A treatment plan acquisition module is used to obtain the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, dose data, and treatment center position; A model building module, configured to perform three-dimensional visualization reconstruction using the treatment plan-related data to construct a three-dimensional model of the patient; A real-time data acquisition module is used to obtain the patient's real-time body surface data through SGRT; A rigid body registration module, configured to perform rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model; a deformation registration module, configured to perform deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and to reconstruct a three-dimensional deformation field according to the deformation registration result; The verification module is used to use the reconstructed 3D deformation field to deform the image and treatment plan-related data into the constructed 3D positioning coordinate system and display it through VR to verify the SGRT positioning; Among them, a point cloud-based rigid body registration method is used to perform rigid body registration, where the loss function is to minimize the distance from the point to the surface; Deformable registration is performed through the RegFormer network. By deforming the surface contour, the overall deformation of the internal organs is obtained and the three-dimensional deformation field is reconstructed. Before the deformable registration, the method further includes: According to the rigid body registration result, the first displacement is obtained; The bed moving operation is performed through the first displacement.

8. A treatment system, characterized in that The system comprises: Radiotherapy devices, used to deliver radiation therapy to patients; The positioning verification device described in claim 7 is used to verify the position of the patient.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the steps of the method according to any one of claims 1 to 6.

Images