VR-based SGRT positioning verification method and related equipment

Abstract

The invention discloses a VR-based SGRT positioning verification method and related equipment, and relates to the technical field of radiotherapy, and the method comprises the steps: obtaining the treatment plan related data of a patient; performing three-dimensional visualization reconstruction by using the treatment plan related data, and constructing a three-dimensional model of the patient; acquiring real-time body surface data of the patient through the SGRT; performing rigid body registration and deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to a deformation registration result; using the reconstructed three-dimensional deformation field to deform the image and the treatment plan related data into the constructed three-dimensional positioning coordinate system, and displaying the image and the treatment plan related data through VR to verify the SGRT positioning; vR and SGRT are combined, through rigid body registration and deformation registration strategies, the precision of positioning verification can be enhanced, real-time interaction feedback can be provided, and the efficiency and effect of radiotherapy are improved.

Description

Technical Field

[0001] The present invention relates to the field of radiotherapy technology, and in particular to a VR-based SGRT positioning verification method and related equipment. Background Art

[0002] In radiation therapy (RT), the accuracy of patient positioning is extremely important, especially in high-dose irradiation situations. Any positioning error may cause damage to normal tissue or miss the tumor target area. To ensure positioning accuracy, traditional methods rely on image-guided radiation therapy (IGRT), such as CT or X-rays, which use real-time imaging to confirm whether the patient's internal structure is consistent with the treatment plan. However, the disadvantage of IGRT is that it requires repeated imaging, resulting in additional exposure of the patient to radiation, and the operation process is complicated.

[0003] As a new technology, surface-guided radiation therapy (SGRT) uses optical means for non-contact positioning, avoiding the problem of increasing radiation dose in traditional imaging technology. SGRT uses multiple cameras to capture the patient's three-dimensional surface shape and position in real time, compares it with the reference 3D image surface in the treatment plan, and uses rigid or elastic registration algorithms to correct the patient's position, so that the SGRT system can continuously monitor the patient's body surface changes without radiation, which is particularly suitable for dynamic or frequently adjusted treatments.

[0004] The SGRT system relies on a combination of multiple cameras and projectors to generate and register real-time 3D surface point cloud data of patients. After obtaining the patient's 3D surface, the system aligns it with the planned reference image, calculates deviations and adjusts the position in time. This non-ionizing property allows SGRT to be used repeatedly throughout the radiotherapy process without increasing the patient's radiation exposure. The SGRT system has been widely used in clinical scenarios such as breast cancer, abdomen, head and neck, intracranial tumors, and pediatric tumors.

[0005] Although SGRT can monitor the patient's body surface position in real time, it cannot directly provide detailed information about internal organs or tumors, especially when deep or surface movement is not synchronized with internal tissues; when the patient's surface changes, the collected surface contour cannot be accurately aligned with the surface contour in the imported plan, and SGRT surface information alone cannot effectively verify the positioning results; surface contour data has the greatest change in positioning, and the information that can be used for fusion alignment is the least, and there are problems in the alignment process, especially when two inconsistent contours are rigidly aligned and measured, a global optimal solution cannot be obtained; SGRT visualization does not include plan-related information such as 3D imaging, outlining, dose, etc., and cannot provide an intuitive reference for the accuracy of radiotherapy execution. The existing technology of radiotherapy precise field-by-field positioning method based on virtual intelligent medical platform exists by posting marker points and using gold standard registration (VR and patient registration, irradiation field and accelerator registration) as the registration method, and it is necessary to add additional markers; the existing technology of radiotherapy positioning verification method and device based on virtual intelligent medical platform exists by multiple gold mark identification to find the isocenter position, and the isocenter registration method is used for registration, which requires the addition of additional markers, and the isocenter registration method cannot calculate the bed movement angle; the existing technology of radiotherapy positioning method and system exists only through surface rigid body registration, and surface registration cannot describe the deformation of tissues in the body, and cannot perform accurate registration. The movement of the target area and organs at risk in the human body is often inconsistent with the movement direction and movement of the surface, so the surface displacement cannot be directly applied to the target area or isocenter. The displacement of the surface does not truly reflect the transformation of the isocenter. Summary of the invention

[0006] Based on the above problems, the purpose of the present invention is to provide a VR-based SGRT positioning verification method and related equipment, combining VR with SGRT, through rigid body registration plus deformation registration strategy, which can not only enhance the accuracy of positioning verification, but also provide real-time interactive feedback, which is beneficial to ensure the efficiency and effectiveness of radiotherapy.

[0007] The purpose of the present invention is achieved by the following technical solutions:

[0008] In a first aspect, the present invention proposes a VR-based SGRT positioning verification method, the method comprising:

[0009] Acquiring treatment plan-related data of the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position;

[0010] Using the treatment plan-related data to perform three-dimensional visualization reconstruction to construct a three-dimensional model of the patient;

[0011] Acquire the patient's real-time body surface data through SGRT;

[0012] Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0013] Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0014] Using the reconstructed three-dimensional deformation field, the image and treatment plan-related data are deformed into the constructed three-dimensional positioning coordinate system and displayed through VR to verify the SGRT positioning.

[0015] Preferably, before the deformation registration, the method further comprises:

[0016] According to the rigid body registration result, the first displacement is obtained;

[0017] The bed moving operation is performed through the first displacement.

[0018] Preferably, the method further comprises:

[0019] According to the deformation registration result, a second displacement is obtained;

[0020] The bed moving operation is performed through the second displacement.

[0021] Preferably, a point cloud-based rigid body registration method is used for rigid body registration.

[0022] Preferably, the rigid body registration includes calculating a loss function, and the loss function is:

[0023]

[0024] Among them, Loss is the loss function, p i is the patient’s real-time body surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i The target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points.

[0025] Preferably, deformation registration is performed through the RegFormer network, and the overall deformation of the internal organ is obtained through surface contour deformation to reconstruct the three-dimensional deformation field.

[0026] Preferably, the displaying by VR to verify the SGRT positioning includes:

[0027] Import the patient's three-dimensional model into the VR device to build a virtual environment;

[0028] The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device.

[0029] Preferably, the real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying it in a VR device includes:

[0030] The VR device is used to present the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan;

[0031] The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

[0032] Preferably, after the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device, the method further comprises:

[0033] According to the overlap, the positioning adjustment is performed to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0034] In a second aspect, the present invention provides a VR-based SGRT positioning verification device, the device comprising:

[0035] A treatment plan acquisition module is used to acquire the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, dose data and treatment center position;

[0036] A model building module, used to perform three-dimensional visualization reconstruction using the treatment plan-related data to build a three-dimensional model of the patient;

[0037] A real-time data acquisition module is used to acquire the patient's real-time body surface data through SGRT;

[0038] A rigid body registration module, used for rigid body registration of the real-time body surface data with the surface contour data in the three-dimensional model;

[0039] A deformation registration module, used for performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0040] The verification module is used to use the reconstructed three-dimensional deformation field to deform the image and treatment plan related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning.

[0041] In a third aspect, the present invention provides a treatment system, the system comprising:

[0042] Radiotherapy devices, used to deliver radiation therapy to patients;

[0043] The positioning verification device of the present invention is used to verify the position of a patient.

[0044] In a fourth aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the aforementioned method of the present invention or the functions of the apparatus of the present invention when executing the computer program.

[0045] In a fifth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the steps of the aforementioned method of the present invention.

[0046] Compared with the prior art, the beneficial effects of the present invention include at least: by fusing more three-dimensional information for positioning verification, it is possible to provide more abundant convergence basis or better optimization starting point when positioning fails, thereby effectively avoiding positioning failure. Although surface contour data is important, it varies greatly and the information used for fusion registration is limited. This method introduces more three-dimensional anatomical information by combining VR technology, making the registration process more accurate and reliable. Since the movement of the target area and organs at risk in the human body is often inconsistent with the surface movement, it is difficult for traditional methods to accurately reflect the change of the isocenter. This method introduces three-dimensional information through VR technology, and can observe and adjust the change process of the patient's position and isocenter in real time, thereby significantly improving the positioning accuracy and ensuring the accuracy of treatment; SGRT can usually only provide a rigid body change comparison on the surface, while the changes in tissues, organs and target areas in the body often have more deformations. With the help of VR technology and deformation field, this method can truly simulate the anatomical changes in the target area caused by the positioning process, provide doctors with more comprehensive treatment information, and ensure accurate and reliable radiotherapy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of a VR-based SGRT positioning verification method according to an embodiment of the present invention;

[0048] Figure 2 It is a flowchart of applying the VR-based SGRT positioning verification method according to an embodiment of the present invention to radiotherapy;

[0049] Figure 3 It is a schematic diagram of a VR-based SGRT positioning verification device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.

[0051] The words expressing positions and directions described in the present invention are all explained with reference to the accompanying drawings as examples, but they can be changed as needed, and all such changes are included in the protection scope of the present invention.

[0052] Reference Figure 1 :The present invention provides a VR-based SGRT positioning verification method, the method comprising:

[0053] Acquiring treatment plan-related data of the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, treatment center position, and prescription dose, etc.;

[0054] Using the treatment plan-related data to perform three-dimensional visualization reconstruction to construct a three-dimensional model of the patient;

[0055] Acquiring real-time body surface data of the patient through SGRT; the real-time body surface data includes the real-time body surface contour, position and three-dimensional surface shape of the patient;

[0056] Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model;

[0057] Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model to obtain a deformation registration result, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0058] Using the reconstructed three-dimensional deformation field, the image and treatment plan related data are deformed into the constructed three-dimensional positioning coordinate system, and displayed through VR to verify the SGRT positioning. The image related data may include CT (computed tomography) images or MRI (magnetic resonance imaging) images of the patient.

[0059] Reference Figure 2 As shown, after the positioning is verified, if the positioning verification is passed, radiotherapy is started until the radiotherapy is completed. If the positioning verification is not passed, the process returns to the aforementioned step of obtaining the patient's real-time body surface data through SGRT, i.e., the step of real-time body surface data acquisition, and calculates the bed transfer parameters.

[0060] The working principle of the above technical solution is as follows: First, relevant data is obtained from the patient's treatment plan. These data are the basis of radiotherapy, including the patient's surface contour data (i.e., the shape and position information of the patient's body surface), delineation data (such as the position and shape of tumors and key organs), dose data (i.e., prescribed dose), and treatment isocenter position (i.e., the center point of radiotherapy). The surface contour data is used as the fusion benchmark for SGRT real-time measurement data, and the isocenter position is used to evaluate the actual need for the conversion of actual bed transfer parameters; in addition, images, target area or organ at risk delineation and prescribed dose will be used for VR three-dimensional fusion display.

[0061] Using the acquired treatment plan-related data, computer graphics technology is used to perform 3D visualization reconstruction to build a 3D model that is highly similar to the patient's actual body surface and internal anatomical structure. This model is the basis for subsequent registration and verification.

[0062] During the positioning process, SGRT is used to obtain the patient's body surface data in real time; the SGRT multi-camera array quickly captures the patient's body surface features, and the projector is responsible for projecting specific light patterns, which, after interacting with the patient's body surface, will be captured by the camera and converted into high-precision three-dimensional point cloud data. This process ensures that every subtle change in the patient's body surface can be captured and recorded by the system in real time. On this basis, SGRT can generate key data such as the patient's body surface contour, position, and three-dimensional surface shape in real time; they reflect the patient's actual status during the positioning process.

[0063] The real-time acquired body surface data, i.e., real-time body surface data, is rigidly registered with the surface contour data in the three-dimensional model to obtain the rigid body registration result; rigid body registration refers to the process of aligning corresponding points in two or more images through rigid body transformation (i.e., rotation and translation); a rigid body is an idealized physical model, and its volume and shape do not change after being subjected to external forces. Therefore, in rigid body registration, the distance and direction relationship between the points in the image remain unchanged after being transformed; the main purpose of rigid body registration is to align the real-time acquired body surface data (such as using optical surface scanning of patient position) with the surface contour data in the three-dimensional model; the purpose of this step is to preliminarily align the real-time data and the model data, eliminate large position shifts, and provide a basis for subsequent deformation registration.

[0064] On the basis of rigid body registration, deformation registration is performed. Deformation registration takes into account the deformation of the patient's body surface due to factors such as breathing, so it can more accurately reflect the actual changes in the patient's body surface. Through the deformation registration of the body surface point cloud, the three-dimensional shape grid of the body surface can be calculated, and according to the principle of differential homeomorphism, the three-dimensional deformation field can be reconstructed, that is, the vector field that describes the deformation of the patient's body surface from the model state to the real-time state.

[0065] Using the reconstructed three-dimensional deformation field, the deformed patient images and treatment plan-related data are mapped to the constructed three-dimensional positioning coordinate system; this coordinate system is the reference system used to determine the patient's position and posture during radiotherapy; the overall deformation of the patient's internal organs in the three-dimensional model (such as organ deformation in CT and MRI scans) is inferred from the body surface deformation field.

[0066] Finally, it is displayed through VR technology. VR technology can provide an immersive environment, allowing doctors to intuitively see the patient's body surface, changes in relevant data of the treatment plan, and the relationship between them. In this way, doctors can verify the SGRT positioning and ensure the accuracy of radiotherapy.

[0067] The effects of the above technical solution are:

[0068] By combining SGRT technology and three-dimensional visualization reconstruction, the present invention can obtain the patient's surface data in real time and accurately align it with the pre-constructed three-dimensional model; the present invention does not need to add additional markers for alignment; the rigid body to deformation method more accurately reflects the deformation of organs at risk (OAR) and target areas; through the dual alignment mechanism of rigid body alignment and deformation alignment, the accuracy of radiotherapy is significantly improved, ensuring the accurate matching of the treatment plan with the actual condition of the patient.

[0069] Using VR (Virtual Reality) technology for display, the present invention provides doctors with an intuitive and immersive verification environment. Doctors can clearly see the patient's body surface deformation, treatment plan and the relationship between them, making it easier to make accurate decisions; it helps doctors to promptly discover and correct possible deviations during radiotherapy and improve treatment effects; the present invention can build a personalized three-dimensional model based on the patient's actual body surface data and internal anatomical structure, and perform precise registration and verification, which helps doctors develop more personalized radiotherapy plans for patients and better meet the patient's treatment needs.

[0070] In summary, the present invention significantly improves the accuracy, efficiency and safety of radiotherapy by combining multiple advanced technologies, provides doctors with more intuitive and accurate decision support, and also brings better treatment effects and experience to patients.

[0071] In some embodiments, before the deformation registration, the method further includes:

[0072] According to the rigid body registration result, the first displacement is obtained;

[0073] Through the first displacement, a bed moving operation is performed, that is, the bed carrying the patient is moved by the distance of the first displacement.

[0074] The working principle and effect of the above technical solution are:

[0075] Through the rigid body registration algorithm, the displacement difference between the real-time body surface data and the three-dimensional model, i.e., the first displacement, can be calculated; this displacement difference reflects the deviation between the patient and the ideal position during the positioning process; the main purpose of the bed moving operation is to adjust the patient's position according to the first displacement obtained in the rigid body registration result, so that it matches the ideal position in the radiotherapy plan, i.e., the expected position for implementing the treatment plan. After obtaining the first displacement, the position of the treatment bed is automatically or manually adjusted according to the first displacement information; this adjustment process usually includes translation along the X-axis, Y-axis, and Z-axis, as well as possible rotation adjustment, to ensure that the patient's body surface data is completely consistent with the requirements in the radiotherapy plan. After the bed moving operation is completed, real-time body surface data acquisition and registration verification are usually performed again to ensure the accuracy and effectiveness of the bed moving operation. This helps to ensure the accuracy and safety during the radiotherapy process. Performing rigid body registration and bed moving operations before deformation registration can make the subsequent deformation registration process more accurate and efficient. At the same time, this also provides more reliable data support for the formulation and adjustment of radiotherapy plans, which helps to optimize the entire radiotherapy process.

[0076] In some embodiments, the method further comprises:

[0077] According to the deformation registration result, a second displacement is obtained;

[0078] Through the second displacement, the bed moving operation is performed, that is, the bed carrying the patient is moved by the distance of the second displacement.

[0079] The principle and effect of the above technical solution are: during the positioning process, the patient's body surface morphology may undergo slight changes due to factors such as breathing and muscle tension; deformation registration can monitor and accurately calculate these changes in real time.

[0080] By collecting real-time data of the patient's body surface and comparing it with the three-dimensional model in the radiotherapy plan, the deformation registration algorithm can calculate the morphological difference between the two, that is, the deformation registration result. The deformation registration result reflects the actual changes in the patient's body surface morphology during radiotherapy and is an important basis for subsequent bed transfer operations.

[0081] Based on the deformation registration results, the displacement difference between the real-time body surface morphology and the ideal morphology in the radiotherapy plan can be calculated. This displacement difference is called the second displacement.

[0082] The second displacement is a vector containing multiple dimensions (such as the X-axis, Y-axis, Z-axis, and possible rotation angles), which describes the position and direction that the patient needs to adjust during radiotherapy. Based on the information of the second displacement, the position of the radiotherapy bed can be adjusted automatically or manually to correct the deviation of the patient's body surface morphology during radiotherapy. The bed moving operation usually includes translation along the X-axis, Y-axis, and Z-axis, as well as possible rotation adjustment to ensure that the patient's body surface morphology matches the requirements in the radiotherapy plan. During the bed moving operation, it is necessary to closely monitor the changes in the patient's body surface morphology and make real-time adjustments as needed to ensure the accuracy and safety of radiotherapy. After the bed moving operation is completed, real-time surface data acquisition and deformation registration verification are usually performed again to ensure the accuracy and effectiveness of the bed moving operation. If it is found that there is still a displacement deviation, the bed moving operation can be performed again according to the new deformation registration results until the patient's body surface morphology is completely consistent with the requirements in the radiotherapy plan.

[0083] In summary, the working principle of obtaining the second displacement based on the deformation registration result and performing the bed moving operation through the second displacement is a complex and precise process. It relies on advanced deformation registration technology and precise bed moving equipment to ensure the accuracy and safety of radiotherapy.

[0084] In some embodiments, the VR-based SGRT positioning verification method uses a point cloud-based rigid body registration method to perform rigid body registration, and the rigid body registration includes calculating a loss function, and the loss function is defined as minimizing the point-to-plane distance (Point-to-Plane Distance):

[0085]

[0086] Among them, Loss is the loss function, which is used to calculate the bed transfer parameters, p i is the patient’s real-time body surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i The target point q i The normal vector at the position, R is the rotation matrix, t is the translation vector; N is the number of samples of data points; for each pair of corresponding points p i and q i , calculate the distance between them and project this distance onto the normal vector n i Then the square sum of the distances of all points is calculated and this value is minimized by optimizing R and t. The bed moving parameters may include translation distances and / or rotation angles in the three directions of X, Y, and Z.

[0087] The working principle of the above technical solution is: the patient's body surface point cloud data is collected in real time through SGRT. These point cloud data contain the geometric shape and position information of the patient's body surface. The surface contour point cloud is extracted from the patient's CT or other medical imaging data. These data represent the ideal body surface morphology of the patient in the radiotherapy plan. The point cloud-based rigid body registration method matches the real-time body surface data with the outline data, which usually involves finding the correspondence between the two sets of point clouds, that is, determining the one-to-one correspondence between the body surface points and the contour points.

[0088] In order to evaluate the accuracy of the registration, the aforementioned loss function is defined, which minimizes the point-to-plane distance. This loss function calculates the loss of each pair of corresponding points (p i and q i ) and project this distance to the target point q i The normal vector n at i Then, the squares of the projection distances of all points are summed to obtain the total loss value. The loss function is minimized by optimizing the rotation matrix R and the translation vector t, which usually involves an iterative algorithm, such as the iterative closest point (ICP) algorithm or its variants, to gradually adjust R and t until the best registration result is found.

[0089] The effects of the above technical solution are: by minimizing the distance from point to plane, the spatial relationship between real-time body surface data and skin delineation data can be more accurately reflected; compared with the traditional point-to-point registration method, point-to-plane registration can consider more geometric information, thereby obtaining more accurate registration results; the point-to-plane registration method has better robustness to noise and abnormal points. In practical applications, both real-time body surface data and skin delineation data may be interfered by noise, and this method can reduce the impact of these noises on the registration results. In addition, since the information of the normal vector is taken into account, even if the body surface data is partially missing or deformed, a reasonable registration result can be obtained through the constraint of the normal vector. Compared with other complex registration methods, this method is more efficient in calculation and is suitable for application scenarios with high real-time requirements. In the field of radiotherapy, this method can ensure that the patient's body surface morphology matches the requirements in the radiotherapy plan, thereby improving the accuracy of radiotherapy.

[0090] In some embodiments, deformation registration is performed through the RegFormer network; the overall deformation of the internal organ is obtained through surface contour deformation, and the three-dimensional deformation field is reconstructed.

[0091] RegFormer is a Transformer-based network. Through the self-attention mechanism, the features of each point interact with the features of all other points to generate a global context-aware feature vector, which is very beneficial for deformation registration, especially when there are different degrees of deformation in different areas of the point cloud, and can better capture the overall change trend of the point cloud.

[0092] After the deformation field of the point cloud is obtained, the finite element method (FEM) or elastic deformation model is used to describe the deformation of the entire volume. These models can infer the deformation inside the volume based on the deformation field on the point cloud. That is, the overall deformation of the internal organs (such as organ deformation in CT and MRI scans) can be inferred from the external surface deformation field captured by the point cloud.

[0093] The working principle and effect of the above technical solution are as follows: input the patient's current real-time surface data (or point cloud data) and the reference data in the radiotherapy plan (such as organ contours or point clouds obtained by CT or MRI scans). The input data is preprocessed, such as denoising and standardization, to ensure that the data quality meets the requirements of the network input. The encoder part of the RegFormer network is used to extract features of the input point cloud data. The features of each point interact with the features of all other points to generate a global context-aware feature vector. Using the decoder part of the RegFormer network, the deformation field of the point cloud is predicted based on the extracted feature vector. The deformation field describes the displacement vector of each point from its original position to its deformed position. The deformation field output by the RegFormer network is discrete, that is, only deformation information is given for the input point cloud data.

[0094] In order to obtain the deformation information of the entire volume, mathematical tools such as the finite element method (FEM) or elastic deformation model are needed. These models can infer the deformation inside the volume based on the deformation field on the point cloud. The volume is divided into multiple finite elements, and the deformation inside each element can be calculated based on the deformation field on the point cloud by interpolation method. Assuming that the volume is an elastic body, according to the principle of elastic mechanics, the deformation field inside the volume is obtained by solving partial differential equations.

[0095] Since the RegFormer network processes point cloud data and SGRT collects patient surface information, the first thing obtained is the deformation information of the surface contour. Through the finite element method or elastic deformation model, the overall deformation of the internal organs can be inferred based on the deformation information of the surface contour. The reconstructed three-dimensional deformation field is visualized to facilitate doctors to intuitively understand the patient's deformation. According to the reconstructed deformation field, the radiotherapy plan is adjusted accordingly to ensure the accuracy and safety of radiotherapy.

[0096] In summary, the process of using the RegFormer network for deformation registration and reconstructing the three-dimensional deformation field through surface contour deformation is a process that combines deep learning, mathematical physics models and medical applications; this method can accurately capture the patient's deformation information and provide strong support for the formulation and adjustment of radiotherapy plans.

[0097] In some embodiments, the step of obtaining the overall deformation of the internal organ by deforming the surface contour and reconstructing the three-dimensional deformation field includes:

[0098] Collect historical data of multiple patients and establish a machine learning model; the historical data includes body surface contour data before and after treatment, internal organ imaging data, and surface contour deformation during patient positioning;

[0099] Based on the real-time surface contour deformation of the current patient, the overall deformation of the internal organs is obtained through a machine learning model.

[0100] The working principle of the above technical solution is:

[0101] Collect a large number of patients' pre- and post-treatment body contour data (such as point cloud data obtained through the SGRT system), internal organ imaging data (such as CT or MRI scans), and corresponding treatment plan data; clean, align, and standardize the collected data to ensure data quality and consistency; including removing noise, filling missing values, calibrating data at different time points, and scaling the data to an appropriate range; extract key features from body contour data and internal organ imaging data, which should be able to reflect the relationship between body surface and internal organ deformation. Features can include the shape, size, curvature, volume, position, etc. of the contour. According to the characteristics of the data and the complexity of the problem, select a machine learning model (such as random forest, support vector machine, neural network, etc.). Use the preprocessed historical data to train the model to learn the mapping relationship between body surface contour deformation and the overall deformation of internal organs.

[0102] During the positioning process, the real-time surface contour deformation data of the current patient is continuously collected through the SGRT system or other body surface scanning technologies. The real-time surface contour deformation data is input into the trained machine learning model, and the model will output the predicted overall deformation of the internal organs. Using the predicted internal organ deformation, combined with medical image registration technology, the deformation is mapped into three-dimensional space to construct a three-dimensional deformation field; the predicted deformation needs to be aligned with the reference image (such as CT or MRI images before treatment). Use cross-validation or other validation methods to evaluate the performance of the model to ensure that the model has generalization capabilities. Collect more real-time data and feedback for continuous optimization and iteration of the model.

[0103] The effects of the above technical solution are as follows: the method can predict the deformation of internal organs through surface deformation in real time, obtain accurate three-dimensional deformation field, and improve positioning accuracy; accurate deformation prediction helps reduce the repeated movement and positioning requirements of the patient's body due to inaccurate positioning, and improves the comfort of treatment and the patient's experience. Accurate deformation prediction can reduce the repeated adjustment and verification time during the positioning process, thereby optimizing the preparation process before treatment and improving the overall treatment efficiency.

[0104] In some embodiments, displaying by VR to verify the SGRT positioning includes:

[0105] Import the patient's three-dimensional model into the VR device to build a virtual environment;

[0106] The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device.

[0107] In some embodiments, the real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying the same in a VR device includes:

[0108] The VR device is used to present the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan;

[0109] The VR device can display images, outline data, and a three-dimensional model constructed by planning data in real time. The VR device integrates software and hardware, and the hardware can include VR glasses.

[0110] In some embodiments, after the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device, the method further includes:

[0111] According to the overlap, the positioning adjustment is performed to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0112] The principle and effect of the above technical solution are as follows: first, the patient's CT data, outline and planning data are used to generate a three-dimensional model. Then the three-dimensional model is imported into the VR device to build a virtual environment that is highly similar to the patient's anatomical structure; the patient's position information is obtained in real time through the SGRT system; the patient's position information is fused with the three-dimensional model in the virtual environment in real time. This means that the patient's actual position can be matched with the three-dimensional model in the virtual environment in real time; the fused scene is displayed to the doctor through a VR device, such as VR glasses; the doctor can wear VR glasses and observe a stereoscopic scene that includes both the patient's actual position and the virtual three-dimensional model. Through VR glasses, the doctor can intuitively observe the degree of match between the patient's positioning and the planned CT position, especially the isocenter position. The doctor can clearly see whether the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position. In VR glasses, the three-dimensional model constructed by the image, outline data and planning data, as well as the relevant information in the patient's positioning coordinate system can also be displayed in real time. If a mismatch or deviation is observed, the doctor can make adjustments immediately to ensure that the patient's positioning meets the requirements of the radiotherapy plan and is consistent with the treatment isocenter position. Position verification through VR technology can greatly reduce human errors and machine errors and improve the accuracy of radiotherapy; VR technology enables doctors to observe and analyze the patient's positioning more quickly and intuitively, thereby improving work efficiency.

[0113] An embodiment of the present invention provides a VR-based SGRT positioning verification device, the device comprising:

[0114] A treatment plan acquisition module is used to acquire the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, treatment center position and prescription dose, etc.;

[0115] A model building module, used to perform three-dimensional visualization reconstruction using the treatment plan-related data to build a three-dimensional model of the patient;

[0116] A real-time data acquisition module, used to acquire the patient's real-time body surface data through SGRT; the real-time body surface data includes the patient's real-time body surface contour, position and three-dimensional surface shape;

[0117] A rigid body registration module, used for rigid body registration of the real-time body surface data with the surface contour data in the three-dimensional model;

[0118] A deformation registration module, used for performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result;

[0119] The verification module is used to use the reconstructed three-dimensional deformation field to deform the image and treatment plan related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning.

[0120] In some embodiments, the apparatus further comprises:

[0121] The first displacement module is used to obtain a first displacement according to the rigid body registration result; and perform a bed moving operation through the first displacement.

[0122] In some embodiments, the apparatus further comprises:

[0123] The first displacement module is used to obtain a second displacement according to the deformation registration result; and perform a bed moving operation through the second displacement.

[0124] In some embodiments, the rigid body registration module performs rigid body registration using a point cloud-based rigid body registration method, wherein the loss function is to minimize the distance from the point to the surface:

[0125]

[0126] Among them, Loss is the loss function, p i is the patient’s real-time body surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i The target point q i The normal vector at the position, R is the rotation matrix, t is the translation vector; N is the number of samples of data points; for each pair of corresponding points p i and q i , calculate the distance between them and project this distance onto the normal vector n i Then the square sum of the distances of all points is calculated and this value is minimized by optimizing R and t.

[0127] In some embodiments, the deformation registration module performs deformation registration through the RegFormer network, obtains the overall deformation of the internal organ through surface contour deformation, and reconstructs the three-dimensional deformation field.

[0128] In some embodiments, the verification module includes:

[0129] An import unit is used to import the patient's three-dimensional model into the VR device to build a virtual environment;

[0130] The fusion display unit is used to fuse the patient's position information with the three-dimensional model in the virtual environment in real time and display it in the VR device.

[0131] In some embodiments, the fusion display unit includes:

[0132] A first display unit, used to present the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan through the VR device;

[0133] The second display unit is used to display the three-dimensional model constructed by the image, outline data and planning data in real time through the VR device.

[0134] In some embodiments, after displaying in the VR device, the apparatus further comprises:

[0135] The adjustment module is used to perform positioning adjustment according to the overlap to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

[0136] The working principle and effect of the above technical solution are the same as those in the embodiment of the method of the present invention, and will not be described in detail here.

[0137] An embodiment of the present invention provides a treatment system, the system comprising:

[0138] Radiotherapy devices, used to deliver radiation therapy to patients;

[0139] The positioning verification device described in the embodiment of the present invention is used to verify the position of a patient.

[0140] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods described in the embodiments of the present invention or the functions of the device described in the embodiments of the present invention are implemented.

[0141] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed, the steps of the method in the embodiment of the present invention are implemented. The specific implementation method is consistent with the implementation method and the technical effect achieved recorded in the above method embodiment, and some contents will not be repeated here.

[0142] In the present invention, a readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0143] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or for use in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The program code for performing the operation of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as C language or similar programming languages. The program code may be executed entirely on a user computing device, partially on an associated device, as an independent software package, partially on a user computing device, partially on a remote computing device, or entirely on a remote computing device or server. Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0144] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, substitute and modify the above embodiments within the scope of the invention without departing from the principles and purpose of the present invention. All such changes should fall within the scope of protection of the claims of the present invention.

Claims

1. The VR-based SGRT positioning verification method is characterized by: The method comprises: Acquiring treatment plan-related data of the patient; the treatment plan-related data includes the patient's surface contour data, delineation data, dose data, and treatment isocenter position; Using the treatment plan-related data to perform three-dimensional visualization reconstruction to construct a three-dimensional model of the patient; Acquire the patient's real-time body surface data through SGRT; Performing rigid body registration on the real-time body surface data and the surface contour data in the three-dimensional model; Performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result; Using the reconstructed three-dimensional deformation field, the image and treatment plan-related data are deformed into the constructed three-dimensional positioning coordinate system and displayed through VR to verify the SGRT positioning.

2. The VR-based SGRT positioning verification method according to claim 1, characterized in that: Before the deformation registration, the method further includes: According to the rigid body registration result, the first displacement is obtained; The bed moving operation is performed through the first displacement.

3. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The method further comprises: According to the deformation registration result, a second displacement is obtained; The bed moving operation is performed through the second displacement.

4. The VR-based SGRT positioning verification method according to claim 1, characterized in that: Rigid body registration is performed using a point cloud based rigid body registration method.

5. The VR-based SGRT positioning verification method according to claim 4, characterized in that: The rigid body registration includes calculating a loss function, which is: Among them, Loss is the loss function, p i is the patient’s real-time body surface data point, q i Skin contour point cloud of patient data and p i The corresponding point, n i The target point q i The normal vector at , R is the rotation matrix, t is the translation vector; N is the number of samples of data points.

6. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The RegFormer network is used for deformation registration, and the overall deformation of the internal organs is obtained through surface contour deformation to reconstruct the three-dimensional deformation field.

7. The VR-based SGRT positioning verification method according to claim 1, characterized in that: The displaying by VR to verify the SGRT positioning includes: Import the patient's three-dimensional model into the VR device to build a virtual environment; The patient's position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device.

8. The VR-based SGRT positioning verification method according to claim 7, characterized in that: The real-time fusion of the patient position information with the three-dimensional model in the virtual environment and displaying it in the VR device includes: The VR device is used to present the patient's current position, the target position in the radiotherapy plan, and the degree of overlap between the patient's current position and the target position in the radiotherapy plan; The VR device displays images, outline data, and a three-dimensional model constructed using planning data in real time.

9. The VR-based SGRT positioning verification method according to claim 8, characterized in that: After the patient position information is integrated with the three-dimensional model in the virtual environment in real time and displayed in the VR device, the method further includes: According to the overlap, the positioning adjustment is performed to ensure that the patient's current position is consistent with the target position in the radiotherapy plan, that is, the treatment isocenter position.

10. The VR-based SGRT positioning verification device is characterized by: The device comprises: A treatment plan acquisition module is used to acquire the patient's treatment plan related data; the treatment plan related data includes the patient's surface contour data, delineation data, dose data, and treatment center position; A model building module, used to perform three-dimensional visualization reconstruction using the treatment plan-related data to build a three-dimensional model of the patient; A real-time data acquisition module is used to acquire the patient's real-time body surface data through SGRT; A rigid body registration module, used for rigid body registration of the real-time body surface data with the surface contour data in the three-dimensional model; A deformation registration module, used for performing deformation registration on the real-time body surface data and the surface contour data in the three-dimensional model, and reconstructing a three-dimensional deformation field according to the deformation registration result; The verification module is used to use the reconstructed three-dimensional deformation field to deform the image and treatment plan related data into the constructed three-dimensional positioning coordinate system, and display it through VR to verify the SGRT positioning.

11. A treatment system, characterized in that: The system comprises: Radiotherapy devices, used to deliver radiation therapy to patients; The positioning verification device described in claim 10 is used to verify the position of a patient.

12. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 9 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions, and when a computer reads the computer instructions, the computer executes the steps of the method according to any one of claims 1 to 9.

Images