High-precision AR (Augmented Reality) guiding system and method for radiotherapy positioning in network-free environment

Abstract

The invention discloses a high-precision AR guiding system and method for radiotherapy positioning in a network-free environment, and belongs to the technical field of computer vision, and the method comprises the steps: pre-loading three-dimensional point cloud data of a target region in a radiotherapy plan in an offline database; the three-dimensional reconstruction and registration module is used for acquiring imaging data of a patient and performing three-dimensional registration on the imaging data and a target area in a radiotherapy plan; the respiration synchronization module monitors respiration monitoring data of a patient in real time, and determines an optimal time window of radiotherapy by adopting a respiration synchronization algorithm according to the respiration monitoring data; in the radiotherapy process, the user interface dynamically adjusts the registration parameters according to the real-time breathing state of the patient and displays the registration parameters in real time. According to the method, the problem of data processing in a network-free environment is solved, the positioning precision and adaptability in the radiotherapy process are remarkably improved, and a powerful technical support is provided for realizing personalized and efficient cancer treatment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a high-precision AR guidance system and method for radiotherapy positioning in a network-free environment. Background Art

[0002] Radiation therapy (RT) is an integral part of cancer treatment, using high-energy rays to damage the DNA of tumor cells, thereby inhibiting or killing cancer cells. To ensure the effectiveness and safety of treatment, accurate radiotherapy positioning is essential. Inaccurate positioning may lead to unnecessary radiation exposure to healthy tissues, increase the risk of side effects, and reduce the effective dose to the tumor area, affecting the treatment effect.

[0003] Traditional radiotherapy positioning relies on markers, laser positioning systems, and the physician's experience. However, this method has certain limitations: the patient's position may change at different time points, and respiratory movement may also cause the position of internal organs to move, all of which will affect the accuracy of positioning. In addition, in the absence of a network connection, real-time data transmission and processing become particularly difficult, which places higher demands on radiotherapy in a network-free environment.

[0004] Existing radiotherapy guidance systems are mostly based on online networking modes, using imaging equipment such as CT and MRI to obtain the patient's anatomical structure information, and combined with computer-aided design (CAD) software for three-dimensional reconstruction and registration. However, such systems usually require a stable network connection to support the transmission of large amounts of data, cloud computing resources, and remote expert consultation. The applicability of traditional systems is limited in scenarios where network stability cannot be guaranteed in remote areas or emergency situations. In addition, existing systems are also insufficient in dealing with changes in the position of organs in the body caused by respiratory movement. Although some advanced radiotherapy equipment is equipped with respiratory gating technology to reduce errors caused by breathing, they often rely on additional hardware facilities, which increases cost and complexity. Moreover, most systems lack an integrated solution that can seamlessly combine imaging data, real-time monitoring data, and user interaction to provide an intuitive and easy-to-operate user interface.

[0005] Therefore, how to provide a high-precision AR guidance system and method for radiotherapy positioning in a network-free environment is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0006] In view of this, the present invention provides a high-precision AR guidance system and method for radiotherapy positioning in a network-free environment, which overcomes the problems of network dependence and insufficient respiratory motion compensation in the prior art.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A high-precision AR guidance system for radiotherapy positioning in a network-free environment, comprising:

[0009] Offline database, 3D reconstruction and registration module, respiratory synchronization module, user interface;

[0010] Wherein, the offline database is pre-loaded with three-dimensional point cloud data of the target area in the radiotherapy plan;

[0011] The three-dimensional reconstruction and registration module acquires the imaging data of the patient and performs three-dimensional registration of the imaging data with the target area in the radiotherapy plan;

[0012] The respiratory synchronization module monitors the patient's respiratory monitoring data in real time, and uses a respiratory synchronization algorithm to determine the optimal time window for radiotherapy based on the respiratory monitoring data;

[0013] During radiotherapy, the user interface dynamically adjusts the registration parameters according to the patient's real-time respiratory status and displays them in real time.

[0014] Furthermore, the three-dimensional reconstruction and registration module acquires the imaging data of the patient and performs three-dimensional registration of the imaging data with the target area in the radiotherapy plan, including:

[0015] Data acquisition unit: collects the patient's imaging data in real time;

[0016] A data processing unit, performing statistical filtering processing, point cloud region growing processing and voxel filtering processing on the imaging data;

[0017] A feature extraction unit reconstructs the imaging data processed by the data processing unit into three-dimensional point cloud data through a pre-trained neural network model and performs feature extraction to obtain a first feature data set; at the same time, obtains the three-dimensional point cloud data of the target area in the radiotherapy plan from an offline database and performs feature extraction to obtain a second feature data set;

[0018] A feature map construction unit, constructing a first feature map based on the first feature data set, and constructing a second feature map based on the second feature data set;

[0019] A registration unit is configured to register the first feature map with the second feature map.

[0020] Furthermore, the structure of the neural network model includes:

[0021] Point cloud segmentation module: divides the point cloud into cubic voxels and groups them by voxels;

[0022] Downsampling module: randomly selects a fixed number of points within each voxel;

[0023] Voxel encoding layer: learns and combines the features of all points in the voxel to obtain local features;

[0024] Fully connected layer: nonlinearly maps local features to generate new feature representations;

[0025] Weighted extraction layer: Use softmax to calculate the M-dimensional local descriptor of each voxel weight normalization and its corresponding M feature coordinates.

[0026] Furthermore, the mathematical expression of the neural network model for feature extraction is:

[0027] W=f PointRegNet (X 1 )

[0028] S=f PointRegNet (X 2 )

[0029] In the formula, f PointRegNet represents the feature extraction of PointRegNet network, X 1 , X 2 They are the first feature data set and the second feature data set respectively.

[0030] Furthermore, the feature map construction unit includes:

[0031] Set all feature points in W to be connected to form a feature edge set R;

[0032] Set all feature points in S to connect with neighboring feature points to form a feature edge set U;

[0033] Obtain the first feature graph (W, R) based on the feature edge set R;

[0034] Based on the feature edge set U, a second feature graph (S, U) is obtained.

[0035] Furthermore, the registration unit includes:

[0036]

[0037]

[0038] In the formula, P(x|X 1 ,X 2 ) is based on the first feature data set X 1 , the second feature data set X 2 The probability distribution of the GMM model, ω m For the GMM model, m feature weights, μ mis the m feature mean vector of the GMM model, Σ m is the covariance matrix of the GMM model with m features, δ is the multivariate Gaussian distribution, P is the conditional probability density function, Y is the transformation condition, is the optimal registration transformation obtained by maximizing the likelihood estimate.

[0039] Furthermore, the respiratory synchronization algorithm in the respiratory synchronization module is:

[0040]

[0041] Where Q(t) is the radiotherapy time window, t is time, T b For the breathing cycle.

[0042] On the other hand, the present invention also provides a high-precision AR guidance method for radiotherapy positioning in a network-free environment, based on the above-mentioned high-precision AR guidance system for radiotherapy positioning in a network-free environment, comprising:

[0043] S100: pre-loading the three-dimensional point cloud data of the target area in the radiotherapy plan;

[0044] S200: Acquire imaging data of the patient in real time, and perform three-dimensional registration of the imaging data with a target area in a radiotherapy plan;

[0045] S300: monitoring the patient's respiratory monitoring data in real time, and determining the optimal time window for radiotherapy according to the respiratory monitoring data;

[0046] S400: Dynamically adjust the registration parameters according to the patient's real-time respiratory status.

[0047] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a high-precision AR guidance system and method for radiotherapy positioning in a network-free environment, which not only solves the problem of data processing in a network-free environment, but also significantly improves the positioning accuracy and adaptability during radiotherapy, providing strong technical support for the realization of personalized and efficient cancer treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0049] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0050] Figure 2 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1 The embodiment of the present invention discloses a high-precision AR guidance system for radiotherapy positioning in a network-free environment, including:

[0053] Offline database, 3D reconstruction and registration module, respiratory synchronization module, user interface;

[0054] Wherein, the offline database is pre-loaded with the three-dimensional point cloud data of the target area in the radiotherapy plan;

[0055] The 3D reconstruction and registration module acquires the patient's imaging data and performs 3D registration between the imaging data and the target area in the radiotherapy plan;

[0056] The respiratory synchronization module monitors the patient's respiratory monitoring data in real time, and uses the respiratory synchronization algorithm to determine the optimal time window for radiotherapy based on the respiratory monitoring data;

[0057] During radiotherapy, the user interface dynamically adjusts the registration parameters according to the patient's real-time respiratory status and displays them in real time.

[0058] Specifically, the offline database is pre-loaded with 3D point cloud data of the target area in the radiotherapy plan, ensuring quick access to critical information even when the network is unavailable.

[0059] Specifically, the 3D reconstruction and registration module can obtain the patient's latest imaging data and perform high-precision 3D registration with the pre-stored target area to ensure the accuracy of the treatment site.

[0060] Specifically, the respiratory synchronization module monitors the patient's respiratory status in real time and uses advanced respiratory synchronization algorithms to determine the optimal radiotherapy time window, effectively addressing the uncertainty caused by respiratory movement.

[0061] Specifically, the user interface dynamically adjusts the registration parameters according to the patient's real-time respiratory status and displays them in real time, allowing medical staff to maintain full control of the patient's condition throughout the radiotherapy process, improving the safety and efficiency of treatment.

[0062] In a specific embodiment, the 3D reconstruction and registration module acquires the imaging data of the patient and performs 3D registration between the imaging data and the target area in the radiotherapy plan, including:

[0063] Data acquisition unit: collects the patient's imaging data in real time;

[0064] A data processing unit performs statistical filtering, point cloud region growing and voxel filtering on the imaging data;

[0065] A feature extraction unit reconstructs the imaging data processed by the data processing unit into three-dimensional point cloud data through a pre-trained neural network model and performs feature extraction to obtain a first feature data set; at the same time, obtains the three-dimensional point cloud data of the target area in the radiotherapy plan from an offline database and performs feature extraction to obtain a second feature data set;

[0066] A feature map construction unit, constructing a first feature map based on the first feature data set, and constructing a second feature map based on the second feature data set;

[0067] The registration unit registers the first feature map with the second feature map.

[0068] In a specific embodiment, the structure of the neural network model includes:

[0069] Point cloud segmentation module: divides the point cloud into cubic voxels and groups them by voxels;

[0070] Downsampling module: randomly selects a fixed number of points within each voxel;

[0071] Voxel encoding layer: learns and combines the features of all points in the voxel to obtain local features;

[0072] Fully connected layer: nonlinearly maps local features to generate new feature representations;

[0073] Weighted extraction layer: Use softmax to calculate the M-dimensional local descriptor of each voxel weight normalization and its corresponding M feature coordinates.

[0074] In a specific embodiment, the mathematical expression for feature extraction by the neural network model is:

[0075] W=f PointRegNet (X 1 )

[0076] S=f PointRegNet (X 2 )

[0077] In the formula, f PointRegNet represents the feature extraction of PointRegNet network, X 1, X 2 They are the first feature data set and the second feature data set respectively.

[0078] In a specific embodiment, the feature map construction unit includes:

[0079] Set all feature points in W to be connected to form a feature edge set R;

[0080] Set all feature points in S to connect with neighboring feature points to form a feature edge set U;

[0081] Obtain the first feature graph (W, R) based on the feature edge set R;

[0082] Based on the feature edge set U, a second feature graph (S, U) is obtained.

[0083] In a specific embodiment, the registration unit includes:

[0084]

[0085]

[0086] In the formula, P(x|X 1 ,X 2 ) is based on the first feature data set X 1 , the second feature data set X 2 The probability distribution of the GMM model, ω m For the GMM model, m feature weights, μ m is the m feature mean vector of the GMM model, Σ m is the covariance matrix of the GMM model with m features, δ is the multivariate Gaussian distribution, P is the conditional probability density function, Y is the transformation condition, is the optimal registration transformation obtained by maximizing the likelihood estimate.

[0087] Specifically, during the registration process, P(X 1 |X 2 ,Y) is considered as a probability likelihood function, which measures the probability of applying the transformation Y to X. 1 When on, X 1 With X 2 Similarity Likelihood. Similarity in this embodiment can be defined in a variety of ways, but in the case of using GMM, it is measured by GMM model parameters.

[0088] Specifically, the maximum likelihood estimation (MLE) is specifically: Find the best transformation The process is actually performing maximum likelihood estimation, that is, finding a value that makes P(X 1 |X 2 ,Y) maximizes Y. This embodiment makes X1 The distribution after transformation is as close as possible to X 2 distribution.

[0089] The specific function of the GMM model is as follows: The GMM model is used as a method to describe the distribution of point clouds in this embodiment. It allows complex point cloud distribution to be simplified into a combination of multiple Gaussian distributions, so that the similarity between two point clouds can be more conveniently calculated. For each point x, P(x|X 1 ,X 2 ) gives the probability that the point belongs to different Gaussian distributions, which helps to construct the probability distribution of the entire point cloud and ultimately helps determine the best transformation

[0090] Specifically, in the entire registration process, the final optimization goal of this embodiment is to maximize P(X 1 |X 2 ,Y), that is, this embodiment is to find an optimal transformation Make X 1 The transformed distribution is most likely to produce X 2 The data points observed in .

[0091] Specifically, the present invention describes the point cloud distribution through the GMM model, and uses the method of finding the optimal transformation to implement the point cloud registration.

[0092] In a specific embodiment, the breathing synchronization algorithm in the breathing synchronization module is:

[0093]

[0094] Where Q(t) is the radiotherapy time window, t is time, T b For the breathing cycle.

[0095] Specifically, within a determined radiotherapy time window, the radiotherapy equipment is started to irradiate the patient, and the edge computing device is used to process the data during the radiotherapy process in real time, and feedback and adjustments are performed as needed. The present invention combines a respiratory synchronization algorithm with a three-dimensional registration algorithm to improve the accuracy and safety of radiotherapy.

[0096] On the other hand, see Figure 2 The embodiment of the present invention discloses a high-precision AR guidance method for radiotherapy positioning in a network-free environment, based on the above-mentioned high-precision AR guidance system for radiotherapy positioning in a network-free environment, including:

[0097] S100: pre-loading the three-dimensional point cloud data of the target area in the radiotherapy plan;

[0098] S200: Acquire the patient's imaging data in real time and perform three-dimensional registration between the imaging data and the target area in the radiotherapy plan;

[0099] S300: monitoring the patient's respiratory monitoring data in real time, and determining the optimal time window for radiotherapy according to the respiratory monitoring data;

[0100] S400: Dynamically adjust the registration parameters according to the patient's real-time respiratory status.

[0101] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0102] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-precision AR guidance system for radiotherapy positioning in a network-free environment, characterized in that: include: Offline database, 3D reconstruction and registration module, respiratory synchronization module, user interface; Wherein, the offline database is pre-loaded with three-dimensional point cloud data of the target area in the radiotherapy plan; The three-dimensional reconstruction and registration module acquires the imaging data of the patient and performs three-dimensional registration of the imaging data with the target area in the radiotherapy plan; The respiratory synchronization module monitors the patient's respiratory monitoring data in real time, and uses a respiratory synchronization algorithm to determine the optimal time window for radiotherapy based on the respiratory monitoring data; During radiotherapy, the user interface dynamically adjusts the registration parameters according to the patient's real-time respiratory status and displays them in real time.

2. According to claim 1, a high-precision AR guidance system for radiotherapy positioning in a network-free environment is characterized in that: The three-dimensional reconstruction and registration module acquires the imaging data of the patient and performs three-dimensional registration of the imaging data with the target area in the radiotherapy plan, including: Data acquisition unit: collects the patient's imaging data in real time; A data processing unit, performing statistical filtering processing, point cloud region growing processing and voxel filtering processing on the imaging data; A feature extraction unit reconstructs the imaging data processed by the data processing unit into three-dimensional point cloud data through a pre-trained neural network model and performs feature extraction to obtain a first feature data set; at the same time, obtains the three-dimensional point cloud data of the target area in the radiotherapy plan from an offline database and performs feature extraction to obtain a second feature data set; A feature map construction unit, constructing a first feature map based on the first feature data set, and constructing a second feature map based on the second feature data set; A registration unit is configured to register the first feature map with the second feature map.

3. According to claim 2, a high-precision AR guidance system for radiotherapy positioning in a network-free environment is characterized in that: The structure of the neural network model includes: Point cloud segmentation module: divides the point cloud into cubic voxels and groups them by voxels; Downsampling module: randomly selects a fixed number of points within each voxel; Voxel encoding layer: learns and combines the features of all points in the voxel to obtain local features; Fully connected layer: nonlinearly maps local features to generate new feature representations; Weighted extraction layer: Use softmax to calculate the M-dimensional local descriptor of each voxel weight normalization and its corresponding M feature coordinates.

4. According to claim 3, a high-precision AR guidance system for radiotherapy positioning in a network-free environment is characterized in that: The mathematical expression of the neural network model for feature extraction is: W=f PointRegNet (X1) S=f PointRegNet (X2) In the formula, f PointRegNet Represents the feature extraction of PointRegNet network, X1 and X2 are the first feature data set and the second feature data set respectively.

5. According to claim 4, a high-precision AR guidance system for radiotherapy positioning in a network-free environment is characterized in that: The feature map construction unit comprises: Set all feature points in W to be connected to form a feature edge set R; Set all feature points in S to connect with neighboring feature points to form a feature edge set U; Obtain the first feature graph (W, R) based on the feature edge set R; Based on the feature edge set U, a second feature graph (S, U) is obtained.

6. A high-precision AR guidance system for radiotherapy positioning in a network-free environment according to claim 5, characterized in that: The registration unit comprises: Where P(x|X1,X2) is the probability distribution of the GMM model based on the first feature data set X1 and the second feature data set X2, ω m For the GMM model, m feature weights, μ m is the m feature mean vector of the GMM model, Σ m is the covariance matrix of the GMM model with m features, δ is the multivariate Gaussian distribution, P is the conditional probability density function, Y is the transformation condition, is the optimal registration transformation obtained by maximizing the likelihood estimate.

7. The high-precision AR guidance system for radiotherapy positioning in a network-free environment according to claim 1, characterized in that: The respiratory synchronization algorithm in the respiratory synchronization module is: Where Q(t) is the radiotherapy time window, t is time, T b For the breathing cycle.

8. A high-precision AR guidance method for radiotherapy positioning in a network-free environment, based on the high-precision AR guidance system for radiotherapy positioning in a network-free environment as described in any one of claims 1 to 7, characterized in that: include: S100: pre-loading the three-dimensional point cloud data of the target area in the radiotherapy plan; S200: Acquire imaging data of the patient in real time, and perform three-dimensional registration of the imaging data with a target area in a radiotherapy plan; S300: monitoring the patient's respiratory monitoring data in real time, and determining the optimal time window for radiotherapy according to the respiratory monitoring data; S400: Dynamically adjust the registration parameters according to the patient's real-time respiratory status.

Images