Online adaptive radiotherapy intra-fraction anatomical structure generation method and system based on three-dimensional point cloud

Abstract

The present invention provides a method and system for generating intra-fraction anatomical structures for online adaptive radiotherapy based on three-dimensional point clouds. This method, which belongs to the technical field of radiotherapy equipment, acquires three-dimensional point cloud data of intra-fraction anatomical structures; processes the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain a three-dimensional point cloud of the anatomical structure at the end of fractionated treatment. The present invention improves the speed of intra-fraction anatomical structure prediction; obtains anatomical structure deformation vectors based on new delineation to guide structure generation for the next fraction, thereby improving the accuracy of intra-fraction anatomical structure prediction; and assesses the magnitude of anatomical structure changes within the patient's current fraction based on the predicted anatomical structure results. This method, combined with clinical decision-making, guides the design of online adaptive radiotherapy plans, significantly reducing intra-fraction errors caused by organ motion within online adaptive radiotherapy fractions, ensuring target coverage while reducing the radiation dose to endangered organs.

Description

Technical Field

[0001] The present invention relates to the technical field of radiotherapy equipment, and in particular to a method and system for generating intra-fraction anatomical structures of online adaptive radiation therapy (OART) based on three-dimensional point clouds, which is suitable for magnetic resonance-guided online adaptive radiation therapy. Background Art

[0002] The advent of magnetic resonance-guided linear accelerators (MR-linacs) has driven the development of MR-guided online adaptive radiotherapy (OART). OART can better protect organs at risk and reduce the risk of complications. Compared with conventional radiotherapy techniques, OART effectively reduces errors caused by inter-fraction organ motion by online adjustment of the treatment plan to accommodate anatomical changes in the tumor target and organs at risk between fractions. The full OART fractionation treatment process includes patient positioning, image acquisition, online contouring, plan update, plan evaluation, and beam delivery. This process typically takes 15 to 30 minutes, but may take longer depending on the treatment center. Internal organ motion during this time can easily cause the shape and position of the target and surrounding normal tissues and organs to change at varying rates and magnitudes, leading to intra-fraction errors. The fullness of digestive and urinary organs significantly affects not only the target location but also the range of target motion, resulting in significant changes in the dose received by the target and normal organs. A limitation of OART is that it assumes that the patient's internal organs remain stationary within the treatment fraction, ignoring internal organ motion (such as respiratory movement and bladder filling), and failing to account for errors caused by intra-fraction organ motion. Therefore, quickly generating the anatomical structure at the end of OART fractionation treatment at the beginning of OART fractionation treatment is of great significance for optimizing OART plan and improving the accuracy of OART radiotherapy. Summary of the Invention

[0003] To address the issue of inaccurate dose distribution caused by the failure to account for intra-fractional anatomical variations in existing MRI-guided online adaptive radiotherapy (AART) procedures, this paper aims to provide a deep learning method based on three-dimensional point clouds for generating intra-fractional anatomical structures in AART. This method effectively reduces errors caused by intra-fraction organ motion, ensuring accurate coverage of the treatment target while reducing the dose to organs at risk.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for generating intra-fraction anatomical structures of radiotherapy in an online adaptive manner based on a three-dimensional point cloud, comprising:

[0006] Acquire 3D point cloud data of intra-fraction anatomical structures;

[0007] The acquired 3D point cloud data is processed using a pre-trained intra-fractional anatomical structure generation model to produce a 3D point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fractional anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the feature extraction results of the latent space representations obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a concatenated feature; and a decoder unit for performing step-by-step upsampling and synchronous dimensionality reduction of the concatenated features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud.

[0008] As a further limitation of the first aspect of the present invention, obtaining three-dimensional point cloud data of intra-fractionated anatomical structures includes: outlining the anatomical structure using high-resolution 3D MRI image data of the tumor region, extracting the contour of the region of interest, and smoothing the data using a mean filter, wherein the mean filter algorithm converts the volume data into a grid structure composed of vertices and edges, and a triangle in each grid is defined by three adjacent points; and using z-axis stratification and random uniform downsampling to downsample the number of point clouds of the anatomical structure to the same number of point clouds as used when training the intra-fractionated anatomical structure generation model, thereby obtaining three-dimensional point cloud data of the intra-fractionated anatomical structure.

[0009] As a further limitation of the first aspect of the present invention, the encoder unit includes four consecutive SA modules that extract point cloud features layer by layer; the first-layer SA module performs preliminary abstraction on the input three-dimensional point cloud data to generate a first-layer feature representation; the second-layer SA module further reduces the number of vertices, increases the dimension of the features, and captures mid-scale spatial information; the third-layer SA module and the fourth-layer SA module continue to gradually extract higher-level and more abstract spatial features, the number of vertices continues to decrease, and the feature dimension continues to increase, and finally a compact latent space representation is obtained.

[0010] As a further limitation of the first aspect of the present invention, in the decoder unit, each upsampling layer is combined with a multi-layer perceptron through 1×1 convolution to gradually expand the scale of the feature space; during the upsampling process, the channel dimension is reduced simultaneously to keep the shape of the output data consistent with the input data; the jump connection directly transfers the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features and enhance the prediction accuracy.

[0011] As a further limitation of the first aspect of the present invention, the intra-fractional anatomical structure generation model uses Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and at the same time introduces an entropy regularization term to improve the stability and efficiency of the calculation.

[0012] As a further limitation of the first aspect of the present invention, a robust point matching algorithm based on thin plate splines is used to find the point correspondence between the three-dimensional point cloud generated by the current fractional prediction and the real point cloud, and to calculate the deformation vector; the deformation vector is used as the input of the intra-fractional anatomical structure generation model to guide the prediction generation of the corresponding anatomical structure in the next fraction.

[0013] In a second aspect, the present invention provides an online adaptive radiotherapy intra-fractionation anatomical structure generation system based on three-dimensional point clouds, comprising:

[0014] An acquisition module, used to obtain three-dimensional point cloud data of intra-fraction anatomical structures;

[0015] The processing module is used to process the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain a three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fraction anatomical structure generation model includes: an encoder unit, which is used to extract point cloud features layer by layer to form a latent space representation; a feature splicing unit, which is used to splice the results of feature extraction of the latent space representation obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain spliced ​​features; a decoder unit, which is used to perform step-by-step upsampling and synchronous dimensionality reduction of the spliced ​​features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud.

[0016] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the online adaptive radiotherapy intra-fractionation anatomical structure generation method based on three-dimensional point cloud is implemented as described in the first aspect.

[0017] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the online adaptive radiotherapy intra-fractionation anatomical structure generation method based on three-dimensional point cloud as described in the first aspect.

[0018] In a fifth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the method for generating anatomical structures within fractionated online adaptive radiotherapy based on three-dimensional point clouds as described in the first aspect.

[0019] The beneficial effects of the present invention are as follows: it avoids the introduction of complex MRI image information by traditional methods and improves the speed of intra-fraction anatomical structure prediction; it can obtain the anatomical structure deformation vector based on the new outline to guide the structure generation of the next fraction, which can improve the accuracy of intra-fraction anatomical structure prediction; it can predict the anatomical structure of the patient at the end of the treatment at the beginning of the patient's fractionated treatment, and the size of the patient's anatomical structure change within the current fraction can be evaluated based on the prediction results, and combined with clinical decision-making to guide the design of online adaptive radiotherapy plans, greatly reducing the intra-fraction error caused by organ movement within the online adaptive radiotherapy fraction, and can ensure the target area coverage while reducing the exposure dose to the endangered organs.

[0020] Additional advantages of the present invention will be more clearly given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a schematic diagram of a 3D point cloud of the rectal structure of a prostate cancer patient according to an embodiment of the present invention. The "x," "y," and "z" axes correspond to the left-right (LR), anteroposterior (AP), and craniofacial (SI) directions of the patient, respectively.

[0023] Figure 2 This is a flowchart for the application of the method for generating intra-fractionated anatomical structures based on 3D point clouds, as described in an embodiment of the present invention. The orange boxes represent pre-treatment procedures for patients; the blue boxes represent the conventional online adaptive radiotherapy patient fractionation treatment process; and the yellow boxes represent the steps for integrating the present invention into the conventional process.

[0024] Figure 3 This is a diagram of the model architecture for generating intra-fractional anatomical structures according to an embodiment of the present invention.

[0025] Figure 4 Schematic diagram of the SA Module of the generation model described in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0027] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0028] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0029] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0030] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise contradictory.

[0031] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0032] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0033] The present invention provides a method for generating intra-fraction anatomical structures in radiotherapy that is safe, fast, and accurate. This method is applicable to all patients receiving magnetic resonance-guided online adaptive radiotherapy. By quickly and accurately generating the anatomical structures at the end of fractionated treatment at the beginning of fractionated treatment, it is possible to judge in advance whether the patient will undergo significant anatomical shape changes during the fractionated treatment. This method provides a reference for radiotherapy physicians, helping them decide whether they need to adjust the online planning strategy to meet clinical dosimetry requirements and further improve the accuracy of online adaptive radiotherapy. With breakthroughs in methods and upgrades in clinical needs, the limitation of magnetic resonance-guided online adaptive radiotherapy that cannot solve intra-fraction errors will be further expanded. Therefore, the method proposed in the present invention can clinically solve the limitations of magnetic resonance-guided online adaptive radiotherapy and has very important clinical significance and application value.

[0034] Example 1

[0035] In this embodiment 1, an online adaptive radiotherapy intra-fraction anatomical structure generation system based on a three-dimensional point cloud is first provided. The system includes: an acquisition module for acquiring three-dimensional point cloud data of the intra-fraction anatomical structure; a processing module for processing the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain a three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the feature extraction results of the latent space representations obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a concatenated feature; and a decoder unit for performing step-by-step upsampling and synchronous dimensionality reduction of the concatenated features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud. Among them, the intra-fractional anatomical structure generation model uses Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and at the same time introduces the entropy regularization term to improve the stability and efficiency of the calculation.

[0036] In this embodiment, the above-mentioned system is used to implement an online adaptive radiotherapy intra-fraction anatomical structure generation method based on a three-dimensional point cloud, including: using an acquisition module to acquire three-dimensional point cloud data of the intra-fraction anatomical structure; using a processing module to process the acquired three-dimensional point cloud data based on a pre-trained intra-fraction anatomical structure generation model to obtain a three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment.

[0037] Acquiring three-dimensional point cloud data of intra-fractionated anatomical structures involves outlining the anatomical structure using high-resolution 3D MRI image data of the tumor region, extracting the contours of the region of interest, and smoothing the data using a mean filter. The mean filter algorithm converts the volumetric data into a grid structure composed of vertices and edges, with each triangle in the grid defined by three adjacent points. The z-axis layering and random uniform downsampling method are then used to downsample the number of point clouds of the anatomical structure to the same number used to train the intra-fractionated anatomical structure generation model, thereby obtaining three-dimensional point cloud data of the intra-fractionated anatomical structure.

[0038] Specifically, data acquisition and processing: At the beginning of a patient's fractionated treatment, a magnetic resonance imaging (MRI) device is used to collect high-resolution 3D MRI image data of the patient's tumor area. Based on the new MRI images, the radiation oncologist prioritizes outlining of key anatomical structures (such as the tumor target volume and adjacent vital tissues and organs). Once the physician completes the outlining, the MRI image and the corresponding anatomical structure RT file are imported into the medical imaging processing platform for subsequent processing. Extracting a 3D point cloud: Within the medical imaging processing platform, the outline of the region of interest is extracted from the patient's RT structure file, converted into a binary mask format, and stored as an MHD file. The input MHD file is smoothed using a mean filter to optimize the structural surface. The mean filtering algorithm converts the volumetric data into a mesh structure composed of vertices and edges, with each triangle in the mesh defined by three adjacent points. The number of vertices in the anatomical structure mesh varies from fraction to fraction. To accelerate the anatomical structure generation process, the present invention uses z-axis slicing and random uniform downsampling to downsample the anatomical structure point cloud to the same number as during model training. Through the above processing, the anatomical structure within the fraction can be represented as a 3D point cloud.

[0039] The encoder unit includes four consecutive SA modules that extract point cloud features layer by layer; the first-layer SA module performs preliminary abstraction on the input three-dimensional point cloud data to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices, increases the dimension of the features, and captures the mid-scale spatial information; the third-layer SA module and the fourth-layer SA module continue to gradually extract higher-level and more abstract spatial features, with the number of vertices continuously decreasing and the feature dimension continuously increasing, ultimately obtaining a compact latent space representation.

[0040] Specifically, the encoder uses four consecutive PointNetSetAbstraction (SA) modules to extract point cloud features layer by layer, gradually reducing the number of three-dimensional point clouds while increasing the feature dimension, and gradually compressing the intrinsic geometric features of the point cloud data into a compact potential representation. Specifically, the encoder uses the PointNetSetAbstraction technology in the PointNet++ structure to gradually enhance the ability to perceive the geometric features of point cloud data in a hierarchical manner, thereby capturing multi-scale spatial information. The first-layer SA module performs a preliminary abstraction on the input three-dimensional point cloud to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices while increasing the feature dimension to capture the mid-scale spatial information in more detail; the third and fourth-layer SA modules continue to gradually extract higher-level and more abstract spatial features, with the number of vertices decreasing and the feature dimension increasing, ultimately forming a compact latent space representation.

[0041] In feature splicing, two encoders are used to extract the three-dimensional point cloud shape of the anatomical structure and the deformation vector of the previous division, and then spliced ​​in the feature dimension to enhance the model's perception of the input data and improve the model's prediction accuracy.

[0042] In the decoder unit, each upsampling layer is combined with a multi-layer perceptron through 1×1 convolution to gradually expand the scale of the feature space; during the upsampling process, the channel dimension is simultaneously reduced to keep the shape of the output data consistent with the input data; and the jump connection directly transmits the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features and enhance the prediction accuracy.

[0043] Specifically, the decoder is specifically designed for progressive feature upsampling and simultaneous channel dimensionality reduction, ensuring that the output point cloud shape is consistent with the input. Each feature upsampling layer aims to expand the spatial feature dimension to finely reconstruct the point cloud representation; the accompanying channel dimensionality reduction layer reduces the number of channels in the feature space, ensuring accurate and efficient output shape. This multi-layered architecture enables the model to efficiently capture and reconstruct the spatial features of the input point cloud while maintaining the integrity and accuracy of the point cloud shape. Each upsampling layer combines 11 convolutions (11Conv) with a multi-layer perceptron (MLP) to gradually expand the feature space. During the upsampling process, the channel dimensionality is simultaneously reduced to maintain the shape of the output data consistent with the input data. Skip connections directly transfer the features output by the corresponding encoder layer to the corresponding decoder layer, helping the model to more effectively reconstruct spatial details and enhance prediction accuracy. The final decoder features are processed through a set of multi-layer perceptrons (MLPs) and 1x1 convolutional layers for final prediction, outputting the predicted anatomical point cloud. The model uses the Sinkhorn distance as the loss function to ensure spatial consistency between the predicted point cloud and the ground-truth point cloud. Sinkhorn distance is a regularized Earth Mover's Distance (EMD) that introduces an entropy regularization term to improve the stability and efficiency of the calculation.

[0044] A robust point matching algorithm based on thin plate splines is used to find point correspondences between the predicted 3D point cloud generated by the current fraction and the ground-truth point cloud, and a deformation vector is calculated. This deformation vector is used as input to the intra-fraction anatomical structure generation model to guide the generation of the corresponding anatomical structure for the next fraction. Specifically, the deformation vector of the anatomical structure of the current fraction is obtained to guide the generation of the anatomical structure for the next fraction. At the end of the patient's treatment for the current fraction, an MRI image is taken. The physician outlines the anatomical structures of interest in the current image, extracts a 3D point cloud, and uses this as the ground-truth. A robust point matching algorithm based on thin plate splines (TPS-RPM) is used to find point correspondences between the predicted 3D point cloud generated by the current fraction and the ground-truth point cloud, and a deformation vector is calculated. This deformation vector is used as input to the model to guide the generation of the corresponding anatomical structure for the next fraction. It is important to note that when a new patient is treated for the first time, the deformation vector is not available, so the deformation vector is zero for the first fraction.

[0045] Example 2

[0046] To address the issue of inaccurate dose distribution caused by the failure to account for intra-fractional anatomical variations in existing MRI-guided online adaptive radiotherapy, this second example proposes a deep learning method based on three-dimensional point clouds for the rapid generation of intra-fractional anatomical structures in online adaptive radiotherapy. This method effectively reduces errors caused by intra-fraction organ motion, ensuring accurate coverage of the treatment target while reducing the radiation dose to organs at risk.

[0047] like Figures 1 to 4 As shown, the deep learning method based on three-dimensional point cloud of this embodiment specifically includes the following steps:

[0048] 1. Data acquisition and processing: At the beginning of a patient's fractionated treatment, a magnetic resonance imaging (MRI) device is used to collect high-resolution 3D MRI image data of the patient's tumor area. Based on the new MRI images, the radiation oncologist prioritizes outlining key anatomical structures (such as the tumor target area and adjacent important tissues and organs). When the doctor completes the outlining, the MRI image and the corresponding anatomical structure RT file are imported into the medical image processing platform for subsequent processing.

[0049] 2. Extract three-dimensional point cloud: In the medical image processing platform, the outline of the region of interest is extracted from the patient's RT structure file, converted into a binary mask format, and stored as an MHD file. The input MHD file is smoothed using a mean filter to optimize the structural surface. The mean filtering algorithm converts the volume data into a grid structure consisting of vertices and edges, and the triangles in each grid are defined by three adjacent points. The number of vertices in the anatomical structure grid of each fraction is inconsistent. In order to speed up the anatomical structure generation process, the present invention uses z-axis stratification and random uniform downsampling to downsample the number of point clouds of the anatomical structure to the same number of point clouds during model training. Through the above processing, the anatomical structure within the fraction can be represented in the form of a three-dimensional point cloud.

[0050] 3. Intra-fractional anatomical structure generative model construction: This paper proposes a generative model architecture based on 3D point clouds (SA-UNet). This model can directly process 3D point cloud data and quickly and accurately predict anatomical structures through deep learning feature extraction methods. The model architecture is as follows:

[0051] Input and Output Definition: The model takes as input two 3D point cloud tensors of dimension [B, 3, X], where B represents the batch size, the channels represent the x, y, and z coordinates of the 3D point cloud, and X represents the number of downsampled points. The output is a tensor of the same size, dimension [B, X, 3], consistent with the input format, representing the predicted anatomical point cloud.

[0052] Encoder design: Hierarchical feature extraction. The encoder part uses four consecutive PointNetSetAbstraction (SA) modules to extract point cloud features layer by layer, gradually reducing the number of three-dimensional point clouds while increasing the feature dimension, and gradually compressing the intrinsic geometric features of the point cloud data into a compact potential representation. Specifically, the encoder uses the PointNetSetAbstraction technology in the PointNet++ structure to gradually enhance the ability to perceive the geometric features of point cloud data in a hierarchical manner, thereby capturing multi-scale spatial information. The first-layer SA module performs a preliminary abstraction on the input three-dimensional point cloud to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices while increasing the feature dimension to capture the mid-scale spatial information in more detail; the third and fourth-layer SA modules continue to gradually extract higher-level and more abstract spatial features, with the number of vertices decreasing and the feature dimension increasing, ultimately forming a compact latent space representation.

[0053] Feature splicing: After extracting the features of the 3D point cloud shape of the anatomical structure and the deformation vector of the previous step through two encoders, they are spliced ​​in the feature dimension to enhance the model's perception of the input data and improve the model's prediction accuracy.

[0054] Decoder Design: Feature Upsampling and Channel Dimensionality Reduction. The decoder is specifically designed for progressive feature upsampling and simultaneous channel dimensionality reduction, ensuring that the output point cloud shape is consistent with the input. Each feature upsampling layer aims to expand the spatial feature dimension to finely reconstruct the point cloud representation; the accompanying channel dimensionality reduction layer reduces the number of channels in the feature space, ensuring accurate and efficient output shape. This multi-layered architecture enables the model to efficiently capture and reconstruct the spatial features of the input point cloud while maintaining the integrity and accuracy of the point cloud shape. Each upsampling layer combines 1x1 convolutions with a multi-layer perceptron (MLP) to gradually expand the feature space. During the upsampling process, the channel dimensionality is simultaneously reduced to maintain the shape of the output data consistent with the input data. Skip connections directly pass the features output by the corresponding encoder layer to the corresponding decoder layer, helping the model to more effectively reconstruct spatial details and enhance prediction accuracy. The final decoder features are processed through a series of multi-layer perceptrons (MLPs) and 1x1 convolutional layers for final prediction, outputting the predicted anatomical point cloud. The model uses the Sinkhorn distance as a loss function to ensure spatial consistency between the predicted point cloud and the ground-truth point cloud. The Sinkhorn distance is a regularized Earth Mover's Distance (EMD) that incorporates an entropy regularization term to improve computational stability and efficiency.

[0055] 4. Intra-fraction anatomical structure generation: The 3D point cloud of the anatomical structure at the beginning of the fractionated treatment is input into the model, and the 3D point cloud of the anatomical structure at the end of the fractionated treatment is quickly predicted. After denormalization, the 3D point cloud is written into the MRI image taken at the beginning of the treatment and stored as an RT structure file.

[0056] 5. Clinical decision-making: Quantitatively analyze the spatial differences between the predicted anatomy within the fraction and the initial anatomy during treatment, calculate the difference between the initial anatomy and the predicted anatomy, and the radiation oncologist will determine whether the difference exceeds the clinically acceptable threshold based on the size of the difference.

[0057] 6. Dose optimization strategy to reduce the impact of errors caused by intra-fraction anatomical structure deformation: Clinicians will determine whether to limit or adjust the dose of areas with large spatial variations during plan optimization to avoid over- or under-dosing. When spatial variations are within an acceptable range, the design is based on conventional adaptive plans to improve the treatment accuracy of online adaptive radiotherapy.

[0058] 7. Obtain the deformation vector of the anatomical structure of the current fraction to guide the generation of the anatomical structure of the next fraction. At the end of the current fractional patient treatment, an MRI image is taken, and the doctor outlines the anatomical structure of relative concern in the current image, extracts the three-dimensional point cloud and uses it as the true value. The robust point matching algorithm based on thin plate splines (TPS-RPM) is used to find the point correspondence between the three-dimensional point cloud predicted and generated by the current fraction and the real point cloud, and calculate the deformation vector. The deformation vector is used as the input of the model to guide the prediction and generation of the corresponding anatomical structure of the next fraction. It is worth noting that when a new patient is treated for the first time, we cannot obtain the deformation vector, so the deformation vector is a zero vector during the first fractional treatment.

[0059] In this embodiment 2, the following describes a specific implementation of the method described in this embodiment 2 on a magnetic resonance accelerator for an online adaptive radiotherapy process for a prostate cancer patient.

[0060] Pre-treatment preparation: After the patient undergoes CT and MRI positioning, the radiation oncologist delineates the target volume and organs at risk. Once delineated, the physicist designs a reference plan, which is then reviewed by the physician and then dose verified.

[0061] Patient positioning and image scanning: The technician positions the patient according to the reference plan center and obtains MR scan images.

[0062] Image registration: Select the online adaptive mode to map the target area and organs at risk on the reference CT image to the currently scanned MR image through deformable registration.

[0063] Delineation of relatively important anatomical structures: For prostate cancer patients, the anatomical structures of relative clinical concern include the target area, bladder, rectum, and seminal vesicle. When delineating, doctors should prioritize delineating these four anatomical structures.

[0064] Extracting 3D point clouds: Once the anatomical structures of relative interest are outlined, an external program extracts the outline of the region of interest from the RT file, converts it into a binary mask format, and stores it as an MHD file. After smoothing the input MHD file using a mean filter, the structure vertices are extracted. Subsequently, the structure vertices are downsampled using z-axis slicing and random uniform downsampling to ensure correct input of the model. After obtaining the 3D point cloud of the current fraction, the TPS-RPM algorithm is used to perform deformation registration between the anatomical structure 3D point cloud of the current fraction and the real 3D point cloud of the previous fraction, find the correspondence between points, and solve the deformation vector.

[0065] Predict and generate anatomical structure: The current 3D point cloud and deformation vector are input into SA-UNet to predict and generate the 3D point cloud shape of the current fraction. After denormalization, the 3D point cloud is written into the MRI image taken at the beginning of treatment and stored as an RT structure file.

[0066] Clinical Decision: The radiation oncologist determines whether the deformation of the anatomical structure of interest significantly impacts the dose distribution. If so, dose limits or supplemental doses can be established by delineating auxiliary structures. If the deformation has little impact, the next step is taken. However, the judgment of deformation size is largely subjective, requiring the radiation oncologist to make an accurate judgment based on clinical guidelines.

[0067] Delineation of the remaining anatomical structures: After clinical decision-making, continue to delineate the remaining organs at risk, such as the femoral head, intestines, etc.

[0068] Online adaptive radiotherapy plan design and dose verification: After the plan is designed by a physicist, the physician determines whether the current dose distribution meets clinical requirements. If so, online dose verification is performed. If not, the physicist adjusts parameters and redesigns the plan accordingly.

[0069] Plan execution: Technicians perform treatments and, at the end of treatment, acquire new MR images, allowing physicians to outline anatomical structures of particular interest.

[0070] Calculating deformation vectors to guide next-fraction prediction: The physician delineates the anatomical structures of particular interest in the new MR image, extracts a 3D point cloud, and uses this as the ground truth. A robust point matching algorithm based on thin-plate splines (TPS-RPM) is used to find point correspondences between the 3D point cloud generated by the current fraction prediction and the ground truth point cloud, and calculates deformation vectors. During the patient's next treatment, this deformation vector is used as input to the model to guide the prediction of the corresponding anatomical structures for the next fraction.

[0071] Example 3

[0072] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the above-mentioned method for generating an anatomical structure for online adaptive radiotherapy fractionation based on a three-dimensional point cloud is implemented. The method includes:

[0073] Acquire 3D point cloud data of intra-fraction anatomical structures;

[0074] The acquired 3D point cloud data is processed using a pre-trained intra-fractional anatomical structure generation model to produce a 3D point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fractional anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the feature extraction results of the latent space representations obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a concatenated feature; and a decoder unit for performing step-by-step upsampling and synchronous dimensionality reduction of the concatenated features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud.

[0075] Example 4

[0076] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned method for generating intra-fractionated anatomical structures for online adaptive radiotherapy based on three-dimensional point clouds, the method comprising:

[0077] Acquire 3D point cloud data of intra-fraction anatomical structures;

[0078] The acquired 3D point cloud data is processed using a pre-trained intra-fractional anatomical structure generation model to produce a 3D point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fractional anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the feature extraction results of the latent space representations obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a concatenated feature; and a decoder unit for performing step-by-step upsampling and synchronous dimensionality reduction of the concatenated features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud.

[0079] Example 5

[0080] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the above-mentioned method for generating intra-fraction anatomical structures for online adaptive radiotherapy based on three-dimensional point clouds, the method comprising:

[0081] Acquire 3D point cloud data of intra-fraction anatomical structures;

[0082] The acquired 3D point cloud data is processed using a pre-trained intra-fractional anatomical structure generation model to produce a 3D point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fractional anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the feature extraction results of the latent space representations obtained by the two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a concatenated feature; and a decoder unit for performing step-by-step upsampling and synchronous dimensionality reduction of the concatenated features to ensure that the output point cloud shape is consistent with the input. The final features are finally predicted by a set of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud.

[0083] In summary, the innovations of the present invention include the following points:

[0084] (1) For the first time, a method for predicting anatomical structures within OART fractions based on three-dimensional point clouds is proposed. This method avoids the introduction of complex MRI image information in traditional methods and improves the speed of predicting anatomical structures within fractions. At the end of the current fraction, the anatomical structure deformation vector can be obtained based on the new outline to guide the structure generation of the next fraction, which can improve the accuracy of anatomical structure prediction within fractions. (2) The present invention can predict the anatomical structure of the patient at the end of the patient's treatment at the beginning of the patient's fractionated treatment. According to the prediction results, the size of the anatomical structure change within the patient's current fraction can be evaluated, and combined with clinical decision-making to guide the design of online adaptive radiotherapy plans, it greatly reduces the intra-fraction error caused by organ movement within the online adaptive radiotherapy fraction, and can ensure the target coverage while reducing the exposure dose to the endangered organs. (3) This method can be integrated into the existing magnetic resonance guided radiotherapy process, is applicable to multiple diseases and multiple parts, and has high clinical applicability and practical value.

[0085] In summary, the present invention proposes a method for generating intra-fraction anatomical structures based on three-dimensional point clouds, which not only overcomes the technical limitation of existing magnetic resonance-guided online adaptive radiotherapy that ignores intra-fraction anatomical changes, but also improves the accuracy of treatment, and has important clinical significance and application prospects.

[0086] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0090] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. A method for generating intra-fraction anatomical structures for online adaptive radiotherapy based on three-dimensional point clouds, characterized in that: include: Obtaining the three-dimensional point cloud data of the anatomical structure within the fraction and the deformation vector of the previous fraction; A pre-trained intra-fraction anatomical structure generation model is used to process the acquired three-dimensional point cloud data and the deformation vector of the previous fraction to obtain a three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment; wherein the intra-fraction anatomical structure generation model includes: an encoder unit, which is used to extract point cloud features layer by layer to form a latent space representation, and uses two encoders to encode the three-dimensional point cloud data of the intra-fraction anatomical structure and the deformation vector of the previous fraction respectively; a feature splicing unit, which is used to splice the encoding results obtained by the two encoders in the feature dimension to obtain spliced ​​features; a decoder unit, which is used to perform step-by-step upsampling and synchronous dimensionality reduction of the spliced ​​features to ensure that the output point cloud shape is consistent with the input, and the final features are finally predicted by a group of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud; wherein the deformation vector is calculated by the difference between the three-dimensional point cloud predicted and generated by the current fraction and the actual point cloud extracted at the end of the current fractionated treatment.

2. The method for generating intra-fraction anatomical structures of online adaptive radiotherapy based on three-dimensional point cloud according to claim 1, characterized in that: Acquiring three-dimensional point cloud data of intra-fractionated anatomical structures involves outlining the anatomical structure using high-resolution 3D MRI image data of the tumor region, extracting the contours of the region of interest, and smoothing the data using a mean filter. The mean filter algorithm converts the volumetric data into a grid structure composed of vertices and edges, with each triangle in the grid defined by three adjacent points. The z-axis layering and random uniform downsampling method are then used to downsample the number of point clouds of the anatomical structure to the same number used to train the intra-fractionated anatomical structure generation model, thereby obtaining three-dimensional point cloud data of the intra-fractionated anatomical structure.

3. The method for generating intra-fraction anatomical structures of online adaptive radiotherapy based on three-dimensional point cloud according to claim 1, characterized in that: The encoder unit includes four consecutive SA modules to extract point cloud features layer by layer; The first-layer SA module performs preliminary abstraction on the input 3D point cloud data to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices, increases the dimension of the features, and captures the mid-scale spatial information; the third-layer SA module and the fourth-layer SA module continue to gradually extract higher-level and more abstract spatial features, with the number of vertices continuously decreasing and the feature dimension continuously increasing, ultimately obtaining a compact latent space representation.

4. The method for generating intra-fraction anatomical structures of radiotherapy based on online adaptive 3D point cloud according to claim 1, characterized in that: In the decoder unit, each upsampling layer is combined with a multi-layer perceptron through 1×1 convolution to gradually expand the scale of the feature space; during the upsampling process, the channel dimension is reduced synchronously to keep the shape of the output data consistent with the input data; and the jump connection directly transmits the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features.

5. The method for generating intra-fraction anatomical structures of online adaptive radiotherapy based on three-dimensional point cloud according to claim 1, characterized in that: The intra-fractional anatomical structure generation model uses the Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and introduces an entropy regularization term.

6. The method for generating intra-fraction anatomical structures of online adaptive radiotherapy based on three-dimensional point cloud according to claim 1, characterized in that: A robust point matching algorithm based on thin plate splines is used to find the point correspondence between the current 3D point cloud generated by the stepwise prediction and the real point cloud, and the deformation vector is calculated; The deformation vector is used as the input of the intra-fraction anatomical structure generation model to guide the prediction and generation of the corresponding anatomical structure for the next fraction.

7. An online adaptive radiotherapy intra-fraction anatomical structure generation system based on three-dimensional point cloud, characterized by: include: An acquisition module, used to obtain the three-dimensional point cloud data of the anatomical structure within the fraction and the deformation vector of the previous fraction; A processing module is used to use a pre-trained intra-fraction anatomical structure generation model to process the acquired three-dimensional point cloud data and the deformation vector of the previous fraction to obtain a three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment; wherein the intra-fraction anatomical structure generation model includes: an encoder unit, used to extract point cloud features layer by layer to form a latent space representation, and use two encoders to encode the three-dimensional point cloud data of the intra-fraction anatomical structure and the deformation vector of the previous fraction respectively; a feature splicing unit, used to splice the encoding results obtained by the two encoders in the feature dimension to obtain spliced ​​features; a decoder unit, used to perform step-by-step upsampling and synchronous dimensionality reduction of the spliced ​​features to ensure that the output point cloud shape is consistent with the input, and the final features are finally predicted by a group of multi-layer perceptrons and 1*1 convolutional layers to obtain a predicted anatomical structure point cloud; wherein the deformation vector is calculated by the difference between the three-dimensional point cloud generated by the current fraction prediction and the real point cloud extracted at the end of the current fractionated treatment.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for generating anatomical structures within fractionated radiotherapy based on three-dimensional point clouds is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the online adaptive radiotherapy intra-fractionation anatomical structure generation method based on three-dimensional point cloud as described in any one of claims 1 to 6.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the method for generating anatomical structures within fractionated online adaptive radiotherapy based on three-dimensional point clouds as described in any one of claims 1 to 6.

Images