Method and system for generating anatomical structure in online self-adaptive radiotherapy fractionation based on three-dimensional point cloud

Abstract

The invention provides a three-dimensional point cloud-based online adaptive radiotherapy fractional internal anatomical structure generation method and system, and belongs to the technical field of radiotherapy equipment, and the method comprises the steps: obtaining three-dimensional point cloud data of a fractional internal anatomical structure; and processing the acquired three-dimensional point cloud data by using a pre-trained fractional internal anatomical structure generation model to obtain an anatomical structure three-dimensional point cloud when fractional treatment is finished. According to the invention, the speed of predicting the anatomical structure in batches is increased; the deformation vector of the anatomical structure can be obtained according to the new sketching to guide the structure generation of the next fraction, so that the prediction precision of the anatomical structure in the fraction is improved; the change of the anatomical structure of the patient in the current fraction can be evaluated according to the predicted anatomical structure result, and the online self-adaptive radiotherapy plan design is guided in combination with a clinical decision, so that the fraction internal error caused by organ movement in the online self-adaptive radiotherapy fraction is greatly reduced; and the exposure dose endangering organs can be reduced while the coverage rate of the target region can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of radiotherapy equipment, and particularly relates to a method and system for generating in - fraction anatomical structures based on three - dimensional point clouds for magnetic resonance - guided online adaptive radiation therapy (Online Adaptive Radiation Therapy, abbreviated as OART). Background Art

[0002] The emergence of magnetic resonance - guided linear accelerators (MR - linacs) has promoted the development of magnetic resonance - guided online adaptive radiation therapy. OART can better protect organs at risk and reduce the probability of complications. Compared with conventional radiotherapy techniques, OART adapts the treatment plan online to accommodate the anatomical structure changes between fractions of the tumor target area and organs at risk, and can effectively reduce the errors caused by inter - fraction organ movement. The entire process of OART fractionated treatment includes patient positioning, image acquisition, online contouring, plan update, plan evaluation, and beam delivery, which usually takes 15 - 30 minutes, and the time required in different treatment centers may be longer. During this period, the movement of internal organs is likely to cause the shape and position of the target area and surrounding normal tissue organs to change at different rates and amplitudes, resulting in in - fraction errors. The filling degree of digestive and urinary system organs not only significantly affects the position of the target area, but also affects the movement range of the target area, resulting in significant changes in the doses received by the target area and normal organs. One limitation of OART is that it assumes that the internal organs of the patient remain stationary during the treatment fraction, ignores internal organ movements (such as respiratory movement, bladder filling, etc.), and does not consider the errors caused by in - fraction organ movement. Therefore, quickly generating the anatomical structure at the end of the fractionated treatment at the beginning of OART fractionated treatment is of great significance for optimizing the OART plan and improving the radiotherapy accuracy of OART. Summary of the Invention

[0003] Aiming at the problem of inaccurate dose distribution caused by the failure to consider in - fraction anatomical structure changes in the existing magnetic resonance - guided online adaptive radiotherapy process, the purpose of the present invention is to provide a deep - learning method based on three - dimensional point clouds for generating in - fraction anatomical structures in online adaptive radiotherapy. This method can effectively reduce the errors caused by in - fraction organ movement, ensure the accurate coverage of the treatment target area, and at the same time reduce the radiation dose received by organs at risk.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] In the first aspect, the present invention provides a method for generating in - fraction anatomical structures based on three - dimensional point clouds for online adaptive radiotherapy, including:

[0006] Obtaining three - dimensional point cloud data of in - fraction anatomical structures;

[0007] The acquired three-dimensional point cloud data is processed using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. Among them, the intra-fraction anatomical structure generation model includes: an encoder unit for gradually extracting point cloud features to form a latent space representation; a feature concatenation unit for concatenating the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain concatenated features; a decoder unit for gradually upsampling the concatenated features and synchronously reducing the number of channels to ensure that the output point cloud shape is consistent with the input. The final features are subjected to final prediction processing through a group of multi-layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud.

[0008] As a further limitation of the first aspect of the present invention, acquiring the three-dimensional point cloud data of the intra-fraction anatomical structure includes: delineating the anatomical structure from the high-resolution 3D MRI image data of the tumor region, extracting the contour of the region of interest, and performing smoothing processing using mean filtering. Among them, the mean filtering algorithm converts the volume data into a grid structure composed of vertices and edges, and the triangles in each grid are defined by three adjacent points; the number of points in the anatomical structure is downsampled to be the same as the number of points in the point cloud during the training of the intra-fraction anatomical structure generation model using the z-axis stratification and random uniform downsampling method to obtain the three-dimensional point cloud data of the intra-fraction anatomical structure.

[0009] As a further limitation of the first aspect of the present invention, the encoder unit includes four consecutive SA modules for gradually extracting point cloud features; the first layer SA module initially abstracts the input three-dimensional point cloud data to generate the first layer feature representation; the second layer SA module further reduces the number of vertices, increases the dimension of the features, and captures the mesoscale spatial information; the third layer SA module and the fourth layer SA module continue to gradually extract higher-level and more abstract spatial features, with the number of vertices continuously decreasing and the feature dimension continuously increasing, and finally a compact latent space representation is obtained.

[0010] As a further limitation of the first aspect of the present invention, in the decoder unit, each upsampling layer combines 1×1 convolution with a multi-layer perceptron to gradually expand the scale of the feature space; during the upsampling process, the reduction of the channel dimension is synchronously performed to keep the shape of the output data consistent with the input data; skip connections directly transfer the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features and enhance the prediction accuracy.

[0011] As a further limitation of the first aspect of the present invention, the intra-fraction anatomical structure generation model uses the Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and at the same time introduces an entropy regularization term to improve the stability and efficiency of the calculation.

[0012] As a further limitation of the first aspect of the present invention, a robust point matching algorithm based on thin plate splines is used to find the point correspondence between the three-dimensional point cloud generated by the current fraction prediction and the real point cloud, and the deformation vector is calculated; this deformation vector is used as the input of the intra-fraction anatomical structure generation model to guide the prediction and generation of the corresponding anatomical structure in the next fraction.

[0013] In a second aspect, the present invention provides an online adaptive radiotherapy intra-fraction anatomical structure generation system based on three-dimensional point clouds, including:

[0014] An acquisition module for acquiring three-dimensional point cloud data of the intra-fraction anatomical structure;

[0015] A processing module for processing the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of fractionated treatment; wherein, the intra-fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature splicing unit for splicing the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a spliced feature; a decoder unit for performing progressive upsampling and synchronous reduction of the number of channels on the spliced feature to ensure that the output point cloud shape is consistent with the input, and the final feature is finally predicted and processed through a group of multi-layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud.

[0016] In a third aspect, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which when executed by a processor, implement the method for generating an online adaptive radiotherapy intra-fraction anatomical structure based on three-dimensional point clouds as described in the first aspect.

[0017] In a fourth aspect, the present invention provides a computer device including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for generating an online adaptive radiotherapy intra-fraction anatomical structure based on three-dimensional point clouds as described in the first aspect.

[0018] In a fifth aspect, the present invention provides an electronic device including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the method for generating an online adaptive radiotherapy intra-fraction anatomical structure based on three-dimensional point clouds as described in the first aspect.

[0019] Advantages of the present invention: It avoids introducing complex MRI image information in the traditional method, and improves the prediction speed of the anatomical structure within a fraction; it can obtain the deformation vector of the anatomical structure according to the new delineation to guide the generation of the structure in the next fraction, and can improve the prediction accuracy of the anatomical structure within a fraction; it can predict the anatomical structure of the patient at the end of the treatment at the beginning of the patient's fractionated treatment, evaluate the size of the change in the anatomical structure within the current fraction of the patient according to the prediction result, and combine with clinical decisions to guide the design of the online adaptive radiotherapy plan, greatly reducing the intra-fraction error caused by organ movement within the fraction of the online adaptive radiotherapy, and being able to ensure the target coverage rate while reducing the radiation dose received by the organs at risk.

[0020] The advantages of the additional aspects of the present invention will be more clearly given in the following description part, or understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 It is a schematic diagram of the three-dimensional point cloud of the rectal structure of prostate cancer described in the embodiment of the present invention. The "x", "y", and "z" axes respectively correspond to the left-right (LR), anterior-posterior (AP), and superior-inferior (SI) directions of the patient.

[0023] Figure 2 It is an application flowchart of the method for generating the intra-fraction anatomical structure based on the three-dimensional point cloud described in the embodiment of the present invention. The orange box represents the work before the patient's treatment; the blue box represents the fractionated treatment process of the conventional online adaptive radiotherapy patient; the yellow box represents the steps of integrating the present invention into the conventional process.

[0024] Figure 3 It is an architecture diagram of the intra-fraction anatomical structure generation model described in the embodiment of the present invention.

[0025] Figure 4 It is a schematic diagram of the SA Module module of the generation model described in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following details the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described through the drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation of the present invention.

[0027] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0028] It should also be understood that terms such as those defined in a general dictionary should be understood as having a meaning consistent with their meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless defined as here.

[0029] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "the", and "said" used herein may also include the plural forms. It should be further understood that the phrase "comprising" used in the description of the present invention means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0030] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0031] For ease of understanding of the present invention, the following will further explain the present invention with specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0032] Those skilled in the art should understand that the drawings are only schematic diagrams of the embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0033] The present invention provides a method for generating intra-fraction anatomical structures that is safe, fast, and accurate for radiotherapy. This method is applicable to all patients undergoing magnetic resonance-guided online adaptive radiotherapy. By quickly and accurately generating the anatomical structures at the end of a fractionated treatment at the beginning of the fractionated treatment, it can be used to pre-judge whether significant anatomical shape changes will occur in the patient during this fractionated treatment. This method provides a reference for radiotherapy doctors to help them decide whether it is necessary to adjust the online planning strategy to meet the clinical dosimetry requirements, and further improve the accuracy of online adaptive radiotherapy. With the breakthrough of the method and the upgrading of clinical needs, the limitation that magnetic resonance-guided online adaptive radiotherapy cannot solve the intra-fraction error will be further expanded. Therefore, the method proposed by the present invention can clinically solve the limitations of magnetic resonance-guided online adaptive radiotherapy and has important clinical significance and application value.

[0034] Example 1

[0035] In this Example 1, first, an online adaptive radiotherapy intra-fraction anatomical structure generation system based on three-dimensional point clouds is provided. The system includes: an acquisition module for acquiring three-dimensional point cloud data of intra-fraction anatomical structures; a processing module for processing the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. Among them, the intra-fraction anatomical structure generation model includes: an encoder unit for layer-by-layer extraction of point cloud features to form a latent space representation; a feature splicing unit for splicing the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a spliced feature; a decoder unit for performing progressive upsampling and synchronous reduction of the number of channels on the spliced feature to ensure that the output point cloud shape is consistent with the input. The final feature is finally predicted and processed through a group of multi-layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud. Among them, the intra-fraction anatomical structure generation model uses the Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and at the same time introduces an entropy regularization term to improve the stability and efficiency of the calculation.

[0036] In this embodiment, using the above system, a method for generating intra-fraction anatomical structures for online adaptive radiotherapy based on three-dimensional point clouds is implemented, including: using the acquisition module to acquire three-dimensional point cloud data of intra-fraction anatomical structures; using the processing module to process the acquired three-dimensional point cloud data based on a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment.

[0037] Obtaining the three-dimensional point cloud data of the intra-fraction anatomical structure includes: delineating the anatomical structure from the high-resolution 3D MRI image data of the tumor region, extracting the contour of the region of interest, and performing smoothing processing using mean filtering. Among them, the mean filtering algorithm converts the volume data into a mesh structure composed of vertices and edges, and the triangles in each mesh are defined by three adjacent points; the z-axis stratification and random uniform downsampling method are used to downsample the number of points in the anatomical structure to be the same as the number of points in the point cloud when training the intra-fraction anatomical structure generation model, so as to obtain the three-dimensional point cloud data of the intra-fraction anatomical structure.

[0038] Specifically, data acquisition and processing: At the beginning of the patient's fractionated treatment, use magnetic resonance imaging (MRI) equipment to acquire high-resolution 3D MRI image data of the patient's tumor region; the radiation oncologist delineates the anatomical structures that need to be focused on (such as the tumor target area and adjacent important tissues and organs) according to the new MRI images; when the doctor finishes delineating, import the MRI images and the corresponding anatomical structure RT files into the medical image processing platform for subsequent processing. Extracting the three-dimensional point cloud: In the medical image processing platform, extract the contour of the region of interest from the patient's RT structure file, convert it into a binary mask format, and store it as an MHD file. Use mean filtering to smooth the input MHD file to optimize the structure surface. The mean filtering algorithm converts the volume data into a mesh structure composed of vertices and edges, and the triangles in each mesh are defined by three adjacent points. The number of vertices of the anatomical structure grid for each fraction is inconsistent. In order to speed up the anatomical structure generation process, the present invention uses the z-axis stratification and random uniform downsampling method to downsample the number of points in the anatomical structure to be the same as the number of points in the point cloud during model training. Through the above processing, the intra-fraction anatomical structure can be represented in the form of a three-dimensional point cloud.

[0039] The encoder unit includes four consecutive SA modules to extract point cloud features layer by layer; the first-layer SA module performs preliminary abstraction on the input three-dimensional point cloud data to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices, increases the dimension of the features, and captures the mesoscale spatial information; the third-layer SA module and the fourth-layer SA module continue to gradually extract higher-level and more abstract spatial features, with the number of vertices continuously decreasing and the feature dimension continuously increasing, and finally a compact latent space representation is obtained.

[0040] Specifically, the encoder part uses four consecutive PointNetSetAbstraction (SA) modules to extract point cloud features layer by layer, gradually reducing the number of three-dimensional point clouds while increasing the feature dimension, and gradually compressing the intrinsic geometric features of point cloud data into a compact potential representation. Specifically, the encoder uses the PointNetSetAbstraction technology in the PointNet++ structure to gradually enhance the ability to perceive the geometric features of point cloud data in a hierarchical manner, thereby capturing multi-scale spatial information. The first-layer SA module performs a preliminary abstraction on the input three-dimensional point cloud to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices while increasing the feature dimension to capture the mesoscale spatial information in more detail; the third and fourth-layer SA modules continue to gradually extract higher-level and more abstract spatial features, with the number of vertices decreasing and the feature dimension increasing, ultimately forming a compact latent space representation.

[0041] In feature splicing, the three-dimensional point cloud shape of the anatomical structure and the deformation vector of the previous division are extracted through two encoders, and then spliced in the feature dimension to enhance the model's perception of the input data and improve the model's prediction accuracy.

[0042] In the decoder unit, each upsampling layer is combined with a multi-layer perceptron through 1×1 convolution to gradually expand the scale of the feature space; during the upsampling process, the channel dimension is reduced simultaneously to keep the shape of the output data consistent with the input data; the jump connection directly transmits the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features and enhance the prediction accuracy.

[0043] Specifically, the decoder part is specially designed for the progressive upsampling of features and the synchronous dimensionality reduction of the number of channels, ensuring that the output point cloud shape is consistent with the input. Each feature upsampling layer aims to expand the spatial feature dimension to finely reconstruct the point cloud representation; at the same time, the accompanying channel dimensionality reduction layer can reduce the number of channels in the feature space, ensuring the accuracy and efficiency of the output shape. This multi-level structure enables the model to efficiently capture and reconstruct the spatial features of the input point cloud while maintaining the integrity and accuracy of the point cloud shape. Each upsampling layer combines 1×1 convolution (1×1Conv) with a multi-layer perceptron (MLP) to gradually expand the scale of the feature space; during the upsampling process, the dimensionality reduction of the channel dimension is synchronously performed to keep the shape of the output data consistent with the input data; skip connections directly transfer the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to help the model more effectively reconstruct the spatial detail features and enhance the prediction accuracy. The final features of the decoder are finally predicted and processed through a group of multi-layer perceptrons (MLPs) and 1×1 convolution layers to output the predicted anatomical structure point cloud. The model uses the Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud. The Sinkhorn distance is a regularized Earth Mover's Distance (EMD), which introduces an entropy regularization term to improve the stability and efficiency of the calculation.

[0044] A robust point matching algorithm based on thin plate splines is used to find the point correspondence between the three-dimensional point cloud generated by the current fraction prediction and the real point cloud, and calculate the deformation vector; this deformation vector is used as the input of the intra-fraction anatomical structure generation model to guide the prediction and generation of the corresponding anatomical structure in the next fraction. Specifically, the deformation vector of the current fraction anatomical structure is obtained to guide the generation of the next fraction anatomical structure. At the end of the current fraction of patient treatment, an MRI image is taken, and the doctor outlines the relatively concerned anatomical structure in the current image, extracts the three-dimensional point cloud and uses it as the ground truth. A robust point matching algorithm based on thin plate splines (TPS-RPM) is used to find the point correspondence between the three-dimensional point cloud generated by the current fraction prediction and the real point cloud, and calculate the deformation vector. This deformation vector is used as the input of the model to guide the prediction and generation of the corresponding anatomical structure in the next fraction. It should be noted that when a new patient is treated for the first time, we cannot obtain the deformation vector, so in the first fraction treatment, the deformation vector is a zero vector.

[0045] Embodiment 2

[0046] Aiming at the problem that the dose distribution is inaccurate due to the lack of consideration of the anatomical structure changes within a fraction in the existing magnetic resonance-guided online adaptive radiotherapy process, in this Example 2, a deep learning method based on three-dimensional point cloud is proposed for the rapid generation of the anatomical structure within a fraction in online adaptive radiotherapy. This method can effectively reduce the errors caused by the organ movement within a fraction, ensure the accurate coverage of the treatment target area, and at the same time reduce the radiation dose received by the organs at risk.

[0047] As Figures 1 to 4 shown, the deep learning method based on three-dimensional point cloud in this embodiment specifically includes the following steps:

[0048] 1. Data acquisition and processing: At the beginning of the patient's fractionated treatment, use a magnetic resonance imaging (MRI) device to acquire high-resolution 3D MRI image data of the patient's tumor area; the radiotherapy doctor preferentially delineates the anatomical structures of key concern (such as the tumor target area and adjacent important tissues and organs) according to the new MRI images; when the doctor finishes the delineation, import the MRI images and the corresponding anatomical structure RT files into the medical image processing platform for subsequent processing.

[0049] 2. Extract three-dimensional point cloud: In the medical image processing platform, extract the contour of the region of interest from the patient's RT structure file, convert it into a binary mask format, and store it as an MHD file. Use mean filtering to smooth the input MHD file to optimize the structure surface. The mean filtering algorithm converts the volume data into a grid structure composed of vertices and edges, and the triangles in each grid are defined by three adjacent points. The number of vertices of the anatomical structure grid for each fraction is inconsistent. To accelerate the anatomical structure generation process, the present invention uses the z-axis stratification and random uniform downsampling method to downsample the number of point clouds of the anatomical structure to be the same as the number of point clouds during model training. Through the above processing, the anatomical structure within a fraction can be represented in the form of a three-dimensional point cloud.

[0050] 3. Construction of the anatomical structure generation model within a fraction: The present invention proposes a generation model architecture based on three-dimensional point cloud (SA-UNet), which can directly process three-dimensional point cloud data and quickly and accurately predict the anatomical structure through the feature extraction method of deep learning. The model architecture is as follows:

[0051] Input and output definition: The input data of the model is two three-dimensional point cloud tensors, with the dimension of [B, 3, X], where B represents the batch size, the channels respectively represent the x, y, and z coordinates of the three-dimensional point cloud, and X represents the number of point clouds after downsampling. The output data is a tensor of the same size, with the dimension of [B, X, 3], which is consistent with the input format and is used to represent the predicted anatomical structure point cloud.

[0052] Encoder design: Hierarchical feature extraction. The encoder part uses four consecutive PointNetSetAbstraction (SA) modules to extract point cloud features layer by layer, gradually reducing the number of 3D point clouds while increasing the feature dimension, and gradually compressing the intrinsic geometric features of the point cloud data into a compact potential representation. Specifically, the encoder uses the PointNetSetAbstraction technology in the PointNet++ structure to gradually enhance the ability to perceive the geometric features of point cloud data in a hierarchical manner, thereby capturing multi-scale spatial information. The first-layer SA module performs a preliminary abstraction on the input 3D point cloud to generate the first-layer feature representation; the second-layer SA module further reduces the number of vertices while increasing the feature dimension to capture the mesoscale spatial information in more detail; the third and fourth-layer SA modules continue to gradually extract higher-level and more abstract spatial features, with the number of vertices decreasing and the feature dimension increasing, eventually forming a compact latent space representation.

[0053] Feature splicing: After extracting the features of the 3D point cloud shape of the anatomical structure and the deformation vector of the previous classification through two encoders, they are spliced in the feature dimension to enhance the model's perception of the input data and improve the model's prediction accuracy.

[0054] Decoder design: feature upsampling and channel dimension reduction. The decoder part is specially designed for the step-by-step upsampling of features and the synchronous dimension reduction of the number of channels to ensure that the output point cloud shape is consistent with the input. Each feature upsampling layer aims to expand the spatial feature dimension to finely reconstruct the point cloud representation; at the same time, the accompanying channel dimension reduction layer can reduce the number of channels in the feature space to ensure that the output shape is accurate and efficient. This multi-level structure enables the model to efficiently capture and reconstruct the spatial features of the input point cloud while maintaining the integrity and accuracy of the point cloud shape. Each upsampling layer is combined with a multi-layer perceptron (MLP) through 1*1 convolution to gradually expand the scale of the feature space; during the upsampling process, the channel dimension is reduced synchronously to keep the shape of the output data consistent with the input data; skip connections directly pass the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to help the model more effectively reconstruct the spatial detail features and enhance the prediction accuracy. The final features of the decoder are finally predicted through a set of multi-layer perceptrons (MLP) and 1*1 convolution layers to output the predicted anatomical structure point cloud. The model uses Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud. Sinkhorn distance is a regularized Earth Mover's Distance (EMD), which introduces entropy regularization to improve the stability and efficiency of the calculation.

[0055] 4. Intra-fraction anatomical structure generation: The three-dimensional point cloud of the anatomical structure at the start of the fractionated treatment is input into the model, and the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment is quickly predicted. After inverse normalization processing, the three-dimensional point cloud is written into the MRI image taken at the start of the treatment and stored as an RT structure file.

[0056] 5. Clinical decision-making: Quantitatively analyze the spatial differences between the predicted intra-fraction anatomical structure and the initial treatment anatomical structure, calculate the differences between the initial anatomical structure and the predicted anatomical structure, and determine by the radiation oncologist whether it exceeds the clinically acceptable threshold according to the size of the differences.

[0057] 6. Dose optimization strategy to reduce the error impact caused by intra-fraction anatomical structure deformation: It is determined by the clinician that for regions with large spatial changes, dose limits or adjustments are made to this region during plan optimization to avoid overdose or underdose; when the spatial changes are within the acceptable range, it is designed according to the conventional adaptive plan to improve the treatment accuracy of online adaptive radiotherapy.

[0058] 7. Obtain the deformation vector of the current fraction anatomical structure to guide the generation of the next fraction anatomical structure. At the end of the current fraction patient treatment, an MRI image is taken, and the radiation oncologist delineates the relatively concerned anatomical structures in the current image, extracts the three-dimensional point cloud and uses this as the ground truth. The Thin Plate Spline-based Robust Point Matching algorithm (TPS-RPM) is used to find the point correspondence between the three-dimensional point cloud predicted and generated in the current fraction and the real point cloud, and the deformation vector is calculated. This deformation vector is used as the input of the model to guide the prediction and generation of the corresponding anatomical structure in the next fraction. It should be noted that when a new patient is treated for the first time, we cannot obtain the deformation vector, so in the first fraction treatment, the deformation vector is a zero vector.

[0059] In this Embodiment 2, the following introduces the specific implementation manner of the online adaptive radiotherapy process for a prostate cancer patient using the method described above in this Embodiment 2 on a magnetic resonance accelerator.

[0060] Pre-treatment preparation work: After the patient undergoes CT and MRI localization, the radiation oncologist delineates the target area and organs at risk. After the delineation is completed, it is handed over to the medical physicist for reference plan design, and after being reviewed by the doctor, dose verification is carried out.

[0061] Patient positioning and image scanning: After the technician positions the patient according to the reference plan center, MR scan images are obtained.

[0062] Image registration: Select the online adaptive mode, and map the structures such as the target area and organs at risk on the reference CT image to the currently scanned MR image through deformation registration.

[0063] Contouring of anatomies of relative concern: For prostate cancer patients, the anatomies that are clinically relatively concerned about mainly include the target area, bladder, rectum, seminal vesicles, etc. When doctors contour, these 4 anatomies are contoured first.

[0064] Extracting 3D point cloud: When the contouring of the anatomies of relative concern is completed, an external program extracts the contour of the region of interest from the RT file, converts it into a binary mask format, and stores it as an MHD file. After smoothing the input MHD file using mean filtering, the structure vertices are extracted. Subsequently, z-axis stratification and random uniform downsampling methods are used to downsample the structure vertices to ensure correct input of the model. After obtaining the 3D point cloud of the current fraction, the TPS-RPM algorithm is used to perform deformable registration on the 3D point cloud of the anatomical structure of the current fraction and the real 3D point cloud of the previous fraction, find the corresponding relationship between points, and solve the deformation vector.

[0065] Predicting and generating anatomical structure: Input the current 3D point cloud and deformation vector into SA-UNet to predict and generate the shape of the 3D point cloud of the current fraction. After inverse normalization processing, the 3D point cloud is written into the MRI image taken at the beginning of treatment and stored as an RT structure file.

[0066] Clinical decision-making: It is decided by the radiation oncologist whether the deformation of the anatomies of current concern has a great impact on the dose distribution. If the current deformation size will have a great impact on clinical treatment, dose limits or dose supplementation can be carried out by contouring auxiliary structures. If the deformation size has little impact on clinical treatment, the next step is carried out. There is a problem here, that is, judging the deformation size depends largely on the subjectivity of the radiation oncologist. Therefore, doctors need to make correct judgments according to clinical guidelines.

[0067] Contouring of the remaining anatomies: After clinical decision-making, continue to contour the remaining organs at risk, such as the femoral head, intestines, etc.

[0068] Online adaptive radiotherapy plan design and dose verification: After handing it over to the medical physicist for plan design, it is judged by the doctor whether the current dose distribution meets the clinical requirements. If it meets the clinical requirements, online dose verification can be carried out; if it does not meet the clinical requirements, the medical physicist adjusts the parameters according to the requirements and redesigns the plan.

[0069] Plan execution: The treatment is implemented by the technician. At the end of the patient's treatment, new MR images are obtained. The doctor can contour the anatomies of relative concern according to the new MR images.

[0070] Calculate the deformation vector to guide the next fraction prediction: Outline the relatively concerned anatomical structures from the doctor's new MR image, extract the 3D point cloud and use it as the ground truth. Use the Thin Plate Spline-based Robust Point Matching algorithm (TPS-RPM) to find the point correspondences between the 3D point cloud generated by the current fraction prediction and the real point cloud, and calculate the deformation vector. At the next treatment of this patient, use this deformation vector as the input of the model to guide the prediction generation of the corresponding anatomical structures in the next fraction.

[0071] Example 3

[0072] This Example 3 provides a non-transitory computer-readable storage medium for storing computer instructions, which when executed by a processor, implement the online adaptive radiotherapy intra-fraction anatomical structure generation method based on 3D point cloud as described above. The method includes:

[0073] Obtain the 3D point cloud data of the intra-fraction anatomical structure;

[0074] Use the pre-trained intra-fraction anatomical structure generation model to process the obtained 3D point cloud data to obtain the 3D point cloud of the anatomical structure at the end of fractionated treatment. Among them, the intra-fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature splicing unit for splicing the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain the spliced features; a decoder unit for performing progressive upsampling and synchronous reduction of the number of channels on the spliced features to ensure that the output point cloud shape is consistent with the input, and the final features are finally predicted and processed through a set of multi-layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud.

[0075] Example 4

[0076] This Example 4 provides a computer device including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the online adaptive radiotherapy intra-fraction anatomical structure generation method as described above. The method includes:

[0077] Obtain the 3D point cloud data of the intra-fraction anatomical structure;

[0078] Process the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain concatenated features; a decoder unit for performing progressive upsampling on the concatenated features and synchronously reducing the number of channels to ensure that the output point cloud shape is consistent with the input, and the final features are processed by a set of multi-layer perceptrons and 1*1 convolutional layers for final prediction to obtain the predicted anatomical structure point cloud.

[0079] Embodiment 5

[0080] Embodiment 5 of the present invention provides an electronic device, including: a processor, a memory, and a computer program. The processor is connected to the memory, and the computer program is stored in the memory. When the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the above-mentioned method for generating the intra-fraction anatomical structure in online adaptive radiotherapy based on three-dimensional point clouds. The method includes:

[0081] Obtain the three-dimensional point cloud data of the intra-fraction anatomical structure;

[0082] Process the acquired three-dimensional point cloud data using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment. The intra-fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature concatenation unit for concatenating the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain concatenated features; a decoder unit for performing progressive upsampling on the concatenated features and synchronously reducing the number of channels to ensure that the output point cloud shape is consistent with the input, and the final features are processed by a set of multi-layer perceptrons and 1*1 convolutional layers for final prediction to obtain the predicted anatomical structure point cloud.

[0083] In summary, the innovations of the present invention include the following points:

[0084] (1) For the first time, a method for predicting the intra-fraction anatomical structure based on three-dimensional point cloud is proposed. This method avoids introducing complex MRI image information in traditional methods and improves the speed of intra-fraction anatomical structure prediction. At the end of the current fraction, the anatomical structure deformation vector can be obtained according to the new delineation to guide the generation of the structure in the next fraction, which can improve the accuracy of intra-fraction anatomical structure prediction. (2) The present invention can predict the anatomical structure of a patient at the end of the treatment when the patient's fractionated treatment starts. According to the prediction result, the size of the intra-fraction anatomical structure change of the patient can be evaluated, and combined with clinical decision-making to guide the design of online adaptive radiotherapy plan, greatly reducing the intra-fraction error caused by organ movement during online adaptive radiotherapy fractionation, and being able to reduce the irradiated dose of organs at risk while ensuring the target coverage rate. (3) This method can be integrated into the existing magnetic resonance-guided radiotherapy process, is applicable to multiple diseases and multiple parts, and has high clinical applicability and practical value.

[0085] In summary, the present invention proposes a method for generating intra-fraction anatomical structure based on three-dimensional point cloud, which not only overcomes the technical limitation of ignoring intra-fraction anatomical changes in existing magnetic resonance-guided online adaptive radiotherapy, but also improves the accuracy of treatment, and has important clinical significance and application prospects.

[0086] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0088] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the function specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0090] Although the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, they are not intended to limit the scope of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative efforts should be covered within the scope of the present invention.

Claims

1. An online adaptive radiotherapy fractionated internal anatomical structure generation method based on three-dimensional point cloud, characterized in that, Comprising: Obtaining three-dimensional point cloud data of the in - fraction anatomical structure; Processing the obtained three-dimensional point cloud data using a pre-trained in - fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of in - fraction treatment; wherein, the in - fraction anatomical structure generation model includes: an encoder unit for extracting point cloud features layer by layer to form a latent space representation; a feature splicing unit for splicing the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a spliced feature; a decoder unit for performing progressive upsampling and synchronous reduction of the number of channels on the spliced feature to ensure that the output point cloud shape is consistent with the input, and the final feature is subjected to final prediction processing through a group of multi - layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud.

2. The method for generating an in - fraction anatomical structure online adaptively based on three - dimensional point cloud according to claim 1, wherein, Obtaining three-dimensional point cloud data of the in - fraction anatomical structure includes: delineating the anatomical structure from the high - resolution 3D MRI image data of the tumor region, extracting the contour of the region of interest, and performing smoothing processing using mean filtering. Among them, the mean filtering algorithm converts the volume data into a grid structure composed of vertices and edges, and each triangle in the grid is defined by three adjacent points; using z - axis stratification and random uniform downsampling method to downsample the number of points in the anatomical structure to be the same as the number of points in the point cloud during the training of the in - fraction anatomical structure generation model to obtain the three - dimensional point cloud data of the in - fraction anatomical structure.

3. The online adaptive radiotherapy fractionated internal anatomical structure generation method based on three-dimensional point cloud according to claim 1, wherein The encoder unit includes four consecutive SA modules for extracting point cloud features layer by layer; The first - layer SA module initially abstracts the input three - dimensional point cloud data to generate the first - layer feature representation; the second - layer SA module further reduces the number of vertices and increases the dimension of the features to capture mesoscale spatial information; the third - layer SA module and the fourth - layer SA module continue to gradually extract higher - level and more abstract spatial features, with the number of vertices continuously decreasing and the dimension of the features continuously increasing, finally obtaining a compact latent space representation.

4. The method for generating an in - fraction anatomical structure online adaptively based on 3D point cloud according to claim 1, wherein, In the decoder unit, each upsampling layer combines 1×1 convolution with a multi - layer perceptron to gradually expand the scale of the feature space; during the upsampling process, the reduction of the channel dimension is synchronized to keep the shape of the output data consistent with the input data; skip connections directly transfer the features output by the corresponding layer of the encoder to the corresponding layer of the decoder to reconstruct the spatial detail features and enhance the prediction accuracy.

5. The method for generating an in - fraction anatomical structure online adaptively based on three - dimensional point cloud according to claim 1, wherein, The in - fraction anatomical structure generation model uses the Sinkhorn distance as the loss function to ensure the spatial consistency between the predicted point cloud and the real point cloud, and at the same time introduces an entropy regularization term to improve the stability and efficiency of the calculation.

6. The method for generating an in - fraction anatomical structure online adaptively based on a three - dimensional point cloud according to claim 1, wherein, Using a robust point matching algorithm based on thin - plate splines to find the point - to - point correspondence between the three - dimensional point cloud predicted and generated in the current fraction and the real point cloud, and calculating the deformation vector; Taking this deformation vector as the input of the in - fraction anatomical structure generation model to guide the prediction and generation of the corresponding anatomical structure in the next fraction.

7. An online adaptive radiotherapy fractionated internal anatomical structure generation system based on three-dimensional point cloud, characterized in that, Comprising: An acquisition module for obtaining three - dimensional point cloud data of the in - fraction anatomical structure; A processing module, configured to process the acquired three-dimensional point cloud data by using a pre-trained intra-fraction anatomical structure generation model to obtain the three-dimensional point cloud of the anatomical structure at the end of the fractionated treatment; wherein, the intra-fraction anatomical structure generation model includes: an encoder unit, configured to extract point cloud features layer by layer to form a latent space representation; a feature splicing unit, configured to splice the results of feature extraction of the latent space representations obtained by two encoders and the deformation vector of the previous fraction in the feature dimension to obtain a spliced feature; a decoder unit, configured to perform upsampling step by step and synchronous reduction of the number of channels on the spliced feature to ensure that the output point cloud shape is consistent with the input, and the final feature is subjected to final prediction processing through a group of multi-layer perceptrons and 1*1 convolutional layers to obtain the predicted anatomical structure point cloud.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method for generating an intra-fraction anatomical structure for online adaptive radiotherapy based on three-dimensional point cloud according to any one of claims 1-6 is implemented.

9. A computer device, characterized in that, It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for generating an intra-fraction anatomical structure for online adaptive radiotherapy based on three-dimensional point cloud according to any one of claims 1-6.

10. An electronic device, characterized in that, It includes: A processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes the instructions for implementing the method for generating an intra-fraction anatomical structure for online adaptive radiotherapy based on three-dimensional point cloud according to any one of claims 1-6.

Images