A target position prediction method and system based on body surface point cloud data

Abstract

The present invention discloses a target area position prediction method and system based on body surface point cloud data, which relates to the technical field of medical signal processing. The method comprises obtaining 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of several subjects, establishing an in-vivo point cloud dataset and a body surface point cloud annotation dataset; using the body surface point cloud annotation dataset and the in-vivo point cloud dataset to train a body surface and body surface point cloud association model; inputting the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment into the trained body surface and body surface point cloud association model; obtaining a final predicted phase and a final target area transformation matrix; and combining the 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment to obtain the predicted position of the in-vivo target area on the body surface point cloud contour at the current respiratory moment. The present invention achieves accurate prediction of the target area position on the body surface contour, improves the accuracy of radiotherapy irradiation, and reduces radiation hazards.

Description

Technical Field

[0001] The present invention relates to the technical field of medical signal processing, and more particularly, to a target area position prediction method and system based on body surface point cloud data. Background Art

[0002] During radiotherapy, the location of the patient's target area can vary due to respiratory motion. Accurately locating and tracking the target area in real time can help reduce radiation exposure during treatment. Currently, there are several mainstream radiotherapy approaches for tracking internal targets. The first is marker tracking, which uses external cameras and surface markers to monitor the patient's surface motion in real time. These surface markers are then used to infer the location of internal lesions, i.e., the target area. However, this approach has the disadvantage of tracking surface motion in real time, but the inference of the target area's location is less accurate. Furthermore, this approach relies on the patient's high level of cooperation and the accuracy of the surface markers. It cannot accurately infer the target area's location when surface features change significantly, such as during surgery or swelling. The second approach is ultrasound surface-to-body correlation, which uses ultrasound equipment to monitor the relative positions of the surface and interior of the body in real time and then correlates these using image processing techniques. However, this approach suffers from low ultrasound image resolution and contrast. Furthermore, ultrasound transmission through bone and air is limited, limiting its application to certain parts of the human body and requiring high operator skill. Alternatively, there are other approaches that use high-precision equipment to generate in-vivo images for observation. However, due to the limitations of radiotherapy scenarios, high-precision equipment is difficult to use in real-time clinical settings. Furthermore, high-precision equipment is expensive and technically complex, and the acquisition and processing of images takes a long time, making it impossible to generate in-vivo images in real time. Therefore, there is an urgent need for a method that is efficient, accurate, highly adaptable to specific scenarios, and requires minimal patient cooperation. This method can accurately correlate the patient's internal target area with the body's surface contours, enabling precise positioning and real-time tracking of the target area. Summary of the Invention

[0003] One of the purposes of the present invention is to provide a target area position prediction method based on body surface point cloud data to overcome the defect that the above-mentioned existing technology cannot accurately predict the target area position on the body surface contour; the second purpose is to provide a target area position prediction system based on body surface point cloud data; the third purpose is to provide a computer device; the fourth purpose is to provide a readable storage medium.

[0004] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0005] The present invention provides a target area position prediction method based on body surface point cloud data, comprising:

[0006] Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects;

[0007] Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase were obtained to establish an in vivo point cloud dataset;

[0008] Based on the body surface point cloud data within the respiratory cycle of each subject, a body surface point cloud contour at each respiratory moment of each subject is obtained;

[0009] Based on the in-vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contours of each subject at each breathing moment to establish a body surface point cloud labeling dataset;

[0010] Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model;

[0011] Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix;

[0012] Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current respiratory moment, the final predicted phase of the body surface point cloud contour at the current respiratory moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current respiratory moment is obtained.

[0013] Preferably, based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase are obtained to establish an in vivo point cloud dataset, including:

[0014] For each subject, the 4DCT reconstructed point cloud data is divided evenly according to the phase to obtain the subject's Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target area position coordinates in the body into one element , establish the patient's in vivo point cloud dataset;

[0015] in, Indicates the subject's The reconstructed point cloud contour of the phase, Indicates the subject's The target position in the body of the reconstructed point cloud contour of the phase, Represents the subjects’ The position of the target area in the body of the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases.

[0016] Preferably, obtaining the body surface point cloud contour of each subject at each breathing moment based on the body surface point cloud data within the respiratory cycle of each subject includes:

[0017] For each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject The surface point cloud contour of each breathing moment ,in, Indicates the subject's The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

[0018] Preferably, based on the in-vivo point cloud dataset, phase annotation is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, including:

[0019] For any subject’s body surface point cloud contour at any breathing moment, the contours are compared with the body point cloud dataset of the subject. Align the reconstructed point cloud contours of each phase;

[0020] After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases is obtained The average vector distance;

[0021] Compare The size of the average vector distance is used, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment, thereby obtaining the body surface point cloud contour annotation data at the breathing moment;

[0022] Traversing the body surface point cloud contour of the subject at each breathing moment to obtain body surface point cloud contour annotation data of the subject at each breathing moment;

[0023] All subjects were traversed to obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and a body surface point cloud annotation dataset was established.

[0024] Preferably, for any subject's body surface point cloud contour at any breathing moment, the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method are used to compare the contour of the subject's body surface point cloud dataset with the contour of the subject's body surface point cloud dataset. The reconstructed point cloud contours of the phases are aligned.

[0025] Preferably, after alignment, the body surface point cloud contours at the breathing moment and The average vector distance of the reconstructed point cloud contour of the phases is obtained Average vector distances, including:

[0026] For the aligned body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at any phase, calculate the vector distance between the corresponding point cloud pairs;

[0027] Based on the vector distances of all point cloud pairs, the average vector distance between the body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at the phase is calculated;

[0028] Traversal The reconstructed point cloud contour of the phase is obtained The average vector distance.

[0029] Preferably, the vector distance includes any one of Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance.

[0030] Preferably, the surface point cloud annotated dataset and the in-vivo point cloud dataset are input into a constructed surface-body-in-vivo point cloud association model for optimization training to obtain a trained surface-body-in-vivo point cloud association model, including:

[0031] Constructing a body surface and body body point cloud association model, inputting a body surface point cloud annotation dataset and a body body point cloud dataset, wherein the body surface and body body point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model;

[0032] Inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment;

[0033] Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject;

[0034] For each subject's body surface point cloud contour at each respiratory moment, select the reconstructed point cloud contour corresponding to the final predicted phase from the corresponding in vivo point cloud dataset, input the reconstructed point cloud contour together with the body surface point cloud contour at that respiratory moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of each subject's body surface point cloud contour at each respiratory moment;

[0035] The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body point cloud association model.

[0036] Preferably, the phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer connected in sequence.

[0037] Preferably, the target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer;

[0038] The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer;

[0039] The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

[0040] Preferably, the first loss function is:

[0041]

[0042] Where, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates the The point cloud contour of the individual table belongs to True categories, Indicates the The point cloud contour of the individual table belongs to The predicted probability of the true class.

[0043] Preferably, the second loss function is:

[0044]

[0045] Where, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding true phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

[0046] Preferably, obtaining the predicted position of the target area in the body on the body point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix includes:

[0047] In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

[0048] The present invention also provides a target area position prediction system based on body surface point cloud data, based on the above prediction method, comprising:

[0049] A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within the respiratory cycle of several subjects;

[0050] The first data partitioning module is used to reconstruct the point cloud data of each subject based on the 4DCT, obtain the reconstructed point cloud contour and the position coordinates of the target area in the body of each subject in each phase, and establish an in vivo point cloud data set;

[0051] A second data segmentation module is configured to obtain a body surface point cloud contour at each breathing moment of each subject based on the body surface point cloud data within the respiratory cycle of each subject;

[0052] a data annotation module, configured to perform phase annotation on the body surface point cloud contours of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset;

[0053] A model training module is used to input the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model;

[0054] The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix;

[0055] The target area position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix.

[0056] The present invention also provides a computer device, comprising a memory and a processor;

[0057] The memory is used to store programs;

[0058] The processor is used to execute the program to implement the various steps of the aforementioned target area position prediction method based on body surface point cloud data.

[0059] The present invention also provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various steps of the aforementioned target area position prediction method based on body surface point cloud data are implemented.

[0060] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0061] The present invention first obtains 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of a large number of subjects, divides the 4DCT reconstructed point cloud data according to phase, obtains the reconstructed point cloud contour and the in vivo target area position coordinates of each phase, and establishes an in vivo point cloud dataset; divides the body surface point cloud data within the respiratory cycle according to respiratory time, and obtains the body surface point cloud contour at each respiratory time; then, for the body surface point cloud contour at each respiratory time, searches for the nearest reconstructed point cloud contour, and uses the corresponding phase to perform phase annotation on the body surface point cloud contour at each respiratory time, and establishes a body surface point cloud annotation dataset; the phase annotation focuses on the distance difference feature between the body surface point cloud contour and the reconstructed point cloud contour; then, the body surface point cloud contour is annotated. The cloud annotation dataset and the in-vivo point cloud dataset are input into the constructed surface-in-vivo point cloud association model for optimization training to obtain a trained surface-in-vivo point cloud association model; finally, the patient's 4DCT reconstructed point cloud data and the surface point cloud contour at the current respiratory moment are obtained and input into the trained surface-in-vivo point cloud association model, which can further capture the global distribution characteristics and local distribution characteristics between the surface point cloud contour and the reconstructed point cloud contour, predict the reconstructed point cloud contour that is most similar to the surface point cloud contour, output the final predicted phase and the final target area transformation matrix, and obtain the predicted position of the in-vivo target area on the surface point cloud contour at the current respiratory moment by fusing the in-vivo target area position coordinates of the most similar reconstructed point cloud contour with the final target area transformation matrix. The present invention only needs to perform 4DCT reconstruction once before radiotherapy to obtain 4DCT reconstructed point cloud data, which reduces radiation hazards. Afterwards, the patient is positioned to obtain the body surface point cloud contour at the current breathing moment in real time, and the contours are input into the trained body surface and body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The body target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the body target area on the body surface point cloud contour at the current breathing moment, thereby realizing accurate prediction of the target area position on the body surface contour and improving the accuracy of radiotherapy irradiation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a target area position prediction method based on body surface point cloud data described in Example 1;

[0063] Figure 2 This is a flow chart of establishing a body surface point cloud annotation dataset as described in Example 2;

[0064] Figure 3 This is a schematic diagram of the structure of the phase prediction sub-model described in Example 2;

[0065] Figure 4 This is a schematic diagram of the structure of the target area transformation matrix calculation sub-model described in Example 2;

[0066] Figure 5This is a schematic structural diagram of a target area position prediction system based on body surface point cloud data as described in Example 3;

[0067] Figure 6 This is a structural diagram of a computer device described in Example 3. DETAILED DESCRIPTION

[0068] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0069] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0070] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0071] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0072] Example 1

[0073] This embodiment provides a target area position prediction method based on body surface point cloud data, such as Figure 1 Shown, including:

[0074] S1: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects;

[0075] S2: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase are obtained to establish an in vivo point cloud dataset;

[0076] S3: Based on the body surface point cloud data of each subject during the respiratory cycle, obtain the body surface point cloud contour of each subject at each respiratory moment;

[0077] S4: Based on the in-vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contours of each subject at each breathing moment to establish a body surface point cloud labeling dataset;

[0078] S5: inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model;

[0079] S6: Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix;

[0080] S7: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained.

[0081] In the specific implementation process, this embodiment first obtains 4DCT reconstructed point cloud data and body surface point cloud data within the respiratory cycle of a large number of subjects, divides the 4DCT reconstructed point cloud data according to the phase, obtains the reconstructed point cloud contour and the in vivo target area position coordinates of each phase, and establishes an in vivo point cloud data set; divides the body surface point cloud data within the respiratory cycle according to the respiratory moment, and obtains the body surface point cloud contour at each respiratory moment; then, searches for the nearest reconstructed point cloud contour for the body surface point cloud contour at each respiratory moment, and uses the corresponding phase to perform phase annotation on the body surface point cloud contour at each respiratory moment, and establishes a body surface point cloud annotation data set; the phase annotation focuses on the distance difference feature between the body surface point cloud contour and the reconstructed point cloud contour; then , the surface point cloud annotation dataset and the in-vivo point cloud dataset are input into the constructed surface-in-vivo point cloud association model for optimization training to obtain a trained surface-in-vivo point cloud association model; finally, the patient's 4DCT reconstructed point cloud data and the surface point cloud contour at the current respiratory moment are obtained and input into the trained surface-in-vivo point cloud association model, which can further capture the global distribution characteristics and local distribution characteristics between the surface point cloud contour and the reconstructed point cloud contour, predict the reconstructed point cloud contour that is most similar to the surface point cloud contour, output the final predicted phase and the final target area transformation matrix, and obtain the predicted position of the in-vivo target area on the surface point cloud contour at the current respiratory moment by fusing the in-vivo target area position coordinates of the most similar reconstructed point cloud contour with the final target area transformation matrix. In this embodiment, only one 4DCT reconstruction is required before radiotherapy to obtain 4DCT reconstructed point cloud data, which reduces radiation hazards. Afterwards, the patient is positioned to obtain the surface point cloud contour at the current breathing moment in real time, and the contours are input into the trained surface-body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The in vivo target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the surface point cloud contour at the current breathing moment, thereby achieving accurate prediction of the target area position on the surface contour and improving the accuracy of radiotherapy irradiation.

[0082] Example 2

[0083] This embodiment provides a target area position prediction method based on body surface point cloud data, including:

[0084] S1: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects;

[0085] It should be noted that 4DCT (Four-Dimensional Computed Tomography) reconstruction data is obtained in the CT scanning room. 4DCT reconstruction technology is an imaging technology that can capture and reconstruct anatomical structure information that changes over time. It continuously scans the patient over multiple respiratory cycles to obtain a series of three-dimensional images. Because 4DCT reconstruction technology generates images by emitting X-rays into the body and capturing the signals after they penetrate the body, it emits a certain amount of radiation to the human body. Therefore, the number of CT scans should be minimized during radiotherapy to reduce radiation hazards.

[0086] It's important to note that after the subject is positioned for radiotherapy, a 3D camera collects surface point cloud data for at least one respiratory cycle. Subsequent steps eliminate the need for a CT scan during each radiotherapy session and correlate it with the surface point cloud data, reducing radiation hazards.

[0087] It is understandable that the several subjects include subjects of different ages, treatment sites, and genders, so that the 4DCT reconstructed point cloud data and the body surface point cloud data within the respiratory cycle are sufficiently rich to ensure the accuracy of the subsequently trained body surface and body point cloud association model.

[0088] S2: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase are obtained to establish an in vivo point cloud dataset;

[0089] It can be understood that for each subject, the 4DCT reconstructed point cloud data is divided evenly according to the phase to obtain the subject's Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target area position coordinates in the body into one element , establish the patient's in vivo point cloud dataset;

[0090] in, Indicates the subject's The reconstructed point cloud contour of the phase, Indicates the subject's The target position in the body of the reconstructed point cloud contour of the phase, Represents the subjects’ The position of the target area in the body of the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases; in this embodiment, .

[0091] S3: Based on the body surface point cloud data of each subject during the respiratory cycle, obtain the body surface point cloud contour of each subject at each respiratory moment;

[0092] It is understandable that for each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject's The surface point cloud contour of each breathing moment ,in, Indicates the subject's The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

[0093] S4: Based on the in vivo point cloud dataset, perform phase annotation on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, such as Figure 2 Shown, including:

[0094] S41: For any subject’s body surface point cloud contour at any breathing moment, compare it with the body point cloud dataset of the subject Align the reconstructed point cloud contours of each phase;

[0095] S42: After alignment, calculate the contour of the body surface point cloud at the breathing moment and The average vector distance of the reconstructed point cloud contour of the phases is obtained The average vector distance;

[0096] S43: Comparison The size of the average vector distance is used, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment, thereby obtaining the body surface point cloud contour annotation data at the breathing moment;

[0097] S44: repeating steps S41-S43, traversing the body surface point cloud contour at each breathing moment, and obtaining the body surface point cloud contour annotation data of the subject at each breathing moment;

[0098] S45: Repeat steps S41-S44, traverse each subject, obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and establish a body surface point cloud annotation dataset.

[0099] In this embodiment, LabelFusion is used to label the true phase of the body surface point cloud contour.

[0100] It should be noted that in step S41, there is no limitation on the method for aligning the body surface point cloud contour with the reconstructed point cloud contour, such as the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method. In practical applications, a suitable alignment method is selected according to specific needs and the characteristics of the current point cloud data.

[0101] It should be noted that in step S42, the distance metric for calculating the average vector distance between the surface point cloud contour at any breathing moment and the reconstructed point cloud contour in the in-vivo point cloud dataset is not limited, such as Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance, etc. After calculating the distances between all vector pairs, the average is taken to obtain the average vector distance.

[0102] S5: Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model, including:

[0103] S51: constructing a body surface and body body point cloud association model, inputting a body surface point cloud annotation dataset and a body body point cloud dataset, wherein the body surface and body body point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model;

[0104] S52: inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment;

[0105] It should be noted that if Figure 3 As shown, the phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer connected in sequence.

[0106] The dimension of the body surface point cloud annotation dataset is ( ),in Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, represents the coordinate dimension, Represents the point feature dimension; after the first sampling grouping, the dimension is ( ) of the first point set; after the first point set is extracted by the first PointNet layer, the dimension is ( ), where Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Represents the number of features; the dimension obtained by the second sampling grouping layer is ( ) of the second point set, input the second PointNet layer to obtain a dimension of ( ), where represents the number of points in the second point set, The second feature matrix represents the feature dimension of the points in the second point set. The second feature matrix is input into the self-attention mechanism layer. The first and second PointNet layers can extract the global features of the point cloud, while the self-attention mechanism layer can extract local features of the point cloud, increasing attention to the correlation of local features. Finally, it is input into the fully connected layer for classification prediction, outputting the predicted phase of the body surface point cloud contour.

[0107] S53: Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject;

[0108] The first loss function is:

[0109]

[0110] Where, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates the The point cloud contour of the individual table belongs to True categories, Indicates the The point cloud contour of the individual table belongs to The predicted probability of the true class.

[0111] S54: for the body surface point cloud contour at each respiratory moment of each subject, select the reconstructed point cloud contour corresponding to the final predicted phase from the corresponding in vivo point cloud dataset, input the reconstructed point cloud contour together with the body surface point cloud contour at the respiratory moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of the body surface point cloud contour at each respiratory moment of each subject;

[0112] It should be noted that if Figure 4As shown, the target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer;

[0113] The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer;

[0114] The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

[0115] The second loss function is:

[0116]

[0117] Where, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding true phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

[0118] It should be noted that the trained phase prediction sub-model outputs the final predicted phase Take the example to explain, then in the subject’s in vivo point cloud dataset, select Reconstructed point cloud contours of phases As the first input ; Select the body surface point cloud contour at the breathing moment as the second input ; The first DGCNN layer and the second DGCNN layer are set in parallel, with the same structure and shared parameters. Input the first DGCNN layer, Input the second DGCNN layer, extract the high-dimensional feature vector matrix respectively, and extract the global feature correlation of the first and second Transformer layers to obtain the first residual term. and the second residual ; The first residual term and the second residual Input Pointer layer for PointerGeneration. In order to avoid the occurrence of non-differentiable situations, this embodiment uses the probability method for PointerGeneration, and the generation formula is: The generated pointer is input into the SVD layer to obtain the final target area transformation matrix.

[0119] S55: The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body point cloud association model.

[0120] S6: Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix;

[0121] S7: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current respiratory moment, the final predicted phase of the body surface point cloud contour at the current respiratory moment, and the final target region transformation matrix, the predicted position of the target region in the body on the body surface point cloud contour at the current respiratory moment is obtained, including:

[0122] In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

[0123] In actual use, this embodiment only needs to perform a 4DCT reconstruction before radiotherapy to obtain 4DCT reconstructed point cloud data, which reduces radiation hazards. After that, the patient is positioned to obtain the surface point cloud contour at the current breathing moment in real time, and the contours are input into the trained surface-body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The in vivo target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the surface point cloud contour at the current breathing moment, thereby realizing accurate prediction of the target area position on the surface contour and improving the accuracy of radiotherapy irradiation.

[0124] Example 3

[0125] This embodiment provides a target area position prediction system based on body surface point cloud data, based on the prediction method described in embodiment 1 or 2, such as Figure 5 Shown, including:

[0126] A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within the respiratory cycle of several subjects;

[0127] The first data partitioning module is used to reconstruct the point cloud data of each subject based on the 4DCT, obtain the reconstructed point cloud contour and the position coordinates of the target area in the body of each subject in each phase, and establish an in vivo point cloud data set;

[0128] A second data segmentation module is configured to obtain a body surface point cloud contour at each breathing moment of each subject based on the body surface point cloud data within the respiratory cycle of each subject;

[0129] a data annotation module, configured to perform phase annotation on the body surface point cloud contours of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset;

[0130] A model training module is used to input the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model;

[0131] The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix;

[0132] The target area position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix.

[0133] This embodiment also provides a computer device, such as Figure 6 As shown, at least one processor 01 , at least one communication interface 02 , at least one memory 03 and at least one communication bus 04 .

[0134] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0135] Processor 01 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. Processor 01 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). Processor 01 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, processor 01 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, processor 01 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0136] In the embodiment of the present application, the memory 03 may include one or more readable storage media, which may be non-transitory. The memory 03 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices.

[0137] The memory 03 stores a computer program, and the processor 01 can call the program stored in the memory 03, and the program is used to execute the steps of the target area position prediction method based on body surface point cloud data described in Example 1 or 2.

[0138] The same or similar reference numerals correspond to the same or similar components;

[0139] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0140] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A target area position prediction method based on body surface point cloud data, characterized in that: include: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects; Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase were obtained to establish an in vivo point cloud dataset; Based on the body surface point cloud data within the respiratory cycle of each subject, a body surface point cloud contour at each respiratory moment of each subject is obtained; Based on the in-vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contours of each subject at each breathing moment to establish a body surface point cloud labeling dataset; Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix; Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current respiratory moment, the final predicted phase of the body surface point cloud contour at the current respiratory moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current respiratory moment is obtained.

2. The target area position prediction method based on body surface point cloud data according to claim 1, characterized in that: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contours and the in vivo target area position coordinates of each subject in each phase were obtained to establish an in vivo point cloud dataset, including: For each subject, the 4DCT reconstructed point cloud data is divided evenly according to the phase to obtain the subject's Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target area position coordinates in the body into one element , establish the patient's in vivo point cloud dataset; in, Indicates the subject's The reconstructed point cloud contour of the phase, Indicates the subject's The target position in the body of the reconstructed point cloud contour of the phase, Represents the subjects’ The position of the target area in the body of the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases.

3. The target area position prediction method based on body surface point cloud data according to claim 2, characterized in that: Based on the body surface point cloud data within each subject's respiratory cycle, the body surface point cloud contour of each subject at each respiratory moment is obtained, including: For each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject The surface point cloud contour of each breathing moment ,in, Indicates the subject's The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

4. The target area position prediction method based on body surface point cloud data according to claim 3, characterized in that: Based on the in vivo point cloud dataset, phase annotation is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, including: For any subject’s body surface point cloud contour at any breathing moment, the contours are compared with the body point cloud dataset of the subject. Align the reconstructed point cloud contours of each phase; After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases is obtained The average vector distance; Compare The size of the average vector distance is used, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment, thereby obtaining the body surface point cloud contour annotation data at the breathing moment; Traversing the body surface point cloud contour of the subject at each breathing moment to obtain body surface point cloud contour annotation data of the subject at each breathing moment; All subjects were traversed to obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and a body surface point cloud annotation dataset was established.

5. The target area position prediction method based on body surface point cloud data according to claim 4, characterized in that: For any subject's body surface point cloud contour at any breathing moment, use any of the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method to compare it with the subject's in vivo point cloud dataset. The reconstructed point cloud contours of the phases are aligned.

6. The target area position prediction method based on body surface point cloud data according to claim 4, characterized in that: After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases is obtained Average vector distances, including: For the aligned body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at any phase, calculate the vector distance between the corresponding point cloud pairs; Based on the vector distances of all point cloud pairs, the average vector distance between the body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at the phase is calculated; Traversal The reconstructed point cloud contour of the phase is obtained The average vector distance.

7. The target area position prediction method based on body surface point cloud data according to claim 6, characterized in that: The vector distance includes any one of Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance.

8. The target area position prediction method based on body surface point cloud data according to any one of claims 1 to 7, characterized in that: Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model, including: Constructing a body surface and body body point cloud association model, inputting a body surface point cloud annotation dataset and a body body point cloud dataset, wherein the body surface and body body point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model; Inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment; Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject; For each subject's body surface point cloud contour at each respiratory moment, select the reconstructed point cloud contour corresponding to the final predicted phase from the corresponding in vivo point cloud dataset, input the reconstructed point cloud contour together with the body surface point cloud contour at that respiratory moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of each subject's body surface point cloud contour at each respiratory moment; The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body point cloud association model.

9. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer connected in sequence.

10. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer; The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer; The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

11. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The first loss function is: Where, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates the The point cloud contour of the individual table belongs to True categories, Indicates the The point cloud contour of the individual table belongs to The predicted probability of the true class.

12. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The second loss function is: Where, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding true phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each respiratory moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

13. The target area position prediction method based on body surface point cloud data according to claim 1, characterized in that: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current respiratory moment, the final predicted phase of the body surface point cloud contour at the current respiratory moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current respiratory moment is obtained, including: In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

14. A target area position prediction system based on body surface point cloud data, based on the prediction method according to any one of claims 1 to 13, characterized in that: include: A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within the respiratory cycle of several subjects; The first data partitioning module is used to reconstruct the point cloud data of each subject based on the 4DCT, obtain the reconstructed point cloud contour and the position coordinates of the target area in the body of each subject in each phase, and establish an in vivo point cloud data set; A second data segmentation module is configured to obtain a body surface point cloud contour at each breathing moment of each subject based on the body surface point cloud data within the respiratory cycle of each subject; a data annotation module, configured to perform phase annotation on the body surface point cloud contours of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset; A model training module is used to input the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current respiratory moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current respiratory moment and the final target area transformation matrix; The target area position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix.

15. A computer device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the target area position prediction method based on body surface point cloud data according to any one of claims 1 to 13.

16. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the target area position prediction method based on body surface point cloud data as claimed in any one of claims 1 to 13 is implemented.

Images