Target region position prediction method and system based on body surface point cloud data

Abstract

The invention discloses a target position prediction method and system based on body surface point cloud data, and relates to the technical field of medical signal processing, and the method comprises the steps: obtaining 4DCT reconstruction point cloud data of a plurality of subjects and body surface point cloud data in a respiratory cycle, and building an in-vivo point cloud data set and a body surface point cloud labeling data set; training a body surface and in-vivo point cloud correlation model by using the body surface point cloud labeling data set and the in-vivo point cloud data set, inputting 4DCT reconstruction point cloud data of the patient and the body surface point cloud contour of the current breathing moment into the trained body surface and in-vivo point cloud correlation model, and obtaining a final prediction phase and a final target region transformation matrix; and in combination with the 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, obtaining a predicted position of the in-vivo target region on the body surface point cloud contour at the current breathing moment. According to the invention, the position of the target area can be accurately predicted on the body surface contour, the accuracy of radiotherapy irradiation is improved, and radiation hazards are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical signal processing, and more specifically, to a method and system for predicting target area position based on body surface point cloud data. Background Art

[0002] During radiotherapy, the position of the patient's target area will change to a certain extent due to the influence of respiratory movement. Accurate positioning and real-time tracking of the target area can help patients reduce radiation hazards during treatment. At present, there are several mainstream radiotherapy schemes for tracking the target area in the body. The first is the marker tracking scheme, which uses external cameras and surface markers for analysis, monitors the patient's surface movement in real time, and infers the internal lesions, that is, the position of the target area in the body through the surface markers. The disadvantage of this scheme is that it can track the surface movement in real time, but the position of the target area in the body is not accurate enough. In addition, this scheme depends on the high cooperation of the patient and the accuracy of the selection of surface markers. When the surface characteristics change significantly, such as surgery or swelling, the position of the target area in the body cannot be accurately inferred. The second is the ultrasonic surface and body association scheme, the principle of which is to use ultrasonic equipment to monitor the relative position of the body surface and the body in real time, and then associate them through image processing technology. This scheme has the problem of low resolution and contrast of ultrasonic images, and the transmission of ultrasonic waves in bones and air is limited, which limits its application in certain parts of the human body and has high technical requirements for operators. There are also some schemes that use high-precision equipment to generate in-body images for observation. However, due to the scene restrictions of radiotherapy, high-precision equipment is difficult to use in real time in clinical environments. In addition, high-precision equipment is expensive and technically complex, and the time period for acquiring and processing images is long, and it is impossible to generate in vivo images in real time. Therefore, it is urgent to propose a method that is efficient and accurate, has high scene adaptability, and requires low patient cooperation, so as to accurately associate the patient's in vivo target area position with the body surface contour, and achieve accurate positioning and real-time tracking of the in vivo target area. Summary of the invention

[0003] One of the purposes of the present invention is to provide a target area position prediction method based on body surface point cloud data to overcome the defect that the above-mentioned prior art cannot accurately predict the target area position on the body surface contour; the second purpose is to provide a target area position prediction system based on body surface point cloud data; the third purpose is to provide a computer device; the fourth purpose is to provide a readable storage medium.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows: The present invention provides a target area position prediction method based on body surface point cloud data, comprising: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects; Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target position coordinates of each subject in each phase are obtained to establish an in vivo point cloud data set; Based on the body surface point cloud data within the respiratory cycle of each subject, the body surface point cloud contour of each subject at each respiratory moment is obtained; Based on the in vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud labeling dataset; Inputting the body surface point cloud annotation data set and the body body point cloud data set into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current breathing moment and the final target area transformation matrix; Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained.

[0005] Preferably, based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour of each subject in each phase and the in vivo target area position coordinates are obtained to establish an in vivo point cloud data set, including: For each subject, the 4DCT reconstructed point cloud data is divided evenly according to the phase to obtain the subject's Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target position coordinates in the body into one element , establish the patient's in vivo point cloud dataset; in, Indicates the subject’s The reconstructed point cloud contour of the phase, Indicates the subject’s The target position in the body of the reconstructed point cloud contour of the phase, Respectively represent the subjects’ The position of the target area in the body based on the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases.

[0006] Preferably, based on the body surface point cloud data in the respiratory cycle of each subject, obtaining the body surface point cloud contour of each subject at each respiratory moment includes: For each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject The surface point cloud contour of each breathing moment ,in, Indicates the subject’s The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

[0007] Preferably, based on the in vivo point cloud dataset, phase annotation is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, including: For any subject’s body surface point cloud contour at any breathing moment, the contours are compared with the body point cloud dataset of the subject. Align the reconstructed point cloud contours of each phase; After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases corresponds to the obtained The average vector distance; Compare The size of the average vector distance is determined, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment to obtain the body surface point cloud contour annotation data at the breathing moment; Traversing the body surface point cloud contour of the subject at each breathing moment to obtain body surface point cloud contour annotation data of the subject at each breathing moment; All subjects are traversed to obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and a body surface point cloud annotation dataset is established.

[0008] Preferably, for the body surface point cloud contour of any subject at any breathing moment, the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method are used to compare the contour of the body surface point cloud data set of the subject with the contour of the body surface point cloud data set of the subject. The reconstructed point cloud contours of the phases are aligned.

[0009] Preferably, after alignment, the body surface point cloud contour and The average vector distance of the reconstructed point cloud contour of the phases corresponds to the obtained Mean vector distances, including: For the aligned body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at any phase, calculate the vector distance between the corresponding point cloud pairs; Based on the vector distances of all point cloud pairs, the average vector distance between the body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at the phase is calculated; Traversal The reconstructed point cloud contour of the phase is obtained The average vector distance.

[0010] Preferably, the vector distance includes any one of Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance.

[0011] Preferably, the body surface point cloud annotation dataset and the body body point cloud dataset are input into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model, including: Constructing a body surface and body internal point cloud association model, inputting a body surface point cloud annotation data set and a body internal point cloud data set, wherein the body surface and body internal point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model; Inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment; Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject; For the body surface point cloud contour at each breathing moment of each subject, in the corresponding in vivo point cloud data set, select the reconstructed point cloud contour corresponding to the final predicted phase, and input it together with the body surface point cloud contour at the breathing moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of the body surface point cloud contour at each breathing moment of each subject; The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body body point cloud association model.

[0012] Preferably, the phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer which are connected in sequence.

[0013] Preferably, the target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer; The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer; The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

[0014] Preferably, the first loss function is:

[0015] In the formula, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates The point cloud contour of the individual table belongs to real categories, Indicates The point cloud contour of the individual table belongs to The predicted probability of the true class.

[0016] Preferably, the second loss function is:

[0017] In the formula, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

[0018] Preferably, based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained, including: In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

[0019] The present invention also provides a target area position prediction system based on body surface point cloud data, based on the above prediction method, comprising: A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of several subjects; The first data division module is used to obtain the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase based on the 4DCT reconstructed point cloud data of each subject, and establish an in vivo point cloud data set; A second data division module is used to obtain a body surface point cloud contour of each subject at each breathing moment based on the body surface point cloud data within the breathing cycle of each subject; A data annotation module, used to perform phase annotation on the body surface point cloud contour of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset; A model training module is used to input the body surface point cloud annotation data set and the body body point cloud data set into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current breathing moment and the final target area transformation matrix; The target position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target transformation matrix.

[0020] The present invention also provides a computer device, comprising a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the aforementioned target area position prediction method based on body surface point cloud data.

[0021] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step of the aforementioned target area position prediction method based on body surface point cloud data is implemented.

[0022] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: The present invention first obtains 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of a large number of subjects, divides the 4DCT reconstructed point cloud data according to phase, obtains the reconstructed point cloud contour and the in vivo target area position coordinates of each phase, and establishes an in vivo point cloud data set; divides the body surface point cloud data within the respiratory cycle according to the respiratory moment, and obtains the body surface point cloud contour at each respiratory moment; then, for the body surface point cloud contour at each respiratory moment, searches for the nearest reconstructed point cloud contour, and uses the corresponding phase to phase-label the body surface point cloud contour at each respiratory moment, and establishes a body surface point cloud labeled data set; the phase labeling focuses on the distance difference feature between the body surface point cloud contour and the reconstructed point cloud contour; then, the body surface point cloud contour is labeled. The cloud annotation dataset and the in vivo point cloud dataset are input into the constructed surface and in vivo point cloud association model for optimization training to obtain a trained surface and in vivo point cloud association model; finally, the patient's 4DCT reconstructed point cloud data and the surface point cloud contour at the current breathing moment are obtained and input into the trained surface and in vivo point cloud association model, which can further capture the global distribution characteristics and local distribution characteristics between the surface point cloud contour and the reconstructed point cloud contour, predict the reconstructed point cloud contour that is most similar to the surface point cloud contour, output the final predicted phase and the final target area transformation matrix, and obtain the predicted position of the in vivo target area on the surface point cloud contour at the current breathing moment by fusing the in vivo target area position coordinates of the most similar reconstructed point cloud contour with the final target area transformation matrix. The present invention only needs to perform 4DCT reconstruction once before radiotherapy to obtain 4DCT reconstructed point cloud data, thus reducing radiation hazards. Afterwards, the patient is positioned to obtain the body surface point cloud contour at the current breathing moment in real time, which are input into the trained body surface and body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The body target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the body target area on the body surface point cloud contour at the current breathing moment, thereby achieving accurate prediction of the target area position on the body surface contour and improving the accuracy of radiotherapy irradiation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a target area position prediction method based on body surface point cloud data described in Example 1; Figure 2This is a flow chart of establishing a body surface point cloud annotation dataset as described in Example 2; Figure 3 This is a schematic diagram of the structure of the phase prediction sub-model described in Example 2; Figure 4 It is a schematic diagram of the structure of the target area transformation matrix calculation sub-model described in Example 2; Figure 5 This is a schematic diagram of the structure of a target area position prediction system based on body surface point cloud data described in Example 3; Figure 6 This is a schematic diagram of the structure of a computer device described in Example 3. DETAILED DESCRIPTION

[0024] The drawings are for illustrative purposes only and should not be construed as limiting the present patent; In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0025] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0026] Example 1 This embodiment provides a target area position prediction method based on body surface point cloud data, such as Figure 1 As shown, including: S1: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects; S2: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target position coordinates of each subject in each phase are obtained to establish an in vivo point cloud data set; S3: based on the body surface point cloud data of each subject in the respiratory cycle, obtaining the body surface point cloud contour of each subject at each breathing moment; S4: Based on the in vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud labeling dataset; S5: inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; S6: Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the body surface point cloud contour at the current breathing moment of the patient and the final target area transformation matrix; S7: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained.

[0027] In the specific implementation process, this embodiment first obtains 4DCT reconstructed point cloud data and body surface point cloud data within the respiratory cycle of a large number of subjects, divides the 4DCT reconstructed point cloud data according to phase, obtains the reconstructed point cloud contour and the in vivo target position coordinates of each phase, and establishes an in vivo point cloud data set; divides the body surface point cloud data within the respiratory cycle according to the respiratory moment, and obtains the body surface point cloud contour at each respiratory moment; then, searches for the nearest reconstructed point cloud contour for the body surface point cloud contour at each respiratory moment, and uses the corresponding phase to phase-label the body surface point cloud contour at each respiratory moment to establish a body surface point cloud labeling data set; the phase labeling focuses on the distance difference feature between the body surface point cloud contour and the reconstructed point cloud contour; then , the surface point cloud annotation dataset and the in-vivo point cloud dataset are input into the constructed surface and in-vivo point cloud association model for optimization training to obtain a trained surface and in-vivo point cloud association model; finally, the patient's 4DCT reconstructed point cloud data and the surface point cloud contour at the current breathing moment are obtained and input into the trained surface and in-vivo point cloud association model, which can further capture the global distribution characteristics and local distribution characteristics between the surface point cloud contour and the reconstructed point cloud contour, predict the reconstructed point cloud contour that is most similar to the surface point cloud contour, output the final predicted phase and the final target area transformation matrix, and obtain the predicted position of the in-vivo target area on the surface point cloud contour at the current breathing moment by fusing the in-vivo target area position coordinates of the most similar reconstructed point cloud contour with the final target area transformation matrix. In this embodiment, only one 4DCT reconstruction is required before radiotherapy to obtain 4DCT reconstructed point cloud data, which reduces radiation hazards. After that, the patient is positioned to obtain the body surface point cloud contour at the current breathing moment in real time, and the contours are input into the trained body surface and body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The body target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the body target area on the body surface point cloud contour at the current breathing moment, thereby achieving accurate prediction of the target area position on the body surface contour and improving the accuracy of radiotherapy irradiation.

[0028] Example 2 This embodiment provides a target area position prediction method based on body surface point cloud data, including: S1: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects; It should be noted that 4DCT (Four-Dimensional Computed Tomography) reconstruction data is obtained in the CT scanning room. 4DCT reconstruction technology is an imaging technology that can capture and reconstruct anatomical structure information that changes over time. It obtains a series of three-dimensional images by continuously scanning the patient during multiple respiratory cycles. Since 4DCT reconstruction technology generates images by emitting X-rays to the body and capturing the signals after they penetrate the body, it has a certain amount of radiation to the human body. Therefore, the number of CT scans needs to be minimized during radiotherapy to reduce radiation hazards. It should be noted that after the subject is positioned for radiotherapy, the 3D camera is used to collect body surface point cloud data within at least one breathing cycle. Through subsequent steps, it is not necessary to perform a CT scan every time radiotherapy is performed, and then associate it with the body surface point cloud data, reducing radiation hazards. It is understandable that the several subjects include subjects of different ages, treatment parts, and genders, so that the 4DCT reconstructed point cloud data and the body surface point cloud data within the respiratory cycle are sufficiently rich to ensure the accuracy of the subsequently trained body surface and body point cloud association model.

[0029] S2: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target position coordinates of each subject in each phase are obtained to establish an in vivo point cloud data set; It can be understood that for each subject, the 4DCT reconstructed point cloud data is evenly divided according to the phase to obtain the subject Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target position coordinates in the body into one element , establish the patient's in vivo point cloud dataset; in, Indicates the subject’s The reconstructed point cloud contour of the phase, Indicates the subject’s The target position in the body of the reconstructed point cloud contour of the phase, Respectively represent the subjects’ The position of the target area in the body based on the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases; in this embodiment, .

[0030] S3: based on the body surface point cloud data of each subject in the respiratory cycle, obtaining the body surface point cloud contour of each subject at each breathing moment; It can be understood that, for each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject's The surface point cloud contour of each breathing moment ,in, Indicates the subject’s The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

[0031] S4: Based on the in vivo point cloud dataset, phase annotate the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, such as Figure 2 As shown, including: S41: For any subject’s body surface point cloud contour at any breathing moment, compare it with the body point cloud dataset of the subject Align the reconstructed point cloud contours of each phase; S42: After alignment, calculate the body surface point cloud contour and The average vector distance of the reconstructed point cloud contour of the phases corresponds to the obtained The average vector distance; S43: Comparison The size of the average vector distance is determined, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment to obtain the body surface point cloud contour annotation data at the breathing moment; S44: repeating steps S41-S43, traversing the body surface point cloud contour at each breathing moment, and obtaining the body surface point cloud contour annotation data of the subject at each breathing moment; S45: Repeat steps S41-S44, traverse each subject, obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and establish a body surface point cloud annotation data set.

[0032] In this embodiment, LabelFusion is used to annotate the true phase of the body surface point cloud contour.

[0033] It should be noted that in step S41, there is no limitation on the method for aligning the body surface point cloud contour with the reconstructed point cloud contour, such as the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method, etc. In practical applications, a suitable alignment method is selected according to specific needs and characteristics of the current point cloud data.

[0034] It should be noted that in step S42, the distance metric for calculating the average vector distance between the body surface point cloud contour at any breathing moment and the reconstructed point cloud contour in the body point cloud data set is not limited, such as Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance, etc. After calculating the distances between all vector pairs, the average is taken to obtain the average vector distance.

[0035] S5: Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model, including: S51: constructing a body surface and body body point cloud association model, inputting a body surface point cloud annotation dataset and a body body point cloud dataset, wherein the body surface and body body point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model; S52: inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment; It should be noted that if Figure 3 As shown, the phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer which are connected in sequence.

[0036] The dimension of the body surface point cloud annotation dataset is ( ),in Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, represents the coordinate dimension, Represents the point feature dimension; after the first sampling grouping, the dimension obtained is ( ) of the first point set; after the first point set is extracted by the first PointNet layer, the dimension is ( ), where Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Represents the number of features; the dimension obtained by the second sampling grouping layer is ( ) of the second point set, input the second PointNet layer to obtain a dimension of ( ), where represents the number of points in the second point set, Represents the point feature dimension in the second point set group; the second feature matrix is ​​input to the self-attention mechanism layer. The first PointNet layer and the second PointNet layer can extract the global features of the point cloud. The self-attention mechanism layer can extract the local features of the point cloud and increase the attention to the correlation of local features. Finally, it is input to the fully connected layer for classification prediction and outputs the predicted phase of the body surface point cloud contour.

[0037] S53: Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject; The first loss function is:

[0038] In the formula, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates The point cloud contour of the individual table belongs to real categories, Indicates The point cloud contour of the individual table belongs to The predicted probability of the true class.

[0039] S54: for the body surface point cloud contour at each breathing moment of each subject, in the corresponding in vivo point cloud data set, select the reconstructed point cloud contour corresponding to the final predicted phase, and input it together with the body surface point cloud contour at the breathing moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of the body surface point cloud contour at each breathing moment of each subject; It should be noted that if Figure 4 As shown, the target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer; The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer; The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

[0040] The second loss function is:

[0041] In the formula, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

[0042] It should be noted that the trained phase prediction sub-model outputs the final predicted phase Take this as an example to explain, then in the subject’s in vivo point cloud dataset, select Reconstructed point cloud contours of phases As the first input ; Select the body surface point cloud contour at the breathing moment as the second input ; The first DGCNN layer and the second DGCNN layer are set in parallel, with the same structure and shared parameters. Input the first DGCNN layer, Input the second DGCNN layer, extract the high-dimensional feature vector matrix, and correspondingly input the first Transformer layer and the second Transformer layer to extract the global feature correlation to obtain the first residual term and the second residual ; The first residual term and the second residual Input the Pointer layer for Pointer Generation. In order to avoid the occurrence of non-differentiable situations, this embodiment uses the probability method for Pointer Generation. The generation formula is: . Input the generated pointer into the SVD layer to obtain the final target area transformation matrix.

[0043] S55: The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body body point cloud association model.

[0044] S6: Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the body surface point cloud contour at the current breathing moment of the patient and the final target area transformation matrix; S7: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained, including: In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

[0045] In actual use, this embodiment only needs to perform a 4DCT reconstruction before radiotherapy to obtain 4DCT reconstructed point cloud data, which reduces radiation hazards. After that, the patient is positioned to obtain the surface point cloud contour at the current breathing moment in real time, and the contours are input into the trained surface and body point cloud association model to obtain the final predicted phase and the final target area transformation matrix. The in vivo target area position coordinates of the reconstructed point cloud contour with the same phase as the final predicted phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the surface point cloud contour at the current breathing moment, thereby achieving accurate prediction of the target area position on the surface contour and improving the accuracy of radiotherapy irradiation.

[0046] Example 3 This embodiment provides a target area position prediction system based on body surface point cloud data, based on the prediction method described in embodiment 1 or 2, such as Figure 5 As shown, including: A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of several subjects; The first data division module is used to obtain the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase based on the 4DCT reconstructed point cloud data of each subject, and establish an in vivo point cloud data set; A second data division module is used to obtain a body surface point cloud contour of each subject at each breathing moment based on the body surface point cloud data within the breathing cycle of each subject; A data annotation module, used to perform phase annotation on the body surface point cloud contour of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset; A model training module is used to input the body surface point cloud annotation data set and the body body point cloud data set into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current breathing moment and the final target area transformation matrix; The target position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target transformation matrix.

[0047] This embodiment also provides a computer device, such as Figure 6 As shown, at least one processor 01 , at least one communication interface 02 , at least one memory 03 and at least one communication bus 04 .

[0048] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0049] Processor 01 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. Processor 01 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). Processor 01 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, processor 01 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, processor 01 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0050] In the embodiment of the present application, the memory 03 may include one or more readable storage media, which may be non-transitory. The memory 03 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices.

[0051] The memory 03 stores a computer program, and the processor 01 can call the program stored in the memory 03, and the program is used to execute the steps of the target area position prediction method based on body surface point cloud data described in Example 1 or 2.

[0052] The same or similar reference numerals correspond to the same or similar components; The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent; Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A target area position prediction method based on body surface point cloud data, characterized in that: include: Acquire 4DCT reconstructed point cloud data and body surface point cloud data during the respiratory cycle of several subjects; Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target position coordinates of each subject in each phase are obtained to establish an in vivo point cloud data set; Based on the body surface point cloud data within the respiratory cycle of each subject, the body surface point cloud contour of each subject at each respiratory moment is obtained; Based on the in vivo point cloud dataset, phase-labeling is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud labeling dataset; Inputting the body surface point cloud annotation data set and the body body point cloud data set into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; Obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current breathing moment and the final target area transformation matrix; Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained.

2. The target area position prediction method based on body surface point cloud data according to claim 1, characterized in that: Based on the 4DCT reconstructed point cloud data of each subject, the reconstructed point cloud contour and the in vivo target position coordinates of each subject in each phase are obtained, and an in vivo point cloud data set is established, including: For each subject, the 4DCT reconstructed point cloud data is divided evenly according to the phase to obtain the subject's Reconstructed point cloud contours of phases , and the in vivo target position coordinates of the reconstructed point cloud contours of each phase ; Combine the reconstructed point cloud contour of the subject's corresponding phase and the target position coordinates in the body into one element , establish the patient's in vivo point cloud dataset; in, Indicates the subject’s The reconstructed point cloud contour of the phase, Indicates the subject’s The target position in the body of the reconstructed point cloud contour of the phase, Respectively represent the subjects’ The position of the target area in the body based on the reconstructed point cloud contour of the phase coordinate, Coordinates and coordinate, , Indicates the total number of phases.

3. The target area position prediction method based on body surface point cloud data according to claim 2, characterized in that: Based on the body surface point cloud data of each subject in the respiratory cycle, the body surface point cloud contour of each subject at each breathing moment is obtained, including: For each subject, the body surface point cloud data within the respiratory cycle is divided according to the respiratory moment to obtain the subject The surface point cloud contour of each breathing moment ,in, Indicates the subject’s The surface point cloud contour of each breathing moment, , Indicates the total number of breathing moments.

4. The target area position prediction method based on body surface point cloud data according to claim 3, characterized in that: Based on the in vivo point cloud dataset, phase annotation is performed on the body surface point cloud contour of each subject at each breathing moment to establish a body surface point cloud annotation dataset, including: For any subject’s body surface point cloud contour at any breathing moment, the contours are compared with the body point cloud dataset of the subject. Align the reconstructed point cloud contours of each phase; After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases corresponds to the obtained The average vector distance; Compare The size of the average vector distance is determined, and the phase of the reconstructed point cloud contour corresponding to the minimum average vector distance is used as the true phase of the body surface point cloud contour at the breathing moment to obtain the body surface point cloud contour annotation data at the breathing moment; Traversing the body surface point cloud contour of the subject at each breathing moment, and obtaining the body surface point cloud contour annotation data of the subject at each breathing moment; All subjects are traversed to obtain the body surface point cloud contour annotation data of all subjects at each breathing moment, and a body surface point cloud annotation dataset is established.

5. The target area position prediction method based on body surface point cloud data according to claim 4, characterized in that: For any subject's body surface point cloud contour at any breathing moment, use any of the nearest point iteration algorithm, Umeyama algorithm, Horn quaternion closure solution algorithm, Marta's iterative least squares solution or moving least squares surface approximation method to compare it with the in vivo point cloud dataset of the subject. The reconstructed point cloud contours of the phases are aligned.

6. The target area position prediction method based on body surface point cloud data according to claim 4, characterized in that: After alignment, the surface point cloud contours and The average vector distance of the reconstructed point cloud contour of the phases corresponds to the obtained Mean vector distances, including: For the aligned body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at any phase, calculate the vector distance between the corresponding point cloud pairs; Based on the vector distances of all point cloud pairs, the average vector distance between the body surface point cloud contour at the respiratory moment and the reconstructed point cloud contour at the phase is calculated; Traversal The reconstructed point cloud contour of the phase is obtained The average vector distance.

7. The target area position prediction method based on body surface point cloud data according to claim 6, characterized in that: The vector distance includes any one of Euclidean distance, Manhattan distance, Chebyshev distance or standardized Euclidean distance.

8. The target area position prediction method based on body surface point cloud data according to any one of claims 1 to 7, characterized in that: Inputting the body surface point cloud annotation dataset and the body body point cloud dataset into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model, including: Constructing a body surface and body internal point cloud association model, inputting a body surface point cloud annotation data set and a body internal point cloud data set, wherein the body surface and body internal point cloud association model includes a phase prediction sub-model and a target area transformation matrix calculation sub-model; Inputting the body surface point cloud contour of each subject at each breathing moment into the phase prediction sub-model to obtain the predicted phase of the body surface point cloud contour of each subject at each breathing moment; Based on the predicted phase and the true phase of the body surface point cloud contour at each breathing moment of each subject, a first loss function is established to optimize the phase prediction sub-model to obtain a trained phase prediction sub-model; the trained phase prediction sub-model outputs the final predicted phase of the body surface point cloud contour at each breathing moment of each subject; For the body surface point cloud contour at each breathing moment of each subject, in the corresponding in vivo point cloud data set, select the reconstructed point cloud contour corresponding to the final predicted phase, and input it together with the body surface point cloud contour at the breathing moment into the target area transformation matrix calculation submodel, establish a second loss function to optimize the target area transformation matrix calculation submodel, and obtain a trained target area transformation matrix calculation submodel; the trained target area transformation matrix calculation submodel outputs the final target area transformation matrix of the body surface point cloud contour at each breathing moment of each subject; The trained phase prediction sub-model and the trained target area transformation matrix calculation sub-model constitute a trained body surface and body body point cloud association model.

9. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The phase prediction sub-model includes a first sampling grouping layer, a first PointNet layer, a second sampling grouping layer, a second PointNet layer, a self-attention mechanism layer and a fully connected layer connected in sequence.

10. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The target area transformation matrix calculation submodel includes a first DGCNN layer, a second DGCNN layer, a first Transformer layer, a second Transformer layer, a Pointer layer and an SVD layer; The output end of the first DGCNN layer is connected to the input end of the first Transformer layer, and the output end of the first Transformer layer is connected to the first input end of the Pointer layer; The output end of the second DGCNN layer is connected to the input end of the second Transformer layer, the output end of the second Transformer layer is connected to the second input end of the Pointer layer, and the output end of the Pointer layer is connected to the input end of the SVD layer.

11. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The first loss function is: In the formula, represents the first loss function, Represents the number of body surface point cloud contours in each subject’s body surface point cloud annotation dataset, Represents the number of categories of body surface point cloud contours in the body surface point cloud annotation dataset, Indicates The point cloud contour of the individual table belongs to real categories, Indicates The point cloud contour of the individual table belongs to The predicted probability of the true class.

12. The target area position prediction method based on body surface point cloud data according to claim 8, characterized in that: The second loss function is: In the formula, represents the second loss function, Represents the transposed matrix of the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the rotation matrix that transforms the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the identity matrix, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding final predicted phase, Represents the translation matrix from the body surface point cloud contour at each breathing moment to the reconstructed point cloud contour corresponding to the corresponding real phase, represents the regularization coefficient, Represents training parameters.

13. The target area position prediction method based on body surface point cloud data according to claim 1, characterized in that: Based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target area transformation matrix, the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment is obtained, including: In the 4DCT reconstructed point cloud data of the patient, a reconstructed point cloud contour having the same phase as the final predicted phase of the patient's body surface point cloud contour at the current breathing moment is selected, and the in vivo target area position coordinates of the reconstructed point cloud contour having the same phase are fused with the final target area transformation matrix to obtain the predicted position of the in vivo target area on the body surface point cloud contour at the current breathing moment.

14. A target area position prediction system based on body surface point cloud data, based on the prediction method according to any one of claims 1 to 13, characterized in that: include: A point cloud data acquisition module is used to acquire 4DCT reconstructed point cloud data and body surface point cloud data within a respiratory cycle of several subjects; The first data division module is used to obtain the reconstructed point cloud contour and the in vivo target area position coordinates of each subject in each phase based on the 4DCT reconstructed point cloud data of each subject, and establish an in vivo point cloud data set; A second data division module is used to obtain a body surface point cloud contour of each subject at each breathing moment based on the body surface point cloud data within the breathing cycle of each subject; A data annotation module, used to perform phase annotation on the body surface point cloud contour of each subject at each breathing moment based on the in-vivo point cloud dataset, and establish a body surface point cloud annotation dataset; A model training module is used to input the body surface point cloud annotation data set and the body body point cloud data set into the constructed body surface and body body point cloud association model for optimization training to obtain a trained body surface and body body point cloud association model; The model inference module is used to obtain the patient's 4DCT reconstructed point cloud data and the body surface point cloud contour at the current breathing moment, input them into the trained body surface and body point cloud association model, and obtain the final predicted phase of the patient's body surface point cloud contour at the current breathing moment and the final target area transformation matrix; The target position prediction module is used to obtain the predicted position of the target area in the body on the body surface point cloud contour at the current breathing moment based on the patient's 4DCT reconstructed point cloud data, the body surface point cloud contour at the current breathing moment, the final predicted phase of the body surface point cloud contour at the current breathing moment, and the final target transformation matrix.

15. A computer device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the target area position prediction method based on body surface point cloud data as claimed in any one of claims 1 to 13.

16. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the target area position prediction method based on body surface point cloud data as claimed in any one of claims 1 to 13 is implemented.

Images