Tumor tracking method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and medical treatment, and discloses a tumor tracking method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a 4D CT image of a target user; and generating a 2D DRR image and a projection image only containing a tumor and corresponding to the 2D DRR image according to the 4D CT image, and finally performing model training on the initial tumor tracking network based on the 2D DRR image and the projection image only containing the tumor and corresponding to the 2D DRR image to obtain a trained tumor tracking network. The tumor tracking network is trained through the 2D DRR image and the projection image only containing the tumor and corresponding to the 2D DRR image, the trained tumor tracking network can achieve accurate tumor positioning precision, the robustness and generalization ability of the tumor tracking network are obviously improved in the complex clinical environment with noise interference, the tracking stability is obviously improved, and the tumor positioning accuracy is improved. And the capturing accuracy of the three-dimensional space position and morphological characteristics of the tumor is enhanced.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and medical technology, and in particular to a tumor tracking method, device, computer equipment and storage medium. Background Art

[0002] Respiratory motion management methods in clinical radiotherapy mainly include the internal target volume (I TV) method, abdominal compression and breath-holding techniques, respiratory gating, and real-time tumor tracking. The I TV method covers the tumor motion trajectory by expanding the target area, but this method may not only increase the radiation dose to surrounding healthy tissues, but also fail to fully cover the target area when the tumor drifts. In addition, abdominal compression and breath-holding techniques are rarely used due to patient comfort issues, and respiratory gating relies heavily on the patient to maintain respiratory consistency.

[0003] In contrast, a more ideal solution is real-time tumor tracking, which aims to monitor tumor motion in real time during radiotherapy, compensate for motion errors by adjusting the position of the irradiation field or treatment bed, and ensure that the relative position between the center of the irradiation field and the target area remains stable. This not only allows patients to receive treatment in a natural breathing state, but also ensures target coverage and reduces the need for motion margin for the clinical target volume (CTV). Traditional tumor tracking relies on the implantation of fiducial markers, but this method often leads to complications and imaging artifacts. To overcome these problems, real-time tumor tracking technology based on X-ray projection provides a non-invasive solution that combines real-time X-ray imaging with advanced image processing technology. Early research in this field mainly relied on traditional image processing techniques such as template matching and traditional machine learning algorithms such as support vector machines (SVM) and random forests. However, these methods often face challenges in achieving high accuracy and real-time performance, especially when dealing with complex and diverse clinical situations.

[0004] In recent years, the rapid development of artificial intelligence, especially deep learning, has revolutionized tumor tracking under X-ray projection. Deep learning methods utilize complex neural network architectures and show excellent ability to automatically extract significant features from medical imaging data, thereby achieving accurate tumor localization.

[0005] During radiotherapy of thoracic and abdominal tumors, the position and morphology of the tumor are constantly changing due to respiratory motion and other physiological and pathological processes. This will cause the treatment beam to deviate from the intended target, causing unnecessary radiation-induced damage to normal tissues, and may reduce the tumor control rate, thereby compromising the overall treatment efficacy and increasing the uncertainty of prognosis. The existing studies have poor performance in tumor tracking stability, transferability, and adaptability to complex clinical environments. Summary of the invention

[0006] Based on this, it is necessary to address the technical problem of poor tumor tracking and identification effect in the prior art and propose a tumor tracking method, device, computer equipment and storage medium.

[0007] In a first aspect, a tumor tracking method is provided, the method comprising:

[0008] Acquire a 4D CT image of the target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0009] Extracting the 4D CT image to obtain a motion image and various reference images, and outlining the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0010] Based on the motion image, the reference image and the tumor segmentation mask image, generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image;

[0011] Based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of the Transformer model and the CNN network.

[0012] In a second aspect, a tumor tracking device is provided, the device comprising:

[0013] An acquisition module, used to acquire a 4D CT image of a target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0014] An extraction module is used to extract the 4D CT image to obtain a motion image and various reference images, and to delineate the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0015] A generating module, configured to generate a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image;

[0016] A training module is used to perform model training on an initial tumor tracking network based on the 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image, so as to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on a combination of Transformer and CNN.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned tumor tracking method when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned tumor tracking method are implemented.

[0019] The tumor tracking method proposed in the present invention obtains a 4D CT image of a target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle, and then extracts the 4D CT image to obtain a motion image and each reference image, and delineates the tumor target area in the motion image to obtain a tumor segmentation mask image, thereby generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2DDRR image based on the motion image, the reference image and the tumor segmentation mask image, and finally, based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is model trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on a combination of a Transformer model and a CNN network. The present invention trains a tumor tracking network through 2D DRR images and projection images containing only tumors corresponding to 2D DRR images. The trained tumor tracking network can achieve accurate tumor positioning accuracy, and significantly improves its robustness and generalization ability in a complex clinical environment with noise interference, significantly improves tracking stability, and enhances the accuracy of capturing the three-dimensional spatial position and morphological characteristics of the tumor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] in:

[0022] Figure 1 A diagram showing an application environment of a tumor tracking method in an embodiment;

[0023] Figure 2 is a flow chart of a tumor tracking method in one embodiment;

[0024] Figure 3 is an overall framework of a tumor tracking method in an embodiment;

[0025] Figure 4 A schematic diagram of tumor tracking results at different respiratory stages of a tumor tracking method in one embodiment;

[0026] Figure 5 A comparison diagram of the results of a tumor tracking method in an embodiment;

[0027] Figure 6 A comparison diagram of the effects of a tumor tracking method in one embodiment;

[0028] Figure 7 is a structural block diagram of a tumor tracking device in one embodiment;

[0029] Figure 8 is a structural block diagram of a computer device in one embodiment;

[0030] Fig. 9 It is a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The tumor tracking method provided by the embodiment of the present invention can be applied in Figure 1In an application environment, the client 110 communicates with the server 120 through a network. The server 120 can receive a 4D CT image of a target user through the client 110, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle, and then extract the 4D CT image to obtain a motion image and each reference image, and delineate the tumor target area of ​​the motion image to obtain a tumor segmentation mask image, so that the server 120 generates a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image. Finally, the server 120 performs model training on the initial tumor tracking network based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on a combination of a Transformer model and a CNN network. The present invention trains a tumor tracking network through 2D DRR images and projection images containing only tumors corresponding to the 2D DRR images. The trained tumor tracking network can achieve accurate tumor positioning accuracy, and significantly improves its robustness and generalization ability in a complex clinical environment with noise interference, significantly improves tracking stability, and enhances the accuracy of capturing the three-dimensional spatial position and morphological characteristics of the tumor. Among them, the client 110 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server 120 can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0035] See also Figure 2 As shown, Figure 2 A schematic flow chart of a tumor tracking method provided by an embodiment of the present invention includes the following steps:

[0036] Step S101: Acquire a 4D CT image of a target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0037] Among them, the target user may be a patient.

[0038] In one embodiment, first, a 4D CT image of the patient before treatment is acquired, where the 4D CT image includes 3D CT images of 10 respiratory phases in a complete respiratory cycle.

[0039] Step S102: extracting the 4D CT image to obtain a motion image and various reference images, and outlining the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0040] The motion image refers to a CT image of an end-expiratory phase in a 4D CT image, and each of the reference images refers to a CT image other than the motion image in the 4D CT image.

[0041] In one embodiment, the CT image at the end-expiratory phase is used as the motion image (M CT ), and the remaining 9 respiratory phase CT images are used as reference images (F CT ). And in the end-expiratory phase CT image (M CT ) is to delineate the tumor target area on the motion image and obtain the tumor segmentation mask image (M seg ).

[0042] Step S103: generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image;

[0043] In this embodiment, a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image are generated by amplifying and fusing the motion image, the reference image and the tumor segmentation mask image.

[0044] In one embodiment, step S103 includes:

[0045] Step S1031: generating a non-rigid deformation based on the motion image and each reference image;

[0046] In this embodiment, the intensity-based image registration method aligns the moving image to the reference image in sequence to generate a phase-to-phase deformation vector field; based on the combination of each phase-to-phase deformation vector field, a non-rigid deformation is obtained. As an example, considering the scarcity of real X-ray image data in radiotherapy and the limitation of patient radiation dose, a complex hybrid data augmentation method is used to alleviate this problem. The specific steps are as follows: First, the moving image (M CT ) are aligned to the reference image F CT i(i∈[1,9]), generates the phase-to-phase deformation vector field At the same time, in order to increase the diversity of deformation, any two deformation fields are randomly combined to generate non-rigid deformations suitable for more clinical respiratory scenarios. Non-rigid deformation It is expressed as follows:

[0047]

[0048] Among them, α is a random number in the interval (0,1). In this way, a large number of deformation vector fields can be generated to effectively simulate the influence of respiratory motion on tumor position changes during radiotherapy.

[0049] Step S1032: Processing the motion image and each reference image based on 3D TPS technology to generate a deformation field;

[0050] In this embodiment, during the implementation of actual clinical radiotherapy planning, the deviation of the tumor position is not only affected by respiratory motion, but also by the changes of the tumor itself during the treatment process. Therefore, a three-dimensional thin plate spline (3DTPS) method is used to simulate the small changes of the tumor in a specific respiratory phase: First, N control point (cp) pairs are selected, p i =(x i ,y i ,z i ) and q i =(x i ′,y i ′,z i ′)(i=1,2,…,N),p i represents the control points in the original image, q i Denotes the corresponding control points in the warped enhanced image, where the original image refers to the motion image and the warped enhanced image refers to each reference image. Then a random displacement is applied to each control point, and these displacement vectors create a dense full-resolution deformation field through 3D TPS. In this experiment, the number of control points N is randomly selected between 20 and 60, and the random displacement range of each control point is 0-30 mm. The core of this process is to establish control point pairs (p i and q i ), by minimizing the loss function To find it. Its specific calculation is as follows:

[0051]

[0052] refers to the deformation field, and the represents the distance between the corresponding control point and the mapping point, minimizing it will encourage q i Close to p i , but this may generate physically unrealistic non-smooth deformation fields. So using smooth variables And weight parameter λ to control the rigidity of deformation. The specific calculation is as follows:

[0053]

[0054] Step S1033: determining a mixed deformation field based on the non-rigid deformation and the deformation field;

[0055] In this embodiment, in order to further increase the diversity of training samples, instead of directly superimposing the inter-phase and intra-phase deformations, the weights of the respective deformations are randomly set and combined into a mixed deformation field. In this way, multiple mixed deformation fields representing different breathing stages and within a specific phase can be generated.

[0056] The mixed deformation field is expressed as follows:

[0057]

[0058] in, refers to the mixed deformation field, w1 and w2 are random numbers uniformly distributed in the interval (0, 1), refers to non-rigid deformation, It refers to the deformation field.

[0059] Step S1034: generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the mixed deformation field, the motion image and the tumor segmentation mask image.

[0060] In this embodiment, the blended deformation field is applied to the motion image M CT and its tumor segmentation mask image M seg Perform spatial transformation to generate a large number of synthetic CT images (S CT ) and tumor segmentation label (S seg ). For a large number of synthetic CT images (S CT ) and tumor segmentation label (S seg ), a large number of 2D DRR images and their corresponding projection images containing only tumors were generated at orthogonal angles using a ray projection algorithm as training data for the subsequent tumor tracking model.

[0061] Step S104: Based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of the Transformer model and the CNN network.

[0062] In one embodiment, first, the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image are used as training data, and then the training data is input into the Transformer model in the initial tumor tracking network to obtain the tumor segmentation result output by the Transformer model, and then the tumor segmentation result is input into the CNN network in the initial tumor tracking network to obtain the prediction result output by the CNN network, thereby calculating the loss function based on the prediction result, updating the parameters of the initial tumor tracking network based on the loss function, and determining whether the initial tumor tracking network meets the training end condition, and finally if the initial tumor tracking network meets the training end condition, the initial tumor tracking network that meets the training end condition is used as the trained tumor tracking network. In this embodiment, the generated orthogonal 2D DRR image is input into the tumor tracking network for training to capture tumor location information and morphological features. The network architecture adopts a segmentation network based on the combination of the Transformer model and the CNN network, which optimizes the capture of spatial and detail features by integrating the local feature extraction capability of the convolutional neural network (CNN) and the context capture capability of the Transformer, thereby effectively improving the accuracy of tumor segmentation. In order to obtain the three-dimensional spatial location information of the tumor more accurately, the tumor segmentation results of the orthogonal projection images were effectively integrated and input into the CNN regression network to improve the overall tumor position estimation.

[0063] In one embodiment, after step S104, the method further includes: acquiring a 2D DRR image of the patient as a target image; inputting the target image into the trained tumor tracking network for prediction, and obtaining the three-dimensional spatial position of the tumor output by the tumor tracking network.

[0064] In this embodiment, during the inference test phase, the orthogonal 2D orthogonal projection image is input into the Transformer model in the above-mentioned trained tumor tracking network to obtain the position and morphological characteristics of the tumor in real time and obtain the orthogonal segmentation mask. The orthogonal segmentation mask is then input into the trained CNN network to accurately predict the three-dimensional spatial position of the tumor.

[0065] The tumor tracking method proposed in this embodiment not only achieves sub-millimeter tumor positioning accuracy, but also significantly improves its robustness and generalization ability in a complex clinical environment with noise interference, significantly improves tracking stability, and enhances the accuracy of capturing the three-dimensional spatial position and morphological characteristics of the tumor. The present invention achieves a high degree of accuracy in tumor contour recognition, laying a solid foundation for achieving high-precision three-dimensional spatial positioning. This not only effectively reduces the interference of surrounding background noise, enhances the robustness and accuracy of the model, but also improves its ability to adapt to complex clinical scenarios; at the same time, this design also provides result interpretability, allowing doctors to intuitively check the segmentation results containing tumor morphology and size information, adding transparency and credibility to the entire prediction process.

[0066] For a better understanding of the present invention, refer to Figure 3 , the overall framework of the real-time tumor tracking method of the present invention. (a) Method flow: 4DCT data undergoes two steps of data expansion (b) and DRR projection (c), providing a large number of diverse training samples for the subsequent tumor segmentation model Seg-Net. Subsequently, the tumor segmentation results of orthogonal angles are integrated through the CNN regression network to achieve accurate 3D tumor position prediction. (b) Data augmentation: By combining complex deformations between phases and within phases, hybrid deformations are generated and applied to motion images and tumor labels, thereby obtaining a large number of synthetic CT images (S CT ) and the corresponding tumor signature (S seg (c) DRR projection: 2D DRR training images are generated from the 3D CT volume at orthogonal projection angles of 45° and 135° to enhance the acquisition of spatial information.

[0067] In order to verify the effectiveness of the present invention, a series of experiments were conducted using lung tumor data of real patients and dynamic chest phantom data. Figure 4 As shown, the tumor tracking results of the orthogonal X-ray projection images of the two types of data at different respiratory stages are presented. The top and bottom rows in the figure show the visualization results of the patient and phantom data, respectively, where the red outline and cross represent the tumor position and center of mass of the ground truth (GT), and the blue outline and cross represent the predicted tumor position and center of mass. The results show that in each respiratory stage of the two types of data, both the tumor contour depiction and the center of mass positioning of the present invention are highly consistent with the GT. And after quantitative measurement, the tumor positioning deviation is always kept within 0.6mm.

[0068] In addition, in order to further verify the stability and robustness of the present invention, Gaussian noise of different intensities (S2 = 0, 5, 15, 25) was introduced into the test data set that the model had not seen before, and the tumor tracking performance of the present invention was compared with the traditional target detection method based on Faster R-CNN. To be closer to clinical practice, both methods were trained on noise-free DRR images to test the generalization ability of the model to unknown noise patterns. Figure 5 As shown, the comparison results between the method of the present invention and Faster-RCNN under different noise intensities (S2=0, 5, 15, 25). From left to right, the tracking effects of patient tumors, large tumors of phantoms and small tumors are shown respectively. The bar graph represents the centroid positioning error (left Y-axis): purple is the result of Faster-RCNN, and blue is the result of the present invention. The line graph represents the tracking success rate (right Y-axis): magenta is the result of Faster-RCNN, and indigo is the result of the present invention. As the noise intensity increases, although the performance of both methods decreases, the method of the present invention shows significantly stronger robustness, maintaining lower centroid positioning error, higher intersection over union (IoU) and tracking success rate (TSR) at all noise levels.

[0069] refer to Figure 6 , showing a visual comparison of tumor localization results under different noise levels. The figure shows the tracking results of patients (first two rows), large phantom tumors (middle two rows) and small tumors (bottom two rows) in turn. The red box represents the actual tumor position, the blue box represents the tumor position predicted by the present invention, the green box represents the position detected by Faster R-CNN, and the green cross indicates a detection failure. As the noise increases, the performance of Faster R-CNN deteriorates significantly, and serious detection errors occur under high noise levels, including false detection, serious mismatch between the positioning box and the actual tumor size, and complete tracking failure. In contrast, the present invention performs relatively well and stably at all noise levels, and the predicted tumor position and shape are always highly consistent with the ground truth, which fully demonstrates that the present invention has excellent generalization ability and robustness when dealing with unknown noise interference.

[0070] See also Figure 7 As shown, in one embodiment, a tumor tracking device is provided, the device comprising:

[0071] An acquisition module 10, configured to acquire a 4D CT image of a target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0072] An extraction module 20 is used to extract the 4D CT image to obtain a motion image and various reference images, and to delineate the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0073] A generating module 30, configured to generate a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image;

[0074] The training module 40 is used to perform model training on the initial tumor tracking network based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of Transformer and CNN.

[0075] In one embodiment, the generating module 30 is used to:

[0076] generating a non-rigid deformation based on the motion image and each reference image;

[0077] Processing the motion image and each reference image based on 3D TPS technology to generate a deformation field;

[0078] Determining a mixed deformation field based on the non-rigid deformation and the deformation field;

[0079] Based on the mixed deformation field, the motion image and the tumor segmentation mask image, a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image are generated.

[0080] In one embodiment, the generating module 30 is further used for:

[0081] Intensity-based image registration method, which sequentially aligns the moving images to the reference image and generates the phase-to-phase deformation vector field;

[0082] Based on combining the deformation vector fields between each phase, a non-rigid deformation is obtained.

[0083] The mixed deformation field is expressed as follows:

[0084]

[0085] in, refers to the mixed deformation field, w1 and w2 are random numbers uniformly distributed in the interval (0, 1), refers to non-rigid deformation, It refers to the deformation field.

[0086] In one embodiment, the training module 40 is used to: use the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image as training data;

[0087] Inputting the training data into a Transformer model in an initial tumor tracking network to obtain a tumor segmentation result output by the Transformer model;

[0088] Inputting the tumor segmentation result into a CNN network in an initial tumor tracking network to obtain a prediction result output by the CNN network;

[0089] Calculating a loss function based on the prediction result, updating parameters of an initial tumor tracking network based on the loss function, and determining whether the initial tumor tracking network has reached a training end condition;

[0090] If the initial tumor tracking network reaches the training end condition, the initial tumor tracking network that reaches the training end condition is used as the trained tumor tracking network.

[0091] In one embodiment, the tumor tracking device is further used for:

[0092] Acquire a 2D DRR image of the patient as a target image;

[0093] The target image is input into the trained tumor tracking network for prediction, and the three-dimensional spatial position of the tumor output by the tumor tracking network is obtained.

[0094] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a tumor tracking method.

[0095] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it implements the functions or steps of a tumor tracking method client side.

[0096] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0097] Acquire a 4D CT image of the target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0098] Extracting the 4D CT image to obtain a motion image and various reference images, and outlining the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0099] Based on the motion image, the reference image and the tumor segmentation mask image, generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image;

[0100] Based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of the Transformer model and the CNN network.

[0101] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0102] Acquire a 4D CT image of the target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle;

[0103] Extracting the 4D CT image to obtain a motion image and various reference images, and outlining the tumor target area on the motion image to obtain a tumor segmentation mask image;

[0104] Based on the motion image, the reference image and the tumor segmentation mask image, generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image;

[0105] Based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of the Transformer model and the CNN network.

[0106] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0108] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0109] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A tumor tracking method, characterized in that: The tumor tracking method comprises: Acquire a 4D CT image of the target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle; Extracting the 4D CT image to obtain a motion image and various reference images, and outlining the tumor target area on the motion image to obtain a tumor segmentation mask image; Generate a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image; Based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image, the initial tumor tracking network is trained to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on the combination of the Transformer model and the CNN network.

2. The tumor tracking method according to claim 1, characterized in that: The step of generating a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image comprises: generating a non-rigid deformation based on the motion image and each reference image; Processing the motion image and each reference image based on 3D TPS technology to generate a deformation field; Determining a mixed deformation field based on the non-rigid deformation and the deformation field; A 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image are generated based on the mixed deformation field, the motion image and the tumor segmentation mask image.

3. The tumor tracking method according to claim 2, characterized in that: The step of generating a non-rigid deformation based on the motion image and each reference image comprises: Intensity-based image registration method aligns the moving images to the reference image sequentially to generate the phase-to-phase deformation vector field; Based on combining the deformation vector fields between each phase, a non-rigid deformation is obtained.

4. The tumor tracking method according to claim 2, characterized in that: The mixed deformation field is expressed as follows: in, refers to the mixed deformation field, w 1 and w 2 is a random number uniformly distributed in the interval (0, 1). refers to non-rigid deformation, It refers to the deformation field.

5. The tumor tracking method according to claim 1, characterized in that: The step of training the initial tumor tracking network model based on the 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image to obtain the trained tumor tracking network comprises: The 2D DRR image and the projection image containing only the tumor corresponding to the 2D DRR image are used as training data; Inputting the training data into a Transformer model in an initial tumor tracking network to obtain a tumor segmentation result output by the Transformer model; Inputting the tumor segmentation result into a CNN network in an initial tumor tracking network to obtain a prediction result output by the CNN network; Calculating a loss function based on the prediction result, updating parameters of an initial tumor tracking network based on the loss function, and determining whether the initial tumor tracking network has reached a training end condition; If the initial tumor tracking network reaches the training end condition, the initial tumor tracking network that reaches the training end condition is used as the trained tumor tracking network.

6. The tumor tracking method according to claim 5, characterized in that: If the initial tumor tracking network reaches the training end condition, the step of using the initial tumor tracking network that reaches the training end condition as the trained tumor tracking network includes: Acquire a 2D DRR image of the patient as a target image; The target image is input into the trained tumor tracking network for prediction, and the three-dimensional spatial position of the tumor output by the tumor tracking network is obtained.

7. A tumor tracking device, characterized in that: The tumor tracking device comprises: An acquisition module, used to acquire a 4D CT image of a target user, wherein the 4D CT image includes a 3D CT image of each respiratory phase in a complete respiratory cycle; An extraction module is used to extract the 4D CT image to obtain a motion image and various reference images, and to delineate the tumor target area on the motion image to obtain a tumor segmentation mask image; A generating module, configured to generate a 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image based on the motion image, the reference image and the tumor segmentation mask image; A training module is used to perform model training on an initial tumor tracking network based on the 2D DRR image and a projection image containing only the tumor corresponding to the 2D DRR image, so as to obtain a trained tumor tracking network, wherein the tumor tracking network is a segmentation network based on a combination of Transformer and CNN.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the tumor tracking method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the tumor tracking method according to any one of claims 1 to 6 are implemented.

Images