Automatic positioning method, device and radiotherapy equipment based on deep learning

Abstract

The present invention discloses an automatic positioning method, device and radiotherapy equipment based on deep learning. The automatic positioning method includes the following steps: S1: fix the patient to be radiotherapy on the treatment bed and collect DR images of the corresponding parts; S2: input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the specified parts and reconstruct and generate DRR images; S3: input the DR images obtained by S1 and the DRR images obtained by S2 into the CycleGAN model respectively to obtain DR images containing only the specified parts; S4: align the DR images containing only the specified parts obtained by S3 and the DRR images to be registered to obtain positioning deviations; S5: control the movement of the treatment bed according to the positioning deviations obtained by S4 to achieve automatic positioning. The present invention effectively improves the quality of the images to be registered, improves the accuracy of registration, reduces the prediction time during positioning by deep learning, and finally achieves automatic positioning, avoids manual errors, and improves positioning efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of radiotherapy, and specifically relates to an automatic positioning method and device based on deep learning, and radiotherapy equipment. Background Art

[0002] At present, tumor radiotherapy is generally carried out in sessions, and each treatment uses corresponding thermoplastic film, negative pressure bag and other devices in conjunction with laser light to repeatedly fix the patient according to the position fixation during positioning CT scanning and the resetting during simulation. However, due to various reasons, the patient's position will still have a certain deviation, which is mostly between a few millimeters and one centimeter, or even up to several centimeters.

[0003] To achieve image-guided radiotherapy, existing technologies mainly include: selecting ROI (Region of Interest) based on diagnostic CT and treatment plan, combining CBCT or other tomographic images scanned during treatment, and achieving 3D / 3D image registration. Or collecting MV / KV images to achieve 2D / 2D or 2D / 3D image registration.

[0004] MV image registration is the most difficult problem, as its image contrast is low and the modality difference is large, which makes the traditional registration algorithm based on image gray value density ineffective. In addition, the iterative registration algorithm based on gray value density is usually very time-consuming and has a poor user experience. The feature-based registration algorithm is difficult to select and extract features, and requires technicians to perform it manually, which is time-consuming and labor-intensive, and is prone to human errors. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an automatic positioning method, device and radiotherapy equipment based on deep learning.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] In one aspect, the present invention discloses an automatic positioning method based on deep learning, comprising the following steps:

[0008] S1: Fix the patient to be radiotherapy on the treatment bed and collect DR images of the corresponding parts;

[0009] S2: Input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the specified parts and reconstruct the DRR images;

[0010] S3: Input the DR image obtained in S1 and the DRR image obtained in S2 into the CycleGAN model respectively to obtain a DR image containing only the specified part;

[0011] S4: registering the DR image containing only the designated part obtained in S3 with the DRR image to be registered to obtain the positioning deviation;

[0012] S5: Control the movement of the treatment bed according to the positioning deviation obtained in S4 to achieve automatic positioning.

[0013] Based on the above technical solution, the following improvements can be made:

[0014] As a preferred solution, the training steps of the U-Net model are as follows:

[0015] T1: Collect CT images of various parts for clinical use;

[0016] T2: Divide the CT images into training set, test set and validation set according to the specified ratio;

[0017] T3: Select the U-Net model, with the input being the CT image and the output being the segmentation result of the specified part, and use the data in the training set to train the deep learning model;

[0018] T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

[0019] As a preferred solution, the following steps are included between T1 and T2: preprocessing the collected CT images.

[0020] As a preferred solution, the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

[0021] As a preferred solution, in S2, reconstructing and generating a DRR image specifically includes the following contents:

[0022] Through the DRR reconstruction algorithm, the X-ray light source is simulated to penetrate the CT element, and after attenuation and absorption, it is projected onto the detector plane for imaging, thus obtaining a new DRR image.

[0023] On the other hand, the present invention discloses an automatic positioning device based on deep learning, comprising:

[0024] DR image acquisition module, used to acquire DR images of corresponding parts of the patient after fixing the patient to be radiotherapy on the treatment bed;

[0025] The DRR image generation module is used to input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the specified parts and reconstruct the DRR images;

[0026] A DR image generation module is used to input the DR image acquired by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model respectively to obtain a DR image containing only the specified part;

[0027] A positioning deviation generating module is used to register the DR image containing only the specified part generated by the DR image generating module with the DRR image to be registered, so as to obtain the positioning deviation;

[0028] The automatic positioning module is used to control the movement of the treatment bed according to the positioning deviation generated by the positioning deviation generation module to achieve automatic positioning.

[0029] As a preferred solution, the automatic positioning device further includes: a U-Net model training module, the U-Net model training module is used to train the U-Net model using the following steps;

[0030] The training steps of the U-Net model are as follows:

[0031] T1: Collect CT images of various parts for clinical use;

[0032] T2: Divide the CT images into training set, test set and validation set according to the specified ratio;

[0033] T3: Select the U-Net model, with the input being the CT image and the output being the segmentation result of the specified part, and use the data in the training set to train the deep learning model;

[0034] T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

[0035] As a preferred solution, the following steps are included between T1 and T2: preprocessing the collected CT images.

[0036] As a preferred solution, the preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

[0037] In addition, on the other hand, the present invention also discloses a radiotherapy device, which realizes automatic positioning by using any of the above-mentioned automatic positioning methods, or includes: any of the above-mentioned automatic positioning devices.

[0038] The present invention discloses an automatic positioning method, device and radiotherapy equipment based on deep learning, which utilizes U-Net model and CycleGAN model to effectively improve the quality of images to be registered and the accuracy of registration, and reduces the prediction time during positioning by deep learning, ultimately achieving automatic positioning, avoiding manual errors and improving positioning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 The present invention provides a flowchart of an automatic positioning method.

[0041] Figure 2 A flowchart of training a U-Net model provided in an embodiment of the present invention.

[0042] Figure 3 A schematic diagram of DRR image generation provided by an embodiment of the present invention.

[0043] Among them: 1-simulated light source, 2-CT volume element, 3-DRR plane. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The expression of “comprising” an element is an “open” expression, which merely means that a corresponding component or step exists, and should not be interpreted as excluding additional components or steps.

[0047] To achieve the purpose of the present invention, in some embodiments of an automatic positioning method, device and radiotherapy equipment based on deep learning, since the characteristics of the spine in the neck, chest and abdomen are obvious, the method disclosed in this embodiment is based on spine alignment.

[0048] like Figure 1 As shown, the automatic positioning method includes the following steps:

[0049] S1: Fix the neck, chest and abdomen of the patient to be radiotherapy on the treatment bed and collect DR images of the corresponding parts;

[0050] S2: Input the CT images collected during planning into the trained U-Net model to obtain the spine segmentation results and reconstruct the DRR images;

[0051] S3: Input the DR image obtained in S1 and the DRR image obtained in S2 into the CycleGAN model respectively to obtain a DR image containing only the spine;

[0052] S4: registering the DR image containing only the spine obtained in S3 with the DRR image to be registered to obtain the positioning deviation;

[0053] S5: Control the movement of the treatment bed according to the positioning deviation obtained in S4 to achieve automatic positioning.

[0054] In order to further optimize the implementation effect of the present invention, in some other embodiments, the remaining characteristic technologies are the same, except that, Figure 2 As shown, the training steps of the U-Net model are as follows:

[0055] T1: Collect CT images (cross-section) of the neck, chest, and abdomen for clinical use in nii format;

[0056] T2: Data division: CT images are divided into training set, test set and validation set in a ratio of 6:3:1;

[0057] T3: Select the U-Net model, with the input as CT images and the output as the segmentation result of the spine. Use the data in the training set to train the deep learning model.

[0058] T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

[0059] Further, based on the above embodiment, the following steps are included between T1 and T2: preprocessing the collected CT images.

[0060] Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

[0061] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining characteristic technologies are the same, except that, in S2, reconstructing and generating a DRR image specifically includes the following contents:

[0062] Through the DRR reconstruction algorithm, the X-ray light source is simulated to penetrate the CT element, and after attenuation and absorption, it is projected onto the detector plane for imaging, thus obtaining a new DRR image.

[0063] It is worth noting that when performing image registration, the two entities to be registered should have the same dimension, that is, either 3D-3D registration or 2D-2D registration. When performing 2D-3D registration, the 3D image needs to be reduced to two dimensions, and then 2D-2D registration is performed. The above DRR reconstruction algorithm is a process of reducing the dimension of a 3D model to 2D, such as Figure 3 As shown, the simulated X-ray light source passes through 3D voxels, namely 3D-CT voxels, and is projected onto the DRR plane to generate a DRR image. The whole process is an optical attenuation process, which conforms to the law of the optical absorption model. The attenuation process can be described by the following expression.

[0064]

[0065] Where: s is the length parameter of the optical projection direction;

[0066] I(s) is the optical intensity at distance s;

[0067] x(t) is the attenuation coefficient of optical intensity;

[0068] I0 is the optical intensity entering the CT voxel.

[0069] Depend on Figure 3 As can be seen from the above formula, DRR is a process in which a simulated ray emitted by a simulated light source 1 passes through a CT volume element 2, is projected onto an imaging plane for accumulation after attenuation and absorption, specifically:

[0070] 1. Establish a 3D voxel matrix of the CT image group, which is composed of a number of CT voxels;

[0071] 2. Send out several rays along the virtual light source image CT image group, the number of rays is consistent with the number of pixels in DRR plane 3;

[0072] 3. Obtain the intersection points where each ray passes through this CT voxel and accumulate the electron density values ​​of these points;

[0073] 4. Find the finite ray length of the projection line through the voxel matrix;

[0074] 5. Multiply the accumulated value of electron density by the ray length, and display the obtained value in grayscale, which is the DRR image.

[0075] The embodiment of the present invention discloses an automatic positioning device based on deep learning, comprising:

[0076] DR image acquisition module, used to acquire DR images of the corresponding parts of the patient's neck, chest and abdomen after fixing them on the treatment bed;

[0077] The DRR image generation module is used to input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the spine and reconstruct the DRR images;

[0078] A DR image generation module is used to input the DR image acquired by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model respectively to obtain a DR image containing only the spine;

[0079] A positioning deviation generating module is used to register the DR image containing only the spine generated by the DR image generating module with the DRR image to be registered, so as to obtain a positioning deviation;

[0080] The automatic positioning module is used to control the movement of the treatment bed according to the positioning deviation generated by the positioning deviation generation module to achieve automatic positioning.

[0081] Further, based on the above embodiment, the automatic positioning device further includes: a U-Net model training module, the U-Net model training module is used to train the U-Net model using the following steps;

[0082] The training steps of the U-Net model are as follows:

[0083] T1: Collect CT images (cross-section) of the neck, chest, and abdomen for clinical use in nii format;

[0084] T2: Data partitioning, dividing CT images into training set, test set and validation set in a ratio of 6:3:1;

[0085] T3: Select the U-Net model, with the input as CT images and the output as the segmentation result of the spine. Use the data in the training set to train the deep learning model.

[0086] T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

[0087] Further, based on the above embodiment, the following steps are included between T1 and T2: preprocessing the collected CT images.

[0088] Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

[0089] In addition, an embodiment of the present invention further discloses a radiotherapy device, which realizes automatic positioning by using the automatic positioning method disclosed in any of the above embodiments, or includes: the automatic positioning device disclosed in any of the above embodiments.

[0090] The present invention involves two deep learning models. The first model is the U-Net model, which is trained to automatically segment and extract the spine area in the CT image, and obtain the registered DRR image through the DRR reconstruction algorithm; the second model is the CycleGan model, which removes other redundant information in the DR image and retains the spine, thereby reducing the interference of other factors in the image on the registration result. Extracting the ROI area through the above method helps to improve the positioning accuracy and reduce the positioning time.

[0091] The automatic positioning method, device and radiotherapy equipment based on deep learning of the present invention have the following beneficial effects:

[0092] First, the U-Net deep learning model is used to automatically extract the spine area, and CycleGAN is used to remove the interference factors in the image registration, retain the spine, and register the ROI area in a targeted manner to improve the quality of the image to be registered and the accuracy of the registration;

[0093] Second, it requires low technicians and is easy to use;

[0094] Third, the positioning speed is fast. Excluding the DRR reconstruction process, the positioning time is within 5 seconds, while the manual positioning method takes 5 minutes;

[0095] Fourth, it can be iterated quickly, and each upgrade can be provided to the hospital online without the need for lengthy training.

[0096] The present invention can effectively improve the quality of the images to be registered, improve the accuracy of registration, reduce the prediction time during positioning through deep learning, and ultimately achieve automatic positioning, avoid human errors, and improve positioning efficiency.

[0097] It should be understood that the various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or portions of the method and apparatus of the present invention, can take the form of program codes (i.e., instructions) embedded in a tangible medium, such as a floppy disk, a CD-ROM, a hard disk drive, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0098] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it. They cannot be used to limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic positioning method based on deep learning, characterized in that: The following steps are involved: S1: Fix the patient to be radiotherapy on the treatment bed and collect DR images of the corresponding parts; S2: Input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the specified parts and reconstruct the DRR images; S3: Input the DR image obtained in S1 and the DRR image obtained in S2 into the CycleGAN model respectively to obtain a DR image containing only the specified part; S4: registering the DR image containing only the designated part obtained in S3 with the DRR image to be registered to obtain the positioning deviation; S5: Control the movement of the treatment bed according to the positioning deviation obtained in S4 to achieve automatic positioning; The training steps of the U-Net model are as follows: T1: Collect CT images of various parts for clinical use; T2: Divide the CT images into training set, test set and validation set according to the specified ratio; T3: Select the U-Net model, with the input being the CT image and the output being the segmentation result of the specified part, and use the data in the training set to train the deep learning model; T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

2. The automatic positioning method according to claim 1, characterized in that: The following steps are included between T1 and T2: pre-processing the collected CT images.

3. The automatic positioning method according to claim 2, characterized in that: Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

4. The automatic positioning method according to any one of claims 1 to 3, characterized in that: In S2, the reconstruction and generation of the DRR image specifically includes the following contents: Through the DRR reconstruction algorithm, the X-ray light source is simulated to penetrate the CT element, and after attenuation and absorption, it is projected onto the detector plane for imaging, thus obtaining a new DRR image.

5. An automatic positioning device based on deep learning, characterized in that: include: DR image acquisition module, used to acquire DR images of corresponding parts of the patient after fixing the patient to be radiotherapy on the treatment bed; The DRR image generation module is used to input the CT images collected during planning into the trained U-Net model to obtain the segmentation results of the specified parts and reconstruct the DRR images; A DR image generation module is used to input the DR image acquired by the DR image acquisition module and the DRR image generated by the reconstructed DRR image generation module into the CycleGAN model respectively to obtain a DR image containing only the specified part; A positioning deviation generating module is used to register the DR image containing only the specified part generated by the DR image generating module with the DRR image to be registered, so as to obtain the positioning deviation; An automatic positioning module is used to control the movement of the treatment bed according to the positioning deviation generated by the positioning deviation generation module to achieve automatic positioning; The automatic positioning device further includes: a U-Net model training module, the U-Net model training module is used to train the U-Net model using the following steps; The training steps of the U-Net model are as follows: T1: Collect CT images of various parts for clinical use; T2: Divide the CT images into training set, test set and validation set according to the specified ratio; T3: Select the U-Net model, with the input being the CT image and the output being the segmentation result of the specified part, and use the data in the training set to train the deep learning model; T4: Use the validation set and test set to test the robustness of the U-Net model at different stages until a sufficiently robust U-Net model is trained.

6. The automatic positioning device according to claim 5, characterized in that: The following steps are included between T1 and T2: pre-processing the collected CT images.

7. The automatic positioning device according to claim 6, characterized in that: Preprocessing includes one or more of the following operations: reading and saving CT image slices into png format, normalizing pixel size, and image scaling.

8. Radiotherapy equipment, characterized in that Automatic positioning is achieved using the automatic positioning method as described in any one of claims 1-4, or, comprising: the automatic positioning device as described in any one of claims 5-7.

Images