Method, device and equipment for reconstructing non-coplanar cone beam CT (Computed Tomography) image

Abstract

The invention provides a non-coplanar cone beam CT image reconstruction method, device and equipment, and relates to the technical field of medical image processing. The method comprises the following steps: acquiring a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; the target object is located on target equipment, the coplanar cone beam CT image is a three-dimensional image collected before the target equipment rotates, and the non-coplanar limited angle projection drawing is a two-dimensional projection image collected after the target equipment rotates to a target angle; and inputting the coplanar cone beam CT image and the non-coplanar limited angle projection drawing into a trained image generation model, and outputting a non-coplanar cone beam CT image at a target angle. According to the method, the non-coplanar cone beam CT image which meets clinical requirements and is high in quality can be generated, and the generation speed is higher than that of a conventional iteration generation method.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method, device and equipment for reconstructing non-coplanar cone-beam CT images. Background Art

[0002] Non-coplanar radiotherapy technology can achieve a rapid decrease in the dose outside the target area, thereby reducing the irradiation of nearby organs. Therefore, it has been widely used in stereotactic radiotherapy and stereotactic body radiotherapy. The accurate implementation of non-coplanar radiotherapy is inseparable from the accurate verification of the patient's position, especially the position verification after the bed is turned during the treatment process.

[0003] In the prior art, the patient's position before treatment is usually verified in three dimensions with the help of cone beam computed tomography (CBCT) images. However, in non-coplanar treatment, the possibility of collision between the treatment bed, the gantry, the patient, etc. limits the scanning range of the CBCT image, which leads to the current traditional iterative reconstruction algorithms such as filtered back projection (Feldkamp-Davis-Kress, FDK) and the like. When the projection image obtained within a limited scanning angle range is used to reconstruct the cone beam CT image, the obtained image may have serious stripe artifacts, and the quality of the reconstructed image is not high.

[0004] Therefore, a new cone beam CT image reconstruction method is urgently needed to solve the above problems. Summary of the invention

[0005] The present application provides a method, device and equipment for reconstructing non-coplanar cone-beam CT images, so as to solve the problem of how to reconstruct non-coplanar cone-beam CT images.

[0006] In a first aspect, the present application provides a method for reconstructing a non-coplanar cone-beam CT image, the method comprising:

[0007] Acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle;

[0008] The coplanar cone beam CT image and the non-coplanar limited angle projection image are input into a trained image generation model, and a non-coplanar cone beam CT image at the target angle is output.

[0009] Optionally, the step of acquiring a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of the target object includes:

[0010] Before the target device is rotated, a coplanar cone beam CT image of the target portion of the target object is acquired;

[0011] After obtaining the coplanar cone beam CT image, controlling the target device to rotate to a target angle, and determining a scanning angle range of the imaging device based on the angle after the target device is rotated;

[0012] Determining a plurality of scanning angles based on the angle range and the preset angle value method;

[0013] Based on preset scanning parameters corresponding to the scanning angle, a non-coplanar limited-angle projection image corresponding to the scanning angle is acquired.

[0014] Optionally, the image generation model includes an encoder and a decoder, the encoder is a dual-branch structure, the dual-branch structure is used to extract image features of the coplanar cone beam CT image and the non-coplanar limited angle projection image, and the decoder is used to generate the non-coplanar cone beam CT image based on the extracted image features.

[0015] Optionally, the method further comprises:

[0016] Acquire sample data; wherein each set of sample data includes a sample coplanar cone beam CT image, a sample non-coplanar finite angle projection image, and an ideal non-coplanar cone beam CT image, wherein the sample coplanar cone beam CT image and the sample non-coplanar finite angle projection image are model input parameters, and the ideal non-coplanar cone beam CT image is a calibration parameter;

[0017] The image generation model and the discriminator are trained by the sample data, and the image generation model and the discriminator are optimized by using a joint loss function until preset conditions are met, thereby obtaining a trained image generation model; wherein the discriminator is used to distinguish between the non-coplanar cone beam CT image generated by the image generation model and the ideal non-coplanar cone beam CT image.

[0018] Optionally, obtaining sample data includes:

[0019] Acquire multiple sample coplanar cone beam CT images;

[0020] Rotating the sample coplanar cone beam CT image to a preset angle to obtain a non-coplanar cone beam CT image at the preset angle, and determining that the non-coplanar cone beam CT image at the preset angle is an ideal non-coplanar cone beam CT image;

[0021] The rotation error and the translation error are introduced into the ideal non-coplanar cone beam CT image, and based on the preset angle, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to obtain a sample non-coplanar limited angle projection image.

[0022] Optionally, based on the preset angle, forward projecting the ideal non-coplanar cone beam CT image after the error is introduced to obtain a sample non-coplanar limited angle projection image, including:

[0023] Based on the preset angle, determining an angle range of the sample projection image;

[0024] Determining a plurality of sample scanning angles based on the angle range of the sample projection image and a preset angle value method;

[0025] Based on the preset scanning parameters corresponding to the sample scanning angle, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to obtain a sample non-coplanar limited angle projection image corresponding to the sample scanning angle.

[0026] Optionally, the joint loss function includes adversarial loss, absolute value loss and bone structure loss.

[0027] In a second aspect, the present application provides a device for reconstructing a non-coplanar cone-beam CT image, the device comprising:

[0028] An acquisition unit, configured to acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle;

[0029] The reconstruction unit is used to input the coplanar cone beam CT image and the non-coplanar limited angle projection image into a trained image generation model, and output a non-coplanar cone beam CT image at the target angle.

[0030] In a third aspect, the present application provides an electronic device, the electronic device comprising: a processor, and a memory communicatively connected to the processor;

[0031] The memory stores computer-executable instructions;

[0032] The processor executes the computer-executable instructions stored in the memory to implement the aforementioned method.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the method as described above when executed by a processor.

[0034] In a fifth aspect, the present application provides a computer program product, including a computer program, which is used to implement the method as described above when executed by a processor.

[0035] The present application provides a method, device and equipment for reconstructing a non-coplanar cone beam CT image, comprising: obtaining a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle; the coplanar cone beam CT image and the non-coplanar limited angle projection image are input into a trained image generation model, and the non-coplanar cone beam CT image at the target angle is output. On the one hand, the present application utilizes a pre-trained image generation model to process coplanar cone beam CT images and non-coplanar limited angle projection images, and can directly generate non-coplanar cone beam CT images. The image generation speed is faster and more efficient than the iterative generation method. On the other hand, under the condition of limited angle projection images, the present application uses coplanar cone beam CT images as prior information to supplement the information gaps missing in non-coplanar limited angle projection images, and can reconstruct high-quality non-coplanar cone beam CT images that meet clinical needs. The reconstructed images have higher accuracy, which makes it more accurate to verify the position of patients undergoing non-coplanar treatment during treatment through non-coplanar cone beam CT images, which helps to improve the actual treatment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] Figure 1 A schematic diagram of a flow chart of a method for reconstructing a non-coplanar cone-beam CT image provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the structure of an image generation model and a discriminator provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of an architecture for generating a non-coplanar cone beam CT image provided in an embodiment of the present application;

[0040] Figure 4A flowchart of a method for obtaining a trained image generation model provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a non-coplanar cone-beam CT image reconstruction device provided in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of another non-coplanar cone-beam CT image reconstruction device provided in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0044] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0045] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0046] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] The accurate implementation of non-coplanar radiotherapy is inseparable from the accurate verification of the patient's position, especially the position verification after the bed is turned during the treatment process. In non-coplanar treatment, the verification result of position verification using cone beam CT images is the most accurate. However, due to the possibility of collision between the treatment bed, the gantry, the patient, etc., the scanning range of the cone beam CT image is limited, that is, after the bed is turned, only non-coplanar projection images within a limited angle range can be collected. At present, traditional iterative reconstruction algorithms such as the FDK algorithm are usually used to reconstruct cone beam CT images. This method of multiple iterations to obtain images that meet the requirements, when using the projection images obtained within a limited scanning angle range to reconstruct cone beam CT images, the obtained images will have serious strip artifacts, and the quality of the reconstructed image cannot be guaranteed, which will directly affect the position verification results and thus affect the treatment effect.

[0048] In order to solve the above problems, the present application provides a method, device and equipment for reconstructing non-coplanar cone beam CT images. The present application uses a pre-trained image generation model to process coplanar cone beam CT images and non-coplanar limited angle projection images, and can directly generate non-coplanar cone beam CT images. Not only is the image generation speed fast, but also under the condition of limited angle projection images, the coplanar cone beam CT images are used as prior information to supplement the information gaps missing in the limited angle projection images. It is possible to reconstruct high-quality non-coplanar cone beam CT images that meet clinical needs, making it possible to verify the position of patients undergoing non-coplanar treatment during treatment through non-coplanar cone beam CT images, and also improving the accuracy of position verification. In addition, in addition to position verification, the reconstructed non-coplanar cone beam CT images can also be used for other purposes such as adaptive radiotherapy.

[0049] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0050] Figure 1 A flowchart of a method for reconstructing a non-coplanar cone beam CT image provided in an embodiment of the present application. The execution subject of the embodiment of the present application may be a device for reconstructing a non-coplanar cone beam CT image, and the device for reconstructing a non-coplanar cone beam CT image may be located on an electronic device, and the electronic device may be a computer, etc., which may be connected to a medical device, and the present application does not limit this. The embodiment of the present application is described in detail by taking the execution subject as a device for reconstructing a non-coplanar cone beam CT image as an example.

[0051] like Figure 1 As shown, the method for reconstructing a non-coplanar cone beam CT image provided by this embodiment includes:

[0052] S101. Acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle.

[0053] Exemplarily, in the present application, the target object is the object to be treated, and the target part is the part to be treated. The coplanar cone beam CT image and the non-coplanar limited angle projection image of the target part of the target object can be directly transmitted to the reconstruction device of the non-coplanar cone beam CT image after being acquired by the corresponding image acquisition device, or can be manually input into the reconstruction device of the non-coplanar cone beam CT image after the user obtains the image acquired by the corresponding image acquisition device. The corresponding image acquisition device can also be used as a part of the reconstruction device of the non-coplanar cone beam CT image, and the corresponding image can be directly acquired, which is not limited in the present application.

[0054] Exemplarily, when performing image acquisition, the target object is located on the target device, which is generally a treatment bed. Before the target device is rotated, the gantry can be rotated as needed, and the image acquisition device can acquire a coplanar cone beam CT image of the target part of the target object. When the target device is rotated to a certain angle, in order to avoid collision between the treatment bed, the gantry, the patient, etc., the rotation range of the gantry becomes smaller, and the image acquisition device can only acquire non-coplanar limited-angle projection images within a limited angle range.

[0055] In one example, the non-coplanar limited angle projection images collected by the present application may be multiple, and the more non-coplanar limited angle projection images there are, the more accurate the non-coplanar cone beam CT image subsequently reconstructed will be. Exemplarily, obtaining a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object may include:

[0056] S1. Before the target device rotates, a coplanar cone beam CT image of a target part of a target object is acquired.

[0057] S2. After obtaining the coplanar cone beam CT image, control the target device to rotate to a target angle, and determine the scanning angle range of the imaging device based on the angle after the target device is rotated.

[0058] S3. Determine multiple scanning angles based on the scanning angle range and the preset angle value method.

[0059] S4. Based on preset scanning parameters corresponding to the scanning angle, a non-coplanar limited-angle projection image corresponding to the scanning angle is acquired.

[0060] Exemplarily, before the target device is rotated, the target device is at an initial position, at which the corresponding image acquisition device can acquire a coplanar cone beam CT image of the target part of the target object, so that the reconstruction device of the non-coplanar cone beam CT image of the present application can obtain a coplanar cone beam CT image of the target part of the target object. After obtaining the coplanar cone beam CT image, the target device is rotated to the target angle, and then the angle range is determined based on the angle after the target device is rotated. Among them, the target angle can be the angle determined in the treatment plan, and the angle range is the rotation range of the rack after the target device is rotated to the target angle, and the range can be predetermined according to the angle that the rack can actually rotate. In actual applications, the mapping relationship between the target angle and the angle range can be pre-set, and when the target angle is determined, the angle range can be determined accordingly.

[0061] Further, after determining the angle range, the reconstruction device of the non-coplanar cone beam CT image of the present application can determine multiple scanning angles based on the angle range and the preset angle value method, and obtain the non-coplanar limited angle projection image corresponding to the scanning angle based on the scanning parameters corresponding to the scanning angle. For example, if after the target device is rotated to the target angle, such as 90°, the angle range is determined to be 250° to 310°, and the preset angle value method is to take a non-coplanar limited angle projection image every 0.4°, there can be 150 scanning angles, each scanning angle corresponds to a scanning parameter, and scanning based on the corresponding scanning parameters can collect 150 corresponding non-coplanar limited angle projection images. Among them, the scanning parameters may include the distance from the radiation source to the detection plate, the size of the detection plate, the detection spacing, the projection frequency, etc., which are not limited by the present application.

[0062] S102 , inputting the coplanar cone beam CT image and the non-coplanar limited angle projection image into a trained image generation model, and outputting a non-coplanar cone beam CT image at a target angle.

[0063] Exemplarily, after acquiring a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object, the non-coplanar cone beam CT image reconstruction device of the present application inputs the acquired coplanar cone beam CT image and the non-coplanar limited angle projection image into a trained image generation model for processing, and the model outputs a non-coplanar cone beam CT image at the target angle, thereby quickly obtaining a reconstructed non-coplanar cone beam CT image at the target angle, so as to perform position verification on the target part of the target object.

[0064] Among them, the present application does not limit the specific structure of the image generation model and how to train the image generation model (among which, a possible training process of the image generation model is described in detail in the next embodiment). In one example, the image generation model includes an encoder and a decoder, the encoder is a dual-branch structure, and the dual-branch structure is used to extract image features of a coplanar cone beam CT image and a non-coplanar limited angle projection image, respectively, and the decoder is used to generate a non-coplanar cone beam CT image based on the extracted image features.

[0065] For example, Figure 2 The following is a schematic diagram of the structure of an image generation model and a discriminator provided in an embodiment of the present application. Figure 2 As shown in the image generation model on the left, the image generation model of the present application includes an encoder and a decoder, wherein the encoder can adopt a dual-branch structure, each encoding branch can be composed of multiple convolution modules and two fully connected layers, one branch is used to encode the high-dimensional features in the input coplanar cone beam CT image, and the other branch is used to encode the high-dimensional features in the input non-coplanar limited angle projection image.

[0066] For example, the encoder can be composed of four convolution modules and two fully connected layers, each convolution module can include two convolution layers and one pooling layer, each convolution layer has batch normalization and parameter-corrected linear unit activation functions, and the size of the filter in each convolution layer can be 3×3×3. The features extracted by the dual-branch encoder are merged and input together into the fully connected layer in the decoder. There are four deconvolution modules after the fully connected layer in the decoder, each deconvolution module can include two deconvolution layers and one pooling layer, each deconvolution layer has batch normalization and parameter-corrected linear unit activation functions, and the size of the filter in each deconvolution layer can be 3×3×3. Through multiple deconvolutions, the decoder can finally obtain a non-coplanar cone beam CT image. In addition, the encoder and decoder can also have other structures, which are not limited by this application.

[0067] Based on this, the trained image generation model can use coplanar cone beam CT images as prior information under the condition of limited-angle projection images to fill in the missing information gaps in limited-angle projection images and reconstruct high-quality non-coplanar cone beam CT images that meet clinical needs. This makes it possible to verify the position of patients undergoing non-coplanar treatment during treatment through non-coplanar cone beam CT images.

[0068] The embodiment of the present application provides a method for reconstructing a non-coplanar cone beam CT image, comprising: acquiring a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle; the coplanar cone beam CT image and the non-coplanar limited angle projection image are input into a trained image generation model, and the non-coplanar cone beam CT image at the target angle is output. On the one hand, the present application utilizes a pre-trained image generation model to process coplanar cone beam CT images and non-coplanar limited angle projection images, and can directly generate non-coplanar cone beam CT images. The image generation speed is faster and more efficient than the iterative generation method. On the other hand, under the condition of limited angle projection images, the present application uses coplanar cone beam CT images as prior information to supplement the information gaps missing in non-coplanar limited angle projection images, and can reconstruct high-quality non-coplanar cone beam CT images that meet clinical needs. The reconstructed images have higher accuracy, which makes it more accurate to verify the position of patients undergoing non-coplanar treatment during treatment through non-coplanar cone beam CT images, which helps to improve the actual treatment effect.

[0069] In addition to position verification, the reconstructed non-coplanar cone beam CT images can also be used for other purposes such as adaptive radiotherapy, which is not limited in this application.

[0070] For example, Figure 3 A schematic diagram of an architecture for generating non-coplanar cone beam CT images provided in an embodiment of the present application. Figure 3 As shown in FIG. 1 , the method includes a training phase and an application phase. The training phase is the process of inputting the acquired coplanar cone beam CT images and non-coplanar finite angle projection images into the image generation model to train the model; the application phase is the process of inputting the newly acquired coplanar cone beam CT images and non-coplanar finite angle projection images into the trained image generation model (i.e., the generator in the figure) to generate non-coplanar cone beam CT images.

[0071] For example, Figure 4 A flowchart of a method for obtaining a trained image generation model provided in an embodiment of the present application. Figure 4 As shown, the process of obtaining a trained image generation model may include:

[0072] S401, obtaining sample data; wherein each set of sample data includes a sample coplanar cone beam CT image, a sample non-coplanar finite angle projection image, and an ideal non-coplanar cone beam CT image, the sample coplanar cone beam CT image and the sample non-coplanar finite angle projection image are model input parameters, and the ideal non-coplanar cone beam CT image is a calibration parameter.

[0073] Exemplarily, when performing model training, it is first necessary to obtain sample data. In the present application, the trained image generation model is used to generate non-coplanar cone beam CT images based on coplanar cone beam CT images and non-coplanar finite angle projection images. Therefore, each set of sample data in the present application includes at least model input parameters: sample coplanar cone beam CT images and sample non-coplanar finite angle projection images, as well as an ideal non-coplanar cone beam CT image as a calibration parameter (ie, gold standard). When training, the sample data can be divided into training set data, validation set data, and test set data in proportion, which are used for training, validation, and testing models, respectively, to obtain the final trained image generation model.

[0074] In one example, obtaining sample data may include:

[0075] S10, acquiring a plurality of sample coplanar cone-beam CT images.

[0076] S20, rotating the sample coplanar cone beam CT image to a preset angle to obtain a non-coplanar cone beam CT image at the preset angle, and determining that the non-coplanar cone beam CT image at the preset angle is an ideal non-coplanar cone beam CT image.

[0077] S30, introducing rotation error and translation error into the ideal non-coplanar cone-beam CT image, and forward projecting the ideal non-coplanar cone-beam CT image after the error is introduced based on a preset angle to obtain a sample non-coplanar limited angle projection image.

[0078] Exemplarily, a large amount of sample data is required to train the image generation model. When acquiring the sample data, multiple sample coplanar cone beam CT images can be acquired first, and then the multiple sample coplanar cone beam CT images can be processed to obtain the corresponding sample non-coplanar limited angle projection images and ideal non-coplanar cone beam CT images. Specifically, the acquired sample coplanar cone beam CT images can be rotated as a whole to a preset angle, which is generally the same as the above-mentioned target angle, and the non-coplanar cone beam CT images obtained at the preset angle are used as ideal non-coplanar cone beam CT images. Further, the rotation error and the translation error are randomly introduced into the ideal non-coplanar cone beam CT image, and then based on the preset angle, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to obtain the sample non-coplanar limited angle projection image. Among them, the random introduction of rotation error and translation error is to simulate the positioning error caused by various factors such as the rotation of the treatment bed and the movement of the patient during non-coplanar treatment, simulate a more realistic actual situation, and obtain a more accurate sample non-coplanar limited angle projection image.

[0079] In one example, based on a preset angle, forward projecting the ideal non-coplanar cone beam CT image after introducing an error to obtain a sample non-coplanar limited angle projection image may include:

[0080] S31 . Determine an angle range of a sample projection image based on a preset angle.

[0081] S32: Determine the sample scanning angle based on the angle range of the sample projection image and the preset angle value method.

[0082] S33. Based on preset scanning parameters corresponding to the sample scanning angle, forward project the ideal non-coplanar cone beam CT image after the error is introduced to obtain a sample non-coplanar limited angle projection image corresponding to the sample scanning angle.

[0083] Exemplarily, based on the mapping relationship between the preset angle and the angle range, the angle range of the projection image that can be obtained clinically corresponding to the preset angle can be determined. In general, the closer the treatment bed rotates to plus or minus 90 degrees, the smaller the angle range of the projection image that can be obtained clinically, and the fewer the number of projection images available during model training. In practical applications, the angle range can be set according to the actual range that can be collected. Further, according to the determined angle range of the sample projection image and the preset angle value method, the sample scanning angle can also be determined, and each sample scanning angle is set with corresponding sample scanning parameters, such as the distance from the radiation source to the detection plate, the size of the detection plate, the detection spacing, the projection frequency, etc. Based on these sample scanning parameters, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to simulate the actual scanning situation, and the sample non-coplanar limited angle projection image corresponding to the sample scanning angle can be obtained.

[0084] Among them, the above method can not only quickly obtain sample data, but also solve the problem of difficulty in collecting large quantities of sample data.

[0085] S402. Train the image generation model and the discriminator through sample data, and optimize the image generation model and the discriminator using a joint loss function until preset conditions are met to obtain a trained image generation model; wherein the discriminator is used to distinguish between a non-coplanar cone beam CT image generated by the image generation model and an ideal non-coplanar cone beam CT image.

[0086] Exemplarily, after the sample data is obtained, the sample data can be divided into training set data, validation set data and test set data in proportion, which are used for training, validation and testing models respectively, so as to obtain the final trained image generation model. Specifically, during training, the sample coplanar cone beam CT image and the sample non-coplanar finite angle projection image are input into the image generation model, the encoder structure contained in the image generation model will extract the high-order features of each image, and the decoder structure contained in the image generation model will directly generate the non-coplanar cone beam CT image based on the extracted high-order features; then, the generated non-coplanar cone beam CT image and the ideal non-coplanar cone beam CT image are input into the discriminator for comparison, and the discriminator will output the discrimination result; then, based on the discrimination result, the image generation model and the discriminator are optimized using the joint loss function until the preset conditions are met, and the trained image generation model is obtained.

[0087] For example, the discriminator may be Figure 2 The structure on the right side of the figure is composed of multiple convolutional modules and a fully connected layer. For example, the discriminator can use a patch-based convolutional neural network (CNN), which is composed of five convolutional modules and a fully connected layer. Each convolutional module can contain two convolutional layers and a pooling layer. Each convolutional layer has batch normalization and parameter-corrected linear unit activation functions, and the size of the filter in each convolutional layer is 3×3×3. Of course, the discriminator can also be other structures, which is not limited in this application.

[0088] Among them, when the joint loss function is used to optimize the image generation model and the discriminator, for the discriminator, it is necessary to make the loss function value between the generated non-coplanar cone-beam CT image and the ideal non-coplanar cone-beam CT image larger, so that the discriminator can distinguish the generated non-coplanar cone-beam CT image from the ideal non-coplanar cone-beam CT image as much as possible; and for the image generation model (that is, Figure 3 Generator in ), it is necessary to make the loss function value between the generated non-coplanar cone-beam CT image and the ideal non-coplanar cone-beam CT image smaller, so that the image generation model can generate a non-coplanar cone-beam CT image that is as similar to the ideal non-coplanar cone-beam CT image as possible.

[0089] Exemplarily, the joint loss function includes adversarial loss, absolute value loss, and bone structure loss.

[0090] Among them, the adversarial loss R(G,D) can be defined as:

[0091]

[0092] The absolute value loss can be defined as:

[0093]

[0094] Loss of bone structure can be defined as:

[0095]

[0096] The joint loss function J(G,D) can be defined as:

[0097]

[0098] Among them, G is the generator, D is the discriminator, iData is the input data (i.e., coplanar cone beam CT images and non-coplanar limited angle projection images), is the expected value, λ1 is the weight of the absolute value loss, λ2 is the weight of the bone structure loss, and Bone(·) is a function used to extract the bone structure in the input image. The calculation formula is as follows:

[0099] Bone(I)=Sigmoid(Scale*(I-BoneThrehold))

[0100] Among them, Sigmoid is a sine formula, Scale is a maximum parameter value, I is the input image, and BoneThrehold is the threshold for extracting bone structure. In model training, the threshold is a constant value and cannot be differentiated, so it is necessary to use a sine formula and assign a maximum parameter value to make it differentiable; Edge(·) is the edge extraction formula. At the same time, since there is a large rotational deviation between the coplanar cone beam CT image of the input model of this application and the output non-coplanar cone beam CT image, the central moment CM(·) is added to the bone structure loss. The central moment has rotational and translational invariance, which can ensure the structural consistency between the input coplanar cone beam CT image and the output non-coplanar cone beam CT image.

[0101] At this point, the image generation model and the discriminator are continuously optimized using the above-mentioned joint loss function until the number of training times reaches a preset number, or the error of the output result reaches a preset threshold, and a trained image generation model can be obtained.

[0102] The method for obtaining a trained image generation model provided by an embodiment of the present application includes: obtaining sample data; wherein each group of sample data includes a sample coplanar cone beam CT image, a sample non-coplanar limited angle projection image, and an ideal non-coplanar cone beam CT image, the sample coplanar cone beam CT image and the sample non-coplanar limited angle projection image are model input parameters, and the ideal non-coplanar cone beam CT image is a calibration parameter; the image generation model and the discriminator are trained by the sample data, and the image generation model and the discriminator are optimized by a joint loss function until the preset conditions are met to obtain a trained image generation model; wherein the discriminator is used to distinguish between the non-coplanar cone beam CT image generated by the image generation model and the ideal non-coplanar cone beam CT image. The present application is based on the clinical needs of non-coplanar radiotherapy, obtains sample data, and trains an image generation model that generates non-coplanar cone beam CT images based on coplanar cone beam CT images and non-coplanar limited angle projection images, thereby obtaining a trained image generation model with higher stability and accuracy, and then the reconstructed non-coplanar cone beam CT images can be quickly obtained.

[0103] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0104] Figure 5 The schematic diagram of the structure of a non-coplanar cone beam CT image reconstruction device provided in an embodiment of the present application. Figure 5 As shown, the non-coplanar cone-beam CT image reconstruction device 50 provided in the embodiment of the present application includes an acquisition unit 501 and a reconstruction unit 502 .

[0105] Among them, the acquisition unit 501 is used to acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle.

[0106] The reconstruction unit 502 is used to input the coplanar cone beam CT image and the non-coplanar limited angle projection image into the trained image generation model, and output the non-coplanar cone beam CT image at the target angle; wherein the non-coplanar cone beam CT image is used to verify the position of the target part.

[0107] The device provided in this embodiment can be used to execute the method of the above embodiment. Its implementation principle and technical effect are similar and will not be described in detail here.

[0108] Figure 6 This is a schematic diagram of the structure of another non-coplanar cone beam CT image reconstruction device provided in an embodiment of the present application. Figure 6As shown, the non-coplanar cone-beam CT image reconstruction device 60 provided in the embodiment of the present application includes an acquisition unit 601 and a reconstruction unit 602 .

[0109] Among them, the acquisition unit 601 is used to acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle.

[0110] The reconstruction unit 602 is used to input the coplanar cone beam CT image and the non-coplanar limited angle projection image into the trained image generation model, and output the non-coplanar cone beam CT image at the target angle.

[0111] In an example, the acquisition unit 601 includes a first acquisition module 6011 , a rotation module 6012 , a determination module 6013 , and a second acquisition module 6014 .

[0112] The first acquisition module 6011 is used to acquire a coplanar cone beam CT image of a target part of a target object before the target device rotates.

[0113] The rotation module 6012 is used to control the target device to rotate to a target angle after obtaining the coplanar cone beam CT image, and determine the angle range based on the angle after the target device is rotated.

[0114] The determination module 6013 is used to determine multiple scanning angles based on the angle range and the preset angle value method.

[0115] The second acquisition module 6014 is used to acquire a non-coplanar limited-angle projection image corresponding to the scanning angle based on the scanning parameters corresponding to the scanning angle.

[0116] In one example, an image generation model includes an encoder and a decoder. The encoder has a dual-branch structure, which is used to extract image features of coplanar cone beam CT images and non-coplanar limited angle projection images respectively. The decoder is used to generate non-coplanar cone beam CT images based on the extracted image features.

[0117] In one example, the device 60 further includes a model unit 600 , and the model unit 600 includes a data acquisition module 6001 and a model training module 6002 .

[0118] The data acquisition module 6001 is used to acquire sample data; wherein each set of sample data includes a sample coplanar cone beam CT image, a sample non-coplanar finite angle projection image, and an ideal non-coplanar cone beam CT image, the sample coplanar cone beam CT image and the sample non-coplanar finite angle projection image are model input parameters, and the ideal non-coplanar cone beam CT image is a calibration parameter;

[0119] The model training module 6002 is used to train the image generation model and the discriminator through sample data, and optimize the image generation model and the discriminator using a joint loss function until the preset conditions are met to obtain a trained image generation model; wherein the discriminator is used to distinguish between the non-coplanar cone beam CT image generated by the image generation model and the ideal non-coplanar cone beam CT image.

[0120] In one example, the data acquisition module 6001 includes a first processing module 60011 , a second processing module 60012 , and a third processing module 60013 .

[0121] The first processing module 60011 is used to obtain multiple sample coplanar cone-beam CT images.

[0122] The second processing module 60012 is used to rotate the sample coplanar cone beam CT image to a preset angle, obtain a non-coplanar cone beam CT image at the preset angle, and determine that the non-coplanar cone beam CT image at the preset angle is an ideal non-coplanar cone beam CT image.

[0123] The third processing module 60013 is used to introduce rotation error and translation error into the ideal non-coplanar cone beam CT image, and based on a preset angle, forward project the ideal non-coplanar cone beam CT image after the error is introduced to obtain a sample non-coplanar limited angle projection image.

[0124] In one example, the third processing module 60013 is specifically used to determine the angle range of the sample projection image based on a preset angle; and, based on the angle range of the sample projection image and the preset angle value method, determine the sample scanning angle; and, based on the sample scanning parameters corresponding to the sample scanning angle, forward project the ideal non-coplanar cone beam CT image after the error is introduced to obtain a sample non-coplanar limited angle projection image corresponding to the sample scanning angle.

[0125] In one example, the joint loss function includes adversarial loss, absolute value loss, and bone structure loss.

[0126] The device provided in this embodiment can be used to execute the method of the above embodiment. Its implementation principle and technical effect are similar and will not be described in detail here.

[0127] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, it can also be stored in the memory of the above device in the form of program code, and the functions of the above data processing modules can be called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0128] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 includes: a processor 701, and a memory 702 communicatively connected to the processor.

[0129] The memory 702 stores computer-executable instructions; the processor 701 executes the computer-executable instructions stored in the memory 702 to implement any of the aforementioned methods.

[0130] In the specific implementation of the above electronic device, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The method disclosed in the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor for execution.

[0131] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any of the methods described above.

[0132] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to computer instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0133] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement any of the aforementioned methods when executed by a processor.

[0134] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0135] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for reconstructing non-coplanar cone-beam CT images, characterized in that: The method comprises: Acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle; The coplanar cone beam CT image and the non-coplanar limited angle projection image are input into a trained image generation model, and a non-coplanar cone beam CT image at the target angle is output.

2. The method according to claim 1, characterized in that The step of acquiring a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object comprises: Before the target device is rotated, a coplanar cone beam CT image of the target portion of the target object is acquired; After obtaining the coplanar cone beam CT image, controlling the target device to rotate to a target angle, and determining a scanning angle range of the imaging device based on the angle after the target device is rotated; Determine a plurality of scanning angles based on the scanning angle range and the preset angle value method; Based on preset scanning parameters corresponding to the scanning angle, a non-coplanar limited-angle projection image corresponding to the scanning angle is acquired.

3. The method according to claim 1, characterized in that The image generation model includes an encoder and a decoder. The encoder is a dual-branch structure, which is used to extract image features of the coplanar cone beam CT image and the non-coplanar limited angle projection image respectively. The decoder is used to generate the non-coplanar cone beam CT image based on the extracted image features.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Acquire sample data; wherein each set of sample data includes a sample coplanar cone beam CT image, a sample non-coplanar finite angle projection image, and an ideal non-coplanar cone beam CT image, wherein the sample coplanar cone beam CT image and the sample non-coplanar finite angle projection image are model input parameters, and the ideal non-coplanar cone beam CT image is a calibration parameter; The image generation model and the discriminator are trained by the sample data, and the image generation model and the discriminator are optimized by using a joint loss function until preset conditions are met, thereby obtaining a trained image generation model; wherein the discriminator is used to distinguish between the non-coplanar cone beam CT image generated by the image generation model and the ideal non-coplanar cone beam CT image.

5. The method according to claim 4, characterized in that The obtaining of sample data comprises: Acquire multiple sample coplanar cone beam CT images; Rotating the sample coplanar cone beam CT image to a preset angle to obtain a non-coplanar cone beam CT image at the preset angle, and determining that the non-coplanar cone beam CT image at the preset angle is an ideal non-coplanar cone beam CT image; The rotation error and the translation error are introduced into the ideal non-coplanar cone beam CT image, and based on the preset angle, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to obtain a sample non-coplanar limited angle projection image.

6. The method according to claim 5, characterized in that Based on the preset angle, forward projecting the ideal non-coplanar cone beam CT image after the error is introduced to obtain a sample non-coplanar limited angle projection image, including: Based on the preset angle, determining an angle range of the sample projection image; Determining a plurality of sample scanning angles based on the angle range of the sample projection image and a preset angle value method; Based on the preset scanning parameters corresponding to the sample scanning angle, the ideal non-coplanar cone beam CT image after the error is introduced is forward projected to obtain a sample non-coplanar limited angle projection image corresponding to the sample scanning angle.

7. The method according to claim 4, characterized in that The joint loss function includes adversarial loss, absolute value loss and bone structure loss.

8. A device for reconstructing non-coplanar cone-beam CT images, characterized in that: The device comprises: An acquisition unit, configured to acquire a coplanar cone beam CT image and a non-coplanar limited angle projection image of a target part of a target object; wherein the target object is located on a target device, the coplanar cone beam CT image is a three-dimensional image acquired before the target device is rotated, and the non-coplanar limited angle projection image is a two-dimensional projection image acquired after the target device is rotated to a target angle; The reconstruction unit is used to input the coplanar cone beam CT image and the non-coplanar limited angle projection image into a trained image generation model, and output a non-coplanar cone beam CT image at the target angle.

9. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images