Radiotherapy CBCT sparse sampling image enhancement method and system

Abstract

The invention relates to the technical field of medical image processing, in particular to a radiotherapy CBCT sparse sampling image enhancement method and system, and the method comprises data collection, deep learning model construction, model training and image enhancement. Compared with the prior art that two-dimensional CBCT projection data obtained after spiral CT data conversion is directly adopted for three-dimensional reconstruction, noise interference is large, and image quality is not high, the scheme adopts an over-complete binary wavelet transform technology, an image is decomposed into multiple layers, general picture coefficients and direction sub-bands are extracted, and the image quality is improved. According to the scheme, the influence domain operator is calculated, regular and irregular data are classified and processed, Wiener filtering is applied to irregular and lowest decomposition level data, other data are kept unchanged, and finally the image is reconstructed through wavelet inverse transformation, so that the noise is effectively reduced, the definition and accuracy of the two-dimensional CBCT projection data are remarkably improved, and the method is suitable for large-scale popularization and application. A solid foundation is laid for subsequent three-dimensional reconstruction and image enhancement, and the diagnosis precision and the treatment efficiency of radiotherapy images are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for enhancing sparse sampling images of radiotherapy CBCT. Background Art

[0002] Cone beam computed tomography (CBCT) has become an important imaging technology in the field of radiotherapy. During radiotherapy, CBCT is widely used in image-guided treatment, allowing doctors to obtain three-dimensional images of tumors and surrounding organs in patients, so as to accurately locate and plan before treatment, and monitor the position changes of tumors and organs in real time during treatment to ensure the accuracy and safety of radiotherapy. Especially in tumor treatment centers, the use of CBCT is crucial to achieve accurate radiotherapy.

[0003] Although CBCT has significant advantages in radiotherapy, there are still some problems in directly using the two-dimensional CBCT projection data converted from spiral CT data for three-dimensional reconstruction. Among them, the most important problems are large noise interference and low image quality. This is mainly due to the fact that various factors (such as equipment accuracy, environmental noise, etc.) may lead to a decrease in image quality and an increase in noise during data conversion and transmission. These problems not only affect the clarity of CBCT images, but may also have an adverse impact on the formulation and implementation of radiotherapy plans.

[0004] In order to overcome the problems of noise interference and low image quality, existing technologies usually adopt some image enhancement or filtering methods. However, these methods often have certain limitations. For example, some traditional filtering methods may not be able to effectively remove noise, and may also lose image detail information. Although some advanced image enhancement methods can improve the clarity of the image, they may require complex calculation processes and may not work well for different types of noise and image features. Summary of the invention

[0005] In order to overcome the problems raised in the above background technology, the present invention proposes a method and system for radiotherapy CBCT sparse sampling image enhancement.

[0006] The technical solution of the present invention is: a method for enhancing sparse sampling images of radiotherapy CBCT, comprising the following steps:

[0007] S11: Data collection, first collect the patient's spiral CT image data, then convert the spiral CT data into multi-angle two-dimensional CBCT projection data, then split the two-dimensional CBCT projection data into sparse sampling and full sampling data, and finally use the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data, forming paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data;

[0008] S12: deep learning model construction, constructing a deep learning network model, wherein the constructed deep learning network model is a residual dense generative adversarial network;

[0009] S13: Model training, using paired sparsely sampled CBCT reconstruction data and fully sampled CBCT reconstruction data to pre-train the deep learning model;

[0010] S14: Image enhancement: The sparsely sampled CBCT 3D reconstructed image is used as input, and the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network.

[0011] Preferably, after the spiral CT data is converted into multi-angle two-dimensional CBCT projection data, the two-dimensional CBCT projection data is further subjected to noise reduction processing, wherein the specific steps of the noise reduction processing include:

[0012] S21: Perform an overcomplete dyadic wavelet transform on the two-dimensional CBCT projection data, first decompose the image into J layers, and then obtain the overview coefficient, horizontal sub-band and vertical sub-band of each layer;

[0013] S22: Calculate the influence domain operator of each decomposition level according to the following formula:

[0014]

[0015] Among them, f(x,y) is the original two-dimensional CBCT projection data, is the influence domain of the corresponding point; is the vertical subband, is the horizontal subband, (x 0 ,y 0 ) is the center point of the original two-dimensional CBCT projection data f(x,y), is the influence domain operator of the j-th layer image;

[0016] S23: classifying the horizontal sub-bands and the vertical sub-bands in each decomposition level, and dividing the horizontal sub-bands and the vertical sub-bands into regular data and irregular data;

[0017] S24: For irregular data, Wiener filtering is used for processing. For regular data, Wiener filtering is used for data at the lowest decomposition level, and no processing is performed on other data.

[0018] S25: Perform inverse wavelet transform on the overview coefficients and the processed sub-band coefficients.

[0019] Preferably, when performing an overcomplete dyadic wavelet transform on the two-dimensional CBCT projection data, the following steps are included:

[0020] S31: Decompose each two-dimensional CBCT projection data f(x, y) into J layers, wherein the decomposed image is represented by the following formula:

[0021]

[0022] in, is the profile coefficient, is the vertical subband, is the horizontal subband;

[0023] S32: Calculate the profile coefficient, the horizontal sub-band, and the vertical sub-band using the following formula:

[0024] Profile coefficient:

[0025] Vertical sub-band:

[0026] Horizontal sub-band:

[0027] Among them, * indicates that the wavelet coefficient matrix is ​​convolved with the filter row by row and column by column, G is a high-pass filter, H is a low-pass filter, and D is a Dirac filter.

[0028] Preferably, when the horizontal sub-bands and the vertical sub-bands in each decomposition level are classified into regular data and irregular data, the classification is performed by the following formula:

[0029]

[0030] Wherein, α is the signal Li index, and α is greater than or equal to 0; wherein, the horizontal sub-band and the vertical sub-band that satisfy the above formula are classified as regular data, otherwise they are classified as irregular data.

[0031] Preferably, when using the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data to form paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data, the principle formula of the CBCT three-dimensional reconstruction algorithm used is:

[0032] f(x,y,z)=∫ θ ∫ R P(θ,r)·h(rR(θ,x,y,z))drdθ;

[0033] Among them, f(x,y,z) is the three-dimensional tomographic image data, P(θ,r) is the two-dimensional projection data, h(r) is the filter function, R(θ,x,y,z) is the mapping relationship between the projection ray and the three-dimensional space, r represents the sub-variable, which is used to traverse all positions in the projection data, and θ represents the projection angle, that is, the rotation angle of the object relative to the projection plane.

[0034] Preferably, when constructing a deep learning network model, the structure of the constructed deep learning network model is:

[0035] A11: Residual dense network, including shallow feature extraction network, residual dense block, and dense feature fusion and upsampling network, is used to receive input image data, extract and fuse features through the residual dense network structure, and generate high-quality CBCT images;

[0036] A12: Adversarial network, which is a binary classifier used to determine whether the input image is a real CBCT image or a fake image generated by the residual dense network.

[0037] Preferably, the residual dense blocks in the residual dense network are set to multiple groups, and the multiple groups of residual dense blocks all use all feature maps of the previous residual dense block as input, that is, the output of the previous residual dense block directly reaches the next residual dense block, and the residual dense block includes a dense connection layer, a local feature fusion module and a local residual learning module, wherein the dense connection layer is a group of densely connected convolutional layers, the local feature fusion module has a Contat layer and a convolutional layer with a convolution kernel size of 1×1, and the local residual learning module is used to add the input feature map to the fused feature map to form a residual connection, wherein the loss function of the residual dense network is:

[0038] L G =λ content *L content +λ adv *L adv ;

[0039] Among them, L G represents the total loss of the residual dense network, λ content and λ adv is the weight coefficient used to balance the content loss and adversarial loss, L content represents the content loss, which is calculated by comparing the distance between the generated image and the real image in the feature space extracted by the VGG network. adv Denotes the adversarial loss, where the adversarial loss L adv The calculation is done by the following formula:

[0040] L adv =-E z [log(D(G(z)))];

[0041] Where z represents noise, E z represents the expectation of noise z, G(z) represents the generated image, and D(G(z)) represents the discriminant probability of the adversarial network for the generated image G(z).

[0042] Preferably, the adversarial network is composed of multiple convolutional layers, batch normalization layers, activation layers and fully connected layers, wherein the convolutional layer is used to extract image features, the batch normalization layer is used to normalize the output of the convolutional layer so that the distribution of the output data remains within a stable range, the activation layer is used to introduce nonlinear transformations so that the model can capture more complex features, and the fully connected layer is located at the last part of the adversarial network, which is used to flatten the feature map output by the convolutional layer into a vector and classify it through multiple fully connected layers, wherein the loss function of the adversarial network is:

[0043] L D =-E x [log(D(x))]-E z [log(1-D(G(z)))];

[0044] Among them, z represents noise, x represents the real image, G(z) represents the generated image, and L D represents the total loss of the adversarial network, E x represents the expectation of the real image x, E z represents the expectation of noise z, D(x) represents the discrimination probability of the adversarial network for the real image x, and D(G(z)) represents the discrimination probability of the adversarial network for the generated image.

[0045] Preferably, after taking the sparsely sampled CBCT 3D reconstructed image as input, the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network, and the enhanced image is registered with the image before the enhancement, wherein the principle formula is:

[0046] T = arg max TI(A, B(T));

[0047] Where A represents the image before enhancement, B represents the image after enhancement, T represents the spatial transformation relationship, B(T) represents the result of image B after spatial transformation T, and I(A,B) is the shared information between image A and image B, that is, mutual information. The mutual information I(A,B) is calculated by the following formula:

[0048]

[0049] Among them, p(a,b) represents the joint probability distribution of image A and image B, that is, the probability that the pixel value in image A is a and the corresponding pixel value in image B is b, p(a) is the edge probability distribution of image A, that is, the probability that the pixel value in image A is a, and p(b) is the edge probability distribution of image B, that is, the probability that the pixel value in image B is b.

[0050] The system of the radiotherapy CBCT sparse sampling image enhancement method comprises:

[0051] A data input module is used to collect the patient's spiral CT image data;

[0052] A data preprocessing module, used for preprocessing the collected spiral CT image data of the patient, wherein the preprocessing operation adopted includes converting the spiral CT data into multi-angle two-dimensional CBCT projection data by using a forward projection algorithm, performing noise reduction processing on the obtained two-dimensional CBCT projection data, and splitting the two-dimensional CBCT projection data into paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data;

[0053] A deep learning model processing module is used to construct a deep learning network model, and use paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data to train the deep learning model, and then use the trained deep learning network model to perform image enhancement processing on the sparse sampling CBCT three-dimensional reconstruction data;

[0054] The image output module is used to post-process and output the enhanced sparse sampling CBCT three-dimensional reconstruction data.

[0055] Beneficial effects of the present invention:

[0056] 1. Compared with the prior art that directly uses the two-dimensional CBCT projection data converted from spiral CT data for three-dimensional reconstruction, there are disadvantages of large noise interference and low image quality. This scheme adds a noise reduction processing step based on the converted two-dimensional CBCT projection data. Specifically, this scheme uses the overcomplete dyadic wavelet transform technology to decompose the image into multiple layers and extract the overview coefficients and directional subbands. By calculating the influence domain operator and classifying and processing regular and irregular data, Wiener filtering is applied to irregular classes and lowest decomposition level data, while keeping other data unchanged. Finally, the image is reconstructed by inverse wavelet transform. This scheme effectively reduces noise and significantly improves the clarity and accuracy of two-dimensional CBCT projection data, laying a solid foundation for subsequent three-dimensional reconstruction and image enhancement, thereby improving the diagnostic accuracy and treatment efficiency of radiotherapy images.

[0057] 2. By constructing a deep learning model including residual dense network and adversarial network, efficient feature extraction and fusion of input image data are achieved, and high-quality CBCT images can be generated. Among them, the residual dense network enhances the reuse and transmission of features and improves the quality of image generation; and the introduction of the adversarial network further promotes the fidelity and detail expression of the generated images by distinguishing between real and generated images, and improves the overall generation effect of CBCT images;

[0058] 3. Compared with the existing technology that directly uses sparsely sampled CBCT 3D reconstructed images, which has the disadvantages of low image quality, insufficient detail expression and lack of effective registration means, this scheme takes the sparsely sampled CBCT 3D reconstructed image as input, and first performs image enhancement processing on it through the residual dense generative adversarial network that has been pre-trained and optimized, which significantly improves the image quality. Subsequently, this scheme uses mutual information as the registration criterion to accurately register the enhanced image with the original image to ensure high consistency and spatial correspondence between the images. This scheme not only overcomes the shortcomings of the existing technology in image quality and detail expression, but also effectively solves the problem of image registration, providing more accurate and reliable image information for subsequent medical applications, and significantly improving the overall technical effect and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 What is shown is a flow chart of the method for radiotherapy CBCT sparse sampling image enhancement of the present invention;

[0060] Figure 2 Shown is a schematic diagram of the system structure of the method for radiotherapy CBCT sparse sampling image enhancement of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0062] See also Figure 1 The present invention provides an embodiment: a method for enhancing sparse sampling images of radiotherapy CBCT, comprising the following steps:

[0063] S11: Data collection, first collect the patient's spiral CT image data, then convert the spiral CT data into multi-angle two-dimensional CBCT projection data, then split the two-dimensional CBCT projection data into sparse sampling and full sampling data, and finally use the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data, forming paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data;

[0064] S12: deep learning model construction, constructing a deep learning network model, wherein the constructed deep learning network model is a residual dense generative adversarial network;

[0065] S13: Model training, using paired sparsely sampled CBCT reconstruction data and fully sampled CBCT reconstruction data to pre-train the deep learning model;

[0066] S14: Image enhancement: The sparsely sampled CBCT 3D reconstructed image is used as input, and the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network.

[0067] As described above, the patient's spiral CT data is first collected and converted into multi-angle two-dimensional CBCT projections. These projection data are then split into two groups, sparse and fully sampled, and reconstructed into three-dimensional tomographic images to form paired data sets. Next, a residual dense generative adversarial network is constructed as a deep learning model and pre-trained using the above paired data sets. Finally, the sparsely sampled CBCT three-dimensional reconstructed images are input into the trained network for image enhancement processing.

[0068] As described above, the present invention collects and converts the patient's spiral CT image data into multi-angle two-dimensional CBCT projection data, and splits it into sparse sampling and full sampling data for three-dimensional reconstruction, and then uses the residual dense generative adversarial network to build and train a deep learning model. Ultimately, it achieves effective enhancement of sparse sampling CBCT images, significantly improves image quality, and provides more accurate support for image diagnosis and treatment during radiotherapy.

[0069] Preferably, after the spiral CT data is converted into multi-angle two-dimensional CBCT projection data, the two-dimensional CBCT projection data is further subjected to noise reduction processing, wherein the specific steps of the noise reduction processing include:

[0070] S21: Perform an overcomplete dyadic wavelet transform on the two-dimensional CBCT projection data, first decompose the image into J layers, and then obtain the overview coefficient, horizontal sub-band and vertical sub-band of each layer;

[0071] S22: Calculate the influence domain operator of each decomposition level according to the following formula:

[0072]

[0073] Among them, f(x,y) is the original two-dimensional CBCT projection data, is the influence domain of the corresponding point; is the vertical subband, is the horizontal subband, (x 0 ,y0 ) is the center point of the original two-dimensional CBCT projection data f(x,y), is the influence domain operator of the j-th layer image;

[0074] S23: classifying the horizontal sub-bands and the vertical sub-bands in each decomposition level, and dividing the horizontal sub-bands and the vertical sub-bands into regular data and irregular data;

[0075] S24: For irregular data, Wiener filtering is used for processing. For regular data, Wiener filtering is used for data at the lowest decomposition level, and no processing is performed on other data.

[0076] S25: Perform inverse wavelet transform on the overview coefficients and the processed sub-band coefficients.

[0077] Preferably, when performing an overcomplete dyadic wavelet transform on the two-dimensional CBCT projection data, the following steps are included:

[0078] S31: Decompose each two-dimensional CBCT projection data f(x, y) into J layers, wherein the decomposed image is represented by the following formula:

[0079]

[0080] in, is the profile coefficient, is the vertical subband, is the horizontal subband;

[0081] S32: Calculate the profile coefficient, the horizontal sub-band, and the vertical sub-band using the following formula:

[0082] Profile coefficient:

[0083] Vertical sub-band:

[0084] Horizontal sub-band:

[0085] Among them, * indicates that the wavelet coefficient matrix is ​​convolved with the filter row by row and column by column, G is a high-pass filter, H is a low-pass filter, and D is a Dirac filter.

[0086] Preferably, when the horizontal sub-bands and the vertical sub-bands in each decomposition level are classified into regular data and irregular data, the classification is performed by the following formula:

[0087]

[0088] Wherein, α is the signal Li index, and α is greater than or equal to 0; wherein, the horizontal sub-band and the vertical sub-band that satisfy the above formula are classified as regular data, otherwise they are classified as irregular data.

[0089] Preferably, when using the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data to form paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data, the principle formula of the CBCT three-dimensional reconstruction algorithm used is:

[0090] f(x,y,z)=∫ θ ∫ R P(θ,r)·h(rR(θ,x,y,z))drdθ;

[0091] Among them, f(x,y,z) is the three-dimensional tomographic image data, P(θ,r) is the two-dimensional projection data, h(r) is the filter function, R(θ,x,y,z) is the mapping relationship between the projection ray and the three-dimensional space, r represents the sub-variable, which is used to traverse all positions in the projection data, and θ represents the projection angle, that is, the rotation angle of the object relative to the projection plane.

[0092] As described above, the two-dimensional CBCT projection data is first decomposed into multiple layers by complete binary wavelet transform, and the overview coefficients, horizontal and vertical subbands of each layer are obtained. Next, the influence domain operator of each layer is calculated for subsequent processing. Then, the horizontal and vertical subbands are classified according to the Li index formula to distinguish regular and irregular data. For irregular data, Wiener filtering is used; Wiener filtering is also applied to the lowest decomposition level in regular data, and the rest of the data remains unchanged. Finally, the processed overview coefficients and subband coefficients are reconstructed by inverse wavelet transform. In this process, it is also involved to reconstruct the two-dimensional projection data into three-dimensional tomographic image data using the CBCT three-dimensional reconstruction algorithm, forming two reconstructed data sets of sparse sampling and full sampling. The reconstruction process is based on the principle of filtered back projection and is realized by the mapping relationship between the filter function and the projection ray and the three-dimensional space.

[0093] Compared with the prior art that directly uses the two-dimensional CBCT projection data converted from spiral CT data for three-dimensional reconstruction, which has the disadvantages of large noise interference and low image quality, this scheme adds a noise reduction processing step based on the converted two-dimensional CBCT projection data. Specifically, this scheme uses over-complete binary wavelet transform technology to decompose the image into multiple layers and extract overview coefficients and directional sub-bands. By calculating the influence domain operator and classifying and processing regular and irregular data, Wiener filtering is applied to irregular classes and lowest decomposition level data, while keeping other data unchanged. Finally, the image is reconstructed through inverse wavelet transform. This scheme effectively reduces noise and significantly improves the clarity and accuracy of two-dimensional CBCT projection data, laying a solid foundation for subsequent three-dimensional reconstruction and image enhancement, thereby improving the diagnostic accuracy and treatment efficiency of radiotherapy images.

[0094] Preferably, when constructing a deep learning network model, the structure of the constructed deep learning network model is:

[0095] A11: Residual dense network, including shallow feature extraction network, residual dense block, and dense feature fusion and upsampling network, is used to receive input image data, extract and fuse features through the residual dense network structure, and generate high-quality CBCT images;

[0096] A12: Adversarial network, which is a binary classifier used to determine whether the input image is a real CBCT image or a fake image generated by the residual dense network.

[0097] As described above, the present invention realizes efficient feature extraction and fusion of input image data by constructing a deep learning model including a residual dense network and an adversarial network, and can generate high-quality CBCT images. Among them, the residual dense network enhances the reuse and transmission of features and improves the quality of image generation; and the introduction of the adversarial network further promotes the fidelity and detail expression of the generated image by distinguishing between real and generated images, and improves the overall generation effect of CBCT images.

[0098] Preferably, the residual dense blocks in the residual dense network are set to multiple groups, and the multiple groups of residual dense blocks all use all feature maps of the previous residual dense block as input, that is, the output of the previous residual dense block directly reaches the next residual dense block, and the residual dense block includes a dense connection layer, a local feature fusion module and a local residual learning module, wherein the dense connection layer is a group of densely connected convolutional layers, the local feature fusion module has a Contat layer and a convolutional layer with a convolution kernel size of 1×1, and the local residual learning module is used to add the input feature map to the fused feature map to form a residual connection, wherein the loss function of the residual dense network is:

[0099] L G =λcontent *L content +λ adv *L adv ;

[0100] Among them, L G represents the total loss of the residual dense network, λ content and λ adv is the weight coefficient used to balance the content loss and adversarial loss, L content represents the content loss, which is calculated by comparing the distance between the generated image and the real image in the feature space extracted by the VGG network. adv Denotes the adversarial loss, where the adversarial loss L adv The calculation is done by the following formula:

[0101] L adv =-E z [log(D(G(z)))];

[0102] Where z represents noise, E z represents the expectation of noise z, G(z) represents the generated image, and D(G(z)) represents the discriminant probability of the adversarial network for the generated image G(z).

[0103] Preferably, the adversarial network is composed of multiple convolutional layers, batch normalization layers, activation layers and fully connected layers, wherein the convolutional layer is used to extract image features, the batch normalization layer is used to normalize the output of the convolutional layer so that the distribution of the output data remains within a stable range, the activation layer is used to introduce nonlinear transformations so that the model can capture more complex features, and the fully connected layer is located at the last part of the adversarial network, which is used to flatten the feature map output by the convolutional layer into a vector and classify it through multiple fully connected layers, wherein the loss function of the adversarial network is:

[0104] L D =-E x [log(D(x))]-E z [log(1-D(G(z)))];

[0105] Among them, z represents noise, x represents the real image, G(z) represents the generated image, and L D represents the total loss of the adversarial network, E x represents the expectation of the real image x, E z represents the expectation of noise z, D(x) represents the discrimination probability of the adversarial network for the real image x, and D(G(z)) represents the discrimination probability of the adversarial network for the generated image.

[0106] As described above, the present invention constructs a deep learning network model, which specifically includes two parts: a residual dense network and an adversarial network. The residual dense network uses multiple layers of densely connected convolutional layers to extract and fuse image features, improves feature expression capabilities through local feature fusion and residual learning, and generates high-quality CBCT images. At the same time, a specific loss function is used to optimize network performance by combining content loss and adversarial loss. The adversarial network acts as a binary classifier, which discriminates the authenticity of the input image through multiple layers of convolution, batch normalization, activation and fully connected layers, and cooperates with the residual dense network to train, further improving the fidelity and detail expression of the generated image.

[0107] Preferably, after taking the sparsely sampled CBCT 3D reconstructed image as input, the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network, and the enhanced image is registered with the image before the enhancement, wherein the principle formula is:

[0108] T = arg max TI(A, B(T));

[0109] Where A represents the image before enhancement, B represents the image after enhancement, T represents the spatial transformation relationship, B(T) represents the result of image B after spatial transformation T, and I(A,B) is the shared information between image A and image B, that is, mutual information. The mutual information I(A,B) is calculated by the following formula:

[0110]

[0111] Among them, p(a,b) represents the joint probability distribution of image A and image B, that is, the probability that the pixel value in image A is a and the corresponding pixel value in image B is b, p(a) is the edge probability distribution of image A, that is, the probability that the pixel value in image A is a, and p(b) is the edge probability distribution of image B, that is, the probability that the pixel value in image B is b.

[0112] As described above, compared with the prior art that directly uses sparsely sampled CBCT 3D reconstructed images, there are disadvantages such as low image quality, insufficient detail expression and lack of effective registration means. After taking the sparsely sampled CBCT 3D reconstructed image as input, this scheme first performs image enhancement processing on it through the residual dense generative adversarial network that has been pre-trained and optimized, which significantly improves the image quality. Subsequently, this scheme uses mutual information as the registration criterion to accurately register the enhanced image with the original image to ensure high consistency and spatial correspondence between the images. This scheme not only overcomes the shortcomings of the prior art in image quality and detail expression, but also effectively solves the problem of image registration, providing more accurate and reliable image information for subsequent medical applications, and significantly improving the overall technical effect and practicality.

[0113] See also Figure 2 The present invention provides an embodiment: a system for a method for enhancing sparse sampling images of radiotherapy CBCT, comprising:

[0114] A data input module is used to collect the patient's spiral CT image data;

[0115] A data preprocessing module, used for preprocessing the collected spiral CT image data of the patient, wherein the preprocessing operation adopted includes converting the spiral CT data into multi-angle two-dimensional CBCT projection data by using a forward projection algorithm, performing noise reduction processing on the obtained two-dimensional CBCT projection data, and splitting the two-dimensional CBCT projection data into paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data;

[0116] A deep learning model processing module is used to construct a deep learning network model, and use paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data to train the deep learning model, and then use the trained deep learning network model to perform image enhancement processing on the sparse sampling CBCT three-dimensional reconstruction data;

[0117] The image output module is used to post-process and output the enhanced sparse sampling CBCT three-dimensional reconstruction data.

[0118] As described above, the radiotherapy CBCT sparse sampling image enhancement system first collects the patient's spiral CT image data through the data input module, then uses the data preprocessing module to convert the CT data into multi-angle two-dimensional CBCT projection data, and then splits it into sparse sampling and full sampling data pairs after noise reduction. Then, the deep learning model processing module builds and trains the deep learning network model, and uses the trained model to enhance the sparse sampling CBCT three-dimensional reconstruction data. Finally, the image output module post-processes and outputs the enhanced image, thereby providing high-quality radiotherapy image support.

[0119] The radiotherapy CBCT sparse sampling image enhancement system collects spiral CT data through the data input module. The data preprocessing module converts it into multi-angle two-dimensional CBCT projection data and performs noise reduction, and splits it into sparse and full sampling data pairs. The deep learning model processing module uses these data pairs to train the model and then enhances the sparse sampling data. Finally, the image output module post-processes and outputs the enhanced data, effectively improving the quality of radiotherapy CBCT images and providing more accurate and reliable image support for the formulation and implementation of radiotherapy plans.

[0120] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for enhancing sparse sampling images of radiotherapy CBCT; characterized by: The following steps are included: S11: Data collection, first collect the patient's spiral CT image data, then convert the spiral CT data into multi-angle two-dimensional CBCT projection data, then split the two-dimensional CBCT projection data into sparse sampling and full sampling data, and finally use the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data, forming paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data; S12: deep learning model construction, constructing a deep learning network model, wherein the constructed deep learning network model is a residual dense generative adversarial network; S13: Model training, using paired sparsely sampled CBCT reconstruction data and fully sampled CBCT reconstruction data to pre-train the deep learning model; S14: Image enhancement: The sparsely sampled CBCT 3D reconstructed image is used as input, and the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network.

2. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 1, characterized in that: After the spiral CT data is converted into multi-angle two-dimensional CBCT projection data, the two-dimensional CBCT projection data is subjected to noise reduction processing, wherein the specific steps of the noise reduction processing include: S21: Perform an overcomplete dyadic wavelet transform on the two-dimensional CBCT projection data, first decompose the image into J layers, and then obtain the overview coefficient, horizontal sub-band and vertical sub-band of each layer; S22: Calculate the influence domain operator of each decomposition level according to the following formula: Among them, f(x,y) is the original two-dimensional CBCT projection data, is the influence domain of the corresponding point; is the vertical subband, is the horizontal subband, (x0, y0) is the center point of the original two-dimensional CBCT projection data f(x, y), is the influence domain operator of the j-th layer image; S23: classifying the horizontal sub-bands and the vertical sub-bands in each decomposition level, and dividing the horizontal sub-bands and the vertical sub-bands into regular data and irregular data; S24: For irregular data, Wiener filtering is used for processing. For regular data, Wiener filtering is used for data at the lowest decomposition level, and no processing is performed on other data. S25: Perform inverse wavelet transform on the overview coefficients and the processed sub-band coefficients.

3. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 2, characterized in that: When performing an over-complete dyadic wavelet transform on two-dimensional CBCT projection data, the following steps are included: S31: Decompose each two-dimensional CBCT projection data f(x, y) into J layers, wherein the decomposed image is represented by the following formula: in, is the profile coefficient, is the vertical subband, is the horizontal subband; S32: Calculate the profile coefficient, the horizontal sub-band, and the vertical sub-band using the following formula: Profile coefficient: Vertical sub-band: Horizontal sub-band: Among them, * indicates that the wavelet coefficient matrix is ​​convolved with the filter row by row and column by column, G is a high-pass filter, H is a low-pass filter, and D is a Dirac filter.

4. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 3, characterized in that: When the horizontal sub-bands and vertical sub-bands in each decomposition level are classified into regular data and irregular data, the classification is performed using the following formula: Wherein, α is the signal Li index, and α is greater than or equal to 0; wherein, the horizontal sub-band and the vertical sub-band that satisfy the above formula are classified as regular data, otherwise they are classified as irregular data.

5. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 4, characterized in that: When using the CBCT three-dimensional reconstruction algorithm to reconstruct the split two-dimensional projection data into three-dimensional tomographic image data to form paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data, the principle formula of the CBCT three-dimensional reconstruction algorithm used is: f(x,y,z)=∫ θ ∫ R P(θ,r)·h(r-R(θ,x,y,z))drdθ; Among them, f(x,y,z) is the three-dimensional tomographic image data, P(θ,r) is the two-dimensional projection data, h(r) is the filter function, R(θ,x,y,z) is the mapping relationship between the projection ray and the three-dimensional space, r represents the sub-variable, which is used to traverse all positions in the projection data, and θ represents the projection angle, that is, the rotation angle of the object relative to the projection plane.

6. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 5, characterized in that: When constructing a deep learning network model, the structure of the constructed deep learning network model is: A11: Residual dense network, including shallow feature extraction network, residual dense block, and dense feature fusion and upsampling network, is used to receive input image data, extract and fuse features through the residual dense network structure, and generate high-quality CBCT images; A12: Adversarial network, which is a binary classifier used to determine whether the input image is a real CBCT image or a fake image generated by the residual dense network.

7. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 6, characterized in that: The residual dense blocks in the residual dense network are set to multiple groups, and the multiple groups of residual dense blocks all take all the feature maps of the previous residual dense blocks as input, that is, the output of the previous residual dense block directly reaches the next residual dense block, and the residual dense block includes a dense connection layer, a local feature fusion module and a local residual learning module, wherein the dense connection layer is a group of densely connected convolutional layers, the local feature fusion module has a Contat layer and a convolutional layer with a convolution kernel size of 1×1, and the local residual learning module is used to add the input feature map to the fused feature map to form a residual connection, wherein the loss function of the residual dense network is: L G =λ content *L content +λ adv *L adv ; Among them, L G represents the total loss of the residual dense network, λ content and λ adv is the weight coefficient used to balance the content loss and adversarial loss, L content represents the content loss, which is calculated by comparing the distance between the generated image and the real image in the feature space extracted by the VGG network. adv Denotes the adversarial loss, where the adversarial loss L adv The calculation is done by the following formula: L adv =-E z [log(D(G(z)))]; Where z represents noise, E z represents the expectation of noise z, G(z) represents the generated image, and D(G(z)) represents the discriminant probability of the adversarial network for the generated image G(z).

8. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 7, characterized in that: The adversarial network consists of multiple convolutional layers, batch normalization layers, activation layers, and fully connected layers. The convolutional layer is used to extract image features, the batch normalization layer is used to normalize the output of the convolutional layer to keep the distribution of the output data within a stable range, the activation layer is used to introduce nonlinear transformations so that the model can capture more complex features, and the fully connected layer is located at the end of the adversarial network. It is used to flatten the feature map output by the convolutional layer into a vector and classify it through multiple fully connected layers. The loss function of the adversarial network is: L D =-E x [log(D(x))]-E z [log(1-D(G(z)))]; Among them, z represents noise, x represents the real image, G(z) represents the generated image, and L D represents the total loss of the adversarial network, E x represents the expectation of the real image x, E z represents the expectation of noise z, D(x) represents the discrimination probability of the adversarial network for the real image x, and D(G(z)) represents the discrimination probability of the adversarial network for the generated image.

9. The method for radiotherapy CBCT sparse sampling image enhancement according to claim 8, characterized in that: After taking the sparsely sampled CBCT 3D reconstructed image as input, the input image is enhanced by the pre-trained and optimized residual dense generative adversarial network, and the enhanced image is registered with the image before enhancement. The principle formula is: T = arg max TI(A, B(T)); Where A represents the image before enhancement, B represents the image after enhancement, T represents the spatial transformation relationship, B(T) represents the result of image B after spatial transformation T, and I(A,B) is the shared information between image A and image B, that is, mutual information. The mutual information I(A,B) is calculated by the following formula: Among them, p(a,b) represents the joint probability distribution of image A and image B, that is, the probability that the pixel value in image A is a and the corresponding pixel value in image B is b, p(a) is the edge probability distribution of image A, that is, the probability that the pixel value in image A is a, and p(b) is the edge probability distribution of image B, that is, the probability that the pixel value in image B is b.

10. A system for a method for enhancing sparse sampling images of radiotherapy CBCT, used to implement the method for enhancing sparse sampling images of radiotherapy CBCT according to any one of claims 1 to 9, characterized in that: Included are: A data input module is used to collect the patient's spiral CT image data; A data preprocessing module, used for preprocessing the collected spiral CT image data of the patient, wherein the preprocessing operation adopted includes converting the spiral CT data into multi-angle two-dimensional CBCT projection data by using a forward projection algorithm, performing noise reduction processing on the obtained two-dimensional CBCT projection data, and splitting the two-dimensional CBCT projection data into paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data; A deep learning model processing module is used to construct a deep learning network model, and use paired sparse sampling CBCT reconstruction data and full sampling CBCT reconstruction data to train the deep learning model, and then use the trained deep learning network model to perform image enhancement processing on the sparse sampling CBCT three-dimensional reconstruction data; The image output module is used to post-process and output the enhanced sparse sampling CBCT three-dimensional reconstruction data.

Images