Result simulation method, device and equipment based on generative adversarial network

Abstract

The embodiment of the invention provides a result simulation method, device and equipment based on a generative adversarial network, and relates to the field of image processing. The method comprises the following steps: inputting a CT image and a proton beam parameter into a trained generative adversarial network; after a CT image is coded and decoded by the primary generator, a basic tissue deformation field is obtained; after a basic organization deformation field and a Bragg peak distribution diagram obtained based on proton beam parameters are subjected to feature fusion and attention weight mapping of the fine tuning generator, an enhanced image and an output feature vector are obtained; the output feature vector passes through a spatial feature discrimination branch in the dual-channel discriminator to obtain an image authenticity result, and the output feature vector passes through a dosimetry feature discrimination branch in the dual-channel discriminator to obtain a dosimetry evaluation result; and judging the enhanced image according to an image authenticity result and a dosimetry evaluation result to obtain an output image of the trained generative adversarial network. According to the embodiment provided by the invention, the prediction accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a result simulation method based on a generative adversarial network, a result simulation device based on a generative adversarial network, and an electronic device. Background Art

[0002] Current proton therapy plans are often designed and implemented based on static pre-treatment CT images. However, during actual treatment, the position, geometry, and biological characteristics of the tumor and surrounding normal tissues and organs undergo dynamic changes. For example, physiological activities such as breathing, swallowing, and heartbeat, as well as tumor regression or growth during treatment, all lead to real-time changes in anatomical structures. If irradiation is continued according to the initial static treatment plan, it is inevitable that off-target effects (i.e., the proton beam fails to accurately irradiate the tumor) and the radiation dose to surrounding normal tissues and organs will be increased, thereby reducing the efficacy of tumor treatment and increasing the risk of toxic side effects of radiotherapy.

[0003] In proton radiotherapy, the dose conformity and limited range of the proton beam make it extremely sensitive to changes in anatomical structure. Any slight change, such as a reduction in tumor volume or displacement of normal tissue, can lead to significant changes in the dose distribution. For example, a reduction in tumor volume may result in insufficient penetration depth of the originally planned proton beam, thereby affecting tumor control rate; at the same time, displacement of normal tissue may lead to unexpected high-dose irradiation, increasing the risk of side effects. Therefore, adaptive radiotherapy for proton therapy has become an inevitable demand in clinical practice to adjust the treatment plan in real time to adapt to the dynamic changes in the patient's anatomical structure.

[0004] The accurate position of the proton Bragg peak is the key to achieving precise treatment, but its precise positioning within the patient's body is affected by the uncertainty of the proton range. The uncertainty of the proton range mainly comes from the impact of CT imaging artifacts on tissue density. Artifacts in CT images (such as metal artifacts, motion artifacts, etc.) will destroy the accurate measurement of Hounsfield Unit (HU) tissue density, thereby affecting the conversion and calibration of the proton relative stopping power (Stopping Power Ratio, SPR), resulting in deviations in the Bragg peak position. This problem is particularly prominent in proton therapy, because the dose deposition of the proton beam is highly concentrated, and even small errors in the range can lead to significant changes in the dose distribution. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a result simulation method, device and equipment based on a generative adversarial network to solve the problem of insufficient accuracy in side effect prediction and long-term efficacy tracking in proton therapy, especially the technical challenges in dynamic simulation, long-term tracking and multi-center data standardization, so as to at least solve some of the problems in the background technology.

[0006] In order to achieve the above-mentioned purpose, the present application provides a result simulation method based on a generative adversarial network, which includes: inputting a CT image and proton beam parameters into a trained generative adversarial network, wherein the generative adversarial network includes a primary generator, a fine-tuning generator and a dual-channel discriminator; after the CT image is encoded and decoded by the primary generator, a basic tissue deformation field is obtained; after the basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters are subjected to feature fusion and attention weight mapping by the fine-tuning generator, an enhanced image and an output feature vector are obtained; after the output feature vector passes through the spatial feature discrimination branch in the dual-channel discriminator, an image authenticity result is obtained, and after the output feature vector passes through the dose feature discrimination branch in the dual-channel discriminator, a dose evaluation result is obtained; after the enhanced image is discriminated according to the image authenticity result and the dose evaluation result, the output image of the trained generative adversarial network is obtained.

[0007] Optionally, after obtaining the output image of the generative adversarial network, the method further includes: calculating the difference between the output image and the input CT image; and obtaining, according to the difference, the influence of the proton beam based on the proton beam parameters on different types of regions in the input CT image.

[0008] Optionally, the different types of regions include tumor regions; calculating the difference between the output image and the input CT image includes: segmenting the tumor region from the output image based on an adaptive segmentation algorithm, and calculating a first volume of the tumor region; segmenting the tumor region from the input CT image based on the adaptive segmentation algorithm, and calculating a second volume of the tumor region; obtaining a tumor volume difference based on the first volume and the second volume.

[0009] Optionally, the different types of areas include normal areas; calculating the difference between the output image and the input CT image includes: calculating a Monte Carlo dose according to a nuclear physics engine based on the output image and the proton beam parameters; and obtaining a dose distribution difference between a dose distribution map of a normal area in the input CT image and a dose distribution map of a normal area in the output image according to the Monte Carlo dose.

[0010] Optionally, before inputting the CT images and proton beam parameters into the trained generative adversarial network, the method further includes: using a domain-adaptive generative adversarial network to establish an image normalization channel, performing grayscale normalization and spatial registration on the multi-center CT images while retaining high-frequency texture features.

[0011] Optionally, the method further includes: constructing a multimodal incremental learning framework, integrating follow-up CBCT images, biomarker data and quality of life survey scores, and dynamically updating the parameters of the trained generative adversarial network through an elastic weight consolidation algorithm.

[0012] Optionally, the method further includes: deploying a real-time dose tracking system for visually displaying the risk corresponding to the dose; the real-time dose tracking system includes: a deformable registration module for aligning images using a non-rigid registration algorithm; an online reconstruction engine for decomposing the proton beam into multiple pencil beams, calculating their dose deposition separately, and then superimposing the contributions of all pencil beams to generate a three-dimensional dose distribution and output a dose matrix; and a risk visualization interface for generating a color-coded risk map with dose-volume constraints based on the dose matrix.

[0013] Optionally, the method further includes: designing a special prediction subnetwork for each impact result, and obtaining the risk probability of the impact result according to the output of the special prediction subnetwork; the special prediction subnetwork includes: an input layer, a feature fusion layer and an output layer; the input layer includes: a dosimetry feature branch, a biomarker branch and an imaging feature branch; the feature fusion layer is used to use a cross-attention mechanism to perform feature fusion to obtain a fusion result; the output layer is used to activate the fusion result and output the risk probability of the impact result.

[0014] The present application also provides a result simulation device based on a generative adversarial network, which includes: a data input module for inputting a CT image and proton beam parameters into a trained generative adversarial network, wherein the generative adversarial network includes a primary generator, a fine-tuning generator and a dual-channel discriminator; a first processing module for encoding and decoding the CT image by the primary generator to obtain a basic tissue deformation field; a second processing module for subjecting the basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters to feature fusion and attention weight mapping by the fine-tuning generator to obtain an enhanced image and an output feature vector; a third processing module for subjecting the output feature vector to a spatial feature discrimination branch in the dual-channel discriminator to obtain an image authenticity result, and for subjecting the output feature vector to a dose evaluation result by a dose feature discrimination branch in the dual-channel discriminator; and a data output module for discriminating the enhanced image based on the image authenticity result and the dose evaluation result to obtain the output image of the trained generative adversarial network.

[0015] This application also provides an electronic device, comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the aforementioned result simulation method based on a generative adversarial network by executing the instructions stored in the memory.

[0016] The present application also provides a machine-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the processor is configured to execute the aforementioned result simulation method based on a generative adversarial network.

[0017] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the aforementioned result simulation method based on a generative adversarial network.

[0018] The above technical solution has the following beneficial effects: By deeply integrating generative adversarial networks, real-time monitoring technology, and personalized prediction models, the accuracy of automatically generated CT images after proton beam influence has been improved. This provides an accurate data foundation for the precision and efficiency of subsequent proton therapy side effect prediction and efficacy tracking. This method provides clinicians with a highly effective auxiliary tool, promoting the development of proton therapy towards more precise and personalized treatments, and has significant clinical application value.

[0019] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings: Figure 1 Schematically shows a schematic diagram of the steps of the result simulation method based on the generative adversarial network according to an embodiment of the present application; Figure 2 A schematic diagram schematically illustrates a data processing process of a generative adversarial network according to an embodiment of the present application; Figure 3 Schematically shows a schematic diagram of generating an output image of an adversarial network according to an embodiment of the present application; Figure 4 The following schematically shows a schematic diagram of a prediction subnet structure for a specific impact according to an embodiment of the present application; Figure 5Schematically shows an implementation diagram of the result simulation method based on the generative adversarial network according to the embodiment of the present application; Figure 6 Schematically shows a structural diagram of a result simulation device based on a generative adversarial network according to an embodiment of the present application; Figure 7 The internal structure of an electronic device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0021] The following describes the specific implementation of the embodiment of the present application in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present application and is not intended to limit the embodiment of the present application.

[0022] Figure 1 The following schematically shows the steps of the result simulation method based on the generative adversarial network according to the embodiment of the present application. Figure 1 As shown, a result simulation method based on a generative adversarial network, the method comprising: S01. Input CT images and proton beam parameters into a trained generative adversarial network (GAN), consisting of a primary generator, a fine-tuned generator, and a dual-channel discriminator. The GAN was trained using the following data: pre-treatment high-resolution CT images (1 mm slice thickness, 512×512 matrix) of 200 patients with head and neck cancer. Proton therapy parameters, including beam energy (70–230 MeV), Bragg peak spread (SOBP width 3–10 mm), and target dose prescription (54–70 Gy), were simultaneously acquired. CT images were isotropically resampled to 1×1×1 mm³ and grayscale normalized using histogram matching (reference window width: 1500 HU, window level: -500 HU).

[0023] S02. After encoding and decoding the CT image by the primary generator, a basic tissue deformation field is obtained. The primary generator adopts a 3D U-Net architecture and takes as input a 512×512×256 voxel CT image. The grayscale values ​​may need to be normalized to [0, 1]. The encoder for encoding contains five layers of 3D convolutions. The 3D convolutions can be configured with a stride of 2, a convolution kernel of 3×3×3, and padding of 1. Each convolution layer is followed by a ReLU activation function and batch normalization. The decoder for decoding restores the spatial resolution through bilinear upsampling and skip connections, and outputs a basic tissue deformation field of size 512×512×256 with three channels to simulate continuous changes in anatomical structure before and after treatment.

[0024] S03. After the basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters are subjected to feature fusion and attention weight mapping by the fine-tuning generator, an enhanced image and output feature vector are obtained. Based on the primary generator, the fine-tuning generator adds an attention gate module (Attention Gate), which inputs the fused proton Bragg peak distribution map. The size can be 512×512×256 and the number of channels is 1. The attention mechanism dynamically adjusts the feature weights to focus on the key areas of dose deposition. The specific steps of the attention gate module that provides the attention mechanism include: inputting the fused proton Bragg peak distribution map, denoted as X.

[0025] 1) Concatenate the output feature map F of the primary generator (size 512×512×256, number of channels C) with the Bragg peak distribution map X (size 512×512×256, number of channels 1) in the channel dimension to obtain a fused feature map; 2) Attention weight calculation:

[0026] in, and It is a 1×1×1 convolutional layer that maps the fusion feature map to the attention weights. is an activation function that outputs an attention weight map A with a size of 512×512×256 and a channel number of 1; 3) Feature Weighting

[0027] in, Indicates element-by-element multiplication to obtain feature weights ; 4) Output features , as the input of the next layer.

[0028] S04. After the output feature vector passes through the spatial feature discrimination branch of the dual-channel discriminator, an image authenticity result is obtained. After the output feature vector passes through the dosimetric feature discrimination branch of the dual-channel discriminator, a dosimetric evaluation result is obtained. The spatial feature discrimination branch is composed of a 7-layer 3D convolutional network with convolution kernel sizes of 4×4×4, 3×3×3, …, 1×1×1, with a step size of 2. It outputs an image authenticity score with a score interval of (0, 1). The dosimetric feature discrimination branch uses a graph neural network (GNN) and inputs a dose distribution map, which can be represented by a DVH feature, to evaluate the physical rationality of dose deposition. The output dosimetric score interval is consistent with the image authenticity score interval and can be selected as (0, 1). This dual-channel design ensures the dual accuracy of the generated images in terms of anatomical structure and dosimetric characteristics.

[0029] S05. The enhanced image is discriminated based on the image authenticity result and the dose evaluation result to obtain an output image of the trained generative adversarial network. The discriminator in the generative adversarial network performs the discrimination, and its implementation mechanism is based on the operating mechanism of the generative adversarial network.

[0030] Through the above implementation methods, the accuracy of automatically generated CT images after being affected by proton beams is improved by deeply integrating generative adversarial networks, real-time monitoring technology, and personalized prediction models.

[0031] Figure 2 The following schematically shows a data processing process of generating an adversarial network according to an embodiment of the present application. Figure 2 As shown, it illustrates the input data, output data, data processing flow and other information of the primary generator, fine-tuning generator and dual-channel discriminator.

[0032] In some optional implementations, the loss function adopts a weighted combination: the loss function L total Adopting adversarial loss L adv and deformation consistency loss L consist The specific form is , where α=0.7 and β=0.3.

[0033] In some embodiments of the present application, after obtaining the output image of the generative adversarial network, the method further includes: calculating the difference between the output image and the input CT image; and obtaining, based on the difference, the influence of the proton beam based on the proton beam parameters on different types of regions in the input CT image.

[0034] In some embodiments of the present application, the different types of regions include tumor regions; calculating the difference between the output image and the input CT image includes: segmenting the tumor region from the output image based on an adaptive segmentation algorithm and calculating a first volume of the tumor region; segmenting the tumor region from the input CT image based on the adaptive segmentation algorithm and calculating a second volume of the tumor region; and obtaining a tumor volume difference based on the first volume and the second volume. The adaptive segmentation algorithm includes two stages: Phase 1: Based on the improved V-Net model, the CT image and initial dose distribution map were input. Residual blocks were introduced to mitigate the vanishing gradient problem, and deep supervision was added to the decoder to force the network to focus on multi-scale features (Dice coefficient > 0.85). The second stage: Applying a conditional random field (CRF) to post-process the initial segmentation results and generate a risk-sensitive segmentation mask in combination with the dose gradient map. CRF optimizes boundary smoothness and dose sensitivity through an energy function, ensuring that the segmentation results not only accurately reflect the tumor morphology but also highlight high-dose risk areas, providing an accurate basis for subsequent risk assessment. The specific steps of the conditional random field are: 1) Calculate the CRF conditional probability distribution:

[0035] Among them, X represents the observation sequence, Y represents the label sequence, and each node represents a label , and observation associated. Indicates observation With label The relationship between Indicates adjacent labels and The relationship between , Z(X) represents the normalization factor, which is used to ensure that the sum of the probability distribution is 1; 2) Maximize the log-likelihood function: ; 3) Inference phase, find the most likely label sequence .

[0036] Through this embodiment, the change characteristics of tumor volume can be obtained.

[0037] In some embodiments of the present application, the different types of regions include normal regions; calculating the difference between the output image and the input CT image includes: calculating a Monte Carlo dose according to a nuclear physics engine based on the output image and the proton beam parameters; and obtaining a dose distribution difference between a dose distribution map of the normal region in the input CT image and a dose distribution map of the normal region in the output image based on the Monte Carlo dose. This embodiment includes: obtaining the output image of step S05; performing a Monte Carlo dose calculation according to a Monte Carlo dose calculation model; obtaining a biological effect based on the Monte Carlo dose and a dose-to-biological effect mapping; and quantifying the risk of the biological effect.

[0038] The input parameters of this embodiment are: the output image and proton beam parameters of step S05 , and the output parameter is: the damage risk heat map of healthy tissue.

[0039] Among them, the Monte Carlo dose calculation formula is: ; Where r is the tissue location, E is the energy, and x represents the proton. is the energy deposited per unit mass, is the tissue density from CT, for the proton collision stopping power, is the proton flux density.

[0040] Dose-to-biological effect mapping includes:

[0041] Wherein, e is the natural constant, D is the energy deposited per unit mass, IL-6 is interleukin-6, a, b, and c are fitting parameters, and P is the biological effect.

[0042] Figure 3 Schematic diagram showing the output image of the adversarial network generated according to the embodiment of the present application. Figure 3 As shown, it illustrates the identification and division of tumor segmentation and risk areas in the output image.

[0043] In some embodiments of the present application, before inputting the CT images and proton beam parameters into the trained generative adversarial network, the method further includes: using a domain-adapted generative adversarial network to establish an image normalization channel, performing grayscale normalization and spatial registration on the multi-center CT images, while preserving high-frequency texture features. Specifically, the domain-adapted generative adversarial network is used to establish an image normalization channel, performing grayscale normalization (μ = 1200 HU, σ = 50 HU) and spatial registration (resolution 1×1×1 mm³) on the multi-center CT images, while retaining more than 5% of the high-frequency texture features, and retaining more than 5% of the high-frequency texture features, namely, edge information such as organ edges and lesion areas and texture details of microstructures within tissues, to ensure data consistency and cross-institutional applicability of the model.

[0044] In some embodiments of the present application, the method further comprises: constructing a multimodal incremental learning framework, integrating follow-up CBCT images, biomarker data, and quality of life survey scores, and dynamically updating the parameters of the trained generative adversarial network through an elastic weight consolidation algorithm. This embodiment provides incremental learning and model optimization, which mainly comprises the following steps: Step 1, multimodal data integration. Construct an incremental learning dataset containing the following elements: imaging data: follow-up CBCT images (layer thickness 2 mm, registration error <1 mm); biomarkers: serum IL-6, TGF-β1 concentrations (test frequency: once a week); quality of life survey scores: CTC-AE 4.0 toxicity grade, EORTC QLQ-C30 quality of life score. Step 2, applying the elastic weight consolidation algorithm (EWC). Importance weight calculation: Calculate the Fisher information matrix for every 200 new data and screen key parameters (threshold: Top 20%); parameter update rules include: for param in model.parameters(): update=new_grad - λ* fisher_matrix[param] * (param - old_param) param.data -= learning_rate * update.

[0045] The elastic coefficient is searched for the optimal value of λ=0.75 on the validation set through grid search. The learning rate adopts the cosine annealing strategy (initial value 0.001, adjustment period 50 epochs) to dynamically optimize the convergence process and avoid catastrophic forgetting. Memory management: Gradient information of the first 50 key cases is retained, and a ring buffer strategy is used to prevent data forgetting. Preferably, multimodal fusion adopts the cross-attention mechanism:

[0046] Among them, the query vector Q comes from the image features, the key-value pairs K and V come from clinical data to achieve feature alignment, and dk is the dimension of the key, which is set to 64.

[0047] In some embodiments of the present application, the method further includes deploying a real-time dose tracking system to visualize the risk associated with the dose; the real-time dose tracking system is primarily used to update organ segmentation contours based on daily CBCT images (registration error <1 mm) using a dynamic deformation registration algorithm (Demons algorithm, 50 iterations, 0.5 mm step size), combined with online dose reconstruction (GPU-accelerated proton pencil beam algorithm, calculation speed ≥10 fields / second) to generate a risk heat map. The system includes: The deformable registration module is used to align images using a non-rigid registration algorithm. Specifically, the Demons algorithm (50 iterations, 0.5 mm step size) is used to achieve non-rigid registration of daily CBCT and planning CT, with an error control of <1 mm.

[0048] Among them, the Demons non-rigid registration algorithm is used to align images by optimizing the energy function, including:

[0049] in, represents the CBCT image to be registered, Represents the CT image to be registered, x represents the grayscale value at the coordinate of the image, and u represents the displacement field. Indicates the weight of the control smoothing term. Then, the organ contour is updated, including: segmenting the tumor boundary based on the Fast Marching method, setting the growth rate factor: F(x) = 1 / (1 + |∇I(x)|), where I(x) is the image gray value and ∇I(x) is the gray gradient.

[0050] An online reconstruction engine for decomposing a proton beam into multiple pencil beams, calculating their dose depositions respectively, and then superimposing the contributions of all pencil beams to generate a three-dimensional dose distribution and output a dose matrix; a proton pencil beam algorithm based on GPU acceleration, with a calculation speed ≥ 10 fields / second, preferably a calculation speed of 15 fields / second.

[0051] And, a risk visualization interface for generating a color-coded risk map with dose volume constraints based on the dose matrix. For example: generating the following risk heat map: Red warning area: D95% ≥ 50 Gy (such as the brainstem, optic nerve); Yellow observation area: 30 Gy < D95% < 50 Gy (such as the parotid gland, temporal lobe); Green safe area: D95% ≤ 30 Gy.

[0052] In some embodiments, it further includes: generating corresponding alarms according to the color-coded risk map or the risk heat map. For example: when the following situation occurs, the cumulative dose around the spinal cord ≥ 45 Gy and the tumor volume shrinkage rate < 5% for 3 consecutive days, a level 3 alarm is triggered to prompt the risk of drug resistance.

[0053] In some embodiments of the present application, the method further includes: designing a special prediction subnet for each influencing result, and obtaining the risk probability of the occurrence of the influencing result according to the output of the special prediction subnet; the special prediction subnet includes: an input layer, a feature fusion layer, and an output layer; the input layer includes: a dosimetric feature branch, a biomarker branch, and an imaging feature branch; the feature fusion layer is used to perform feature fusion using a cross-attention mechanism to obtain a fusion result; the output layer is used to output the risk probability of the occurrence of the influencing result after activating the fusion result. In order to specifically predict a particular influence, this embodiment provides a special prediction subnet. The core objective of the special prediction subnet is to improve the prediction accuracy through multi-modal data fusion to solve the problem that traditional single subnets are difficult to handle efficiently. Among them, the architecture of the special prediction subnet includes: Input layer: Dosimetric feature branch (DYH parameters), biomarker branch (IL-6, TGF-β1), imaging feature branch (high-risk area generated by GAN). Feature fusion layer: Using a cross-attention mechanism. Output layer: Using a Sigmoid function to output the risk probability of side effects. Figure 4 Schematically shows a schematic diagram of the prediction subnet structure for a specific influence according to an embodiment of the present application. As Figure 4 shown, it schematically shows the prediction subnet structure for the influencing result of neurotoxicity and the output of the prediction result.

[0054] Figure 5 The following schematically shows an implementation diagram of the result simulation method based on the generative adversarial network according to the embodiment of the present application. Figure 5 As shown in the figure, it illustrates the relationship between the steps of GAN-based image output, data integration and model update, and special subnetwork prediction.

[0055] Through the above implementation methods, the accuracy of the prediction of the impact of proton beam and CT images is significantly improved by deeply integrating generative adversarial networks, real-time monitoring technology and personalized prediction models.

[0056] Based on the same inventive concept, this application also provides a result simulation device based on a generative adversarial network. Figure 6 The schematic diagram of the structure of the result simulation device based on the generative adversarial network according to the embodiment of the present application is shown. Figure 6 As shown, the device includes: a data input module for inputting CT images and proton beam parameters into a trained generative adversarial network, wherein the generative adversarial network includes a primary generator, a fine-tuning generator and a dual-channel discriminator; a first processing module for obtaining a basic tissue deformation field after encoding and decoding the CT image by the primary generator; a second processing module for obtaining an enhanced image and an output feature vector after subjecting the basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters to feature fusion and attention weight mapping by the fine-tuning generator; a third processing module for obtaining an image authenticity result after subjecting the output feature vector to a spatial feature discrimination branch in the dual-channel discriminator, and obtaining a dose evaluation result after subjecting the output feature vector to a dose feature discrimination branch in the dual-channel discriminator; and a data output module for discriminating the enhanced image according to the image authenticity result and the dose evaluation result to obtain the output image of the trained generative adversarial network.

[0057] In some optional embodiments, after obtaining the output image of the generative adversarial network, the device further includes: calculating the difference between the output image and the input CT image; and obtaining, based on the difference, the influence of the proton beam based on the proton beam parameters on different types of regions in the input CT image.

[0058] In some optional embodiments, the different types of regions include tumor regions; calculating the difference between the output image and the input CT image includes: segmenting the tumor region from the output image based on an adaptive segmentation algorithm, and calculating a first volume of the tumor region; segmenting the tumor region from the input CT image based on the adaptive segmentation algorithm, and calculating a second volume of the tumor region; and obtaining a tumor volume difference based on the first volume and the second volume.

[0059] In some optional embodiments, the different types of regions include normal regions; calculating the difference between the output image and the input CT image includes: calculating a Monte Carlo dose according to a nuclear physics engine based on the output image and the proton beam parameters; and obtaining a dose distribution difference between a dose distribution map of a normal region in the input CT image and a dose distribution map of a normal region in the output image according to the Monte Carlo dose.

[0060] In some optional embodiments, before inputting the CT images and proton beam parameters into the trained generative adversarial network, the apparatus further comprises: using a domain-adaptive generative adversarial network to establish an image normalization channel, performing grayscale normalization and spatial registration on the multi-center CT images while retaining high-frequency texture features.

[0061] In some optional embodiments, the device further comprises: constructing a multimodal incremental learning framework, integrating follow-up CBCT images, biomarker data, and quality of life survey scores, and dynamically updating the parameters of the trained generative adversarial network through an elastic weight consolidation algorithm.

[0062] In some optional embodiments, the device further comprises: deploying a real-time dose tracking system for visually displaying the risk corresponding to the dose; the real-time dose tracking system comprises: a deformable registration module for aligning images using a non-rigid registration algorithm; an online reconstruction engine for decomposing the proton beam into multiple pencil beams, calculating their dose deposition separately, and then superimposing the contributions of all pencil beams to generate a three-dimensional dose distribution and output a dose matrix; and a risk visualization interface for generating a color-coded risk map with dose-volume constraints based on the dose matrix.

[0063] In some optional embodiments, the device further includes: designing a special prediction subnet for each impact result, and obtaining the risk probability of the impact result according to the output of the special prediction subnet; the special prediction subnet includes: an input layer, a feature fusion layer and an output layer; the input layer includes: a dosimetric feature branch, a biomarker branch and an imaging feature branch; the feature fusion layer is used to use a cross-attention mechanism to perform feature fusion to obtain a fusion result; the output layer is used to activate the fusion result and output the risk probability of the impact result.

[0064] The specific definition of each functional module in the above-mentioned result simulation device based on the generative adversarial network can be found in the above-mentioned definition of the result simulation method based on the generative adversarial network, which will not be repeated here. Each module in the above-mentioned system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It also uses a deep fusion generative adversarial network to achieve the effect of improving the accuracy of image prediction.

[0065] In some embodiments of the present application, an electronic device is further provided, comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the aforementioned result simulation method based on the generative adversarial network. Its internal structure diagram can be as follows Figure 7 shown. Figure 7 The internal structure diagram of an electronic device according to an embodiment of the present application is schematically shown. The electronic device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected via a system bus. Among them, the processor A01 of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The network interface A02 of the electronic device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a result simulation method based on a generative adversarial network is implemented.

[0066] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0067] In one embodiment provided in the present application, a machine-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the aforementioned result simulation method based on a generative adversarial network.

[0068] In one embodiment provided in the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the aforementioned result simulation method based on a generative adversarial network.

[0069] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0073] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0074] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0075] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0077] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A result simulation method based on generative adversarial network, characterized in that: The method comprises: Inputting the CT image and proton beam parameters into a trained generative adversarial network, wherein the generative adversarial network includes a primary generator, a fine-tuned generator, and a dual-channel discriminator; After the CT image is encoded and decoded by the primary generator, a basic tissue deformation field is obtained; The basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters are subjected to feature fusion and attention weight mapping by the fine-tuning generator to obtain an enhanced image and an output feature vector; After the output feature vector passes through the spatial feature discrimination branch in the dual-channel discriminator, an image authenticity result is obtained; after the output feature vector passes through the dose feature discrimination branch in the dual-channel discriminator, a dose evaluation result is obtained; The enhanced image is discriminated according to the image authenticity result and the dose evaluation result to obtain the output image of the trained generative adversarial network.

2. The method according to claim 1, characterized in that After obtaining the output image of the trained generative adversarial network, the method further includes: Calculating the difference between the output image and the input CT image; The influence of the proton beam on different types of regions in the input CT image based on the proton beam parameters is obtained according to the difference.

3. The method according to claim 2, characterized in that The different types of regions include tumor regions; and calculating the difference between the output image and the input CT image includes: Segmenting a tumor region from the output image based on an adaptive segmentation algorithm, and calculating a first volume of the tumor region; Segmenting a tumor region from the input CT image based on the adaptive segmentation algorithm, and calculating a second volume of the tumor region; A tumor volume difference is obtained based on the first volume and the second volume.

4. The method according to claim 2, characterized in that The different types of areas include normal areas; Calculating the difference between the output image and the input CT image includes: calculating a Monte Carlo dose according to a nuclear physics engine based on the output image and the proton beam parameters; A dose distribution difference between a dose distribution map of a normal region in an input CT image and a dose distribution map of a normal region in an output image is obtained according to the Monte Carlo dose.

5. The method according to claim 1, wherein Before inputting the CT image and the proton beam parameters into the trained generative adversarial network, the method further includes: A domain-adaptive generative adversarial network is used to establish an image normalization channel to perform grayscale normalization and spatial registration on multi-center CT images while preserving high-frequency texture features.

6. The method according to claim 1, characterized in that The method also includes: constructing a multimodal incremental learning framework, integrating follow-up CBCT images, biomarker data and quality of life survey scores, and dynamically updating the parameters of the trained generative adversarial network through an elastic weight consolidation algorithm.

7. The method according to claim 1, characterized in that The method further includes: deploying a real-time dose tracking system for visually displaying the risk corresponding to the dose; the real-time dose tracking system includes: Deformable registration module, used to align images using a non-rigid registration algorithm; An online reconstruction engine that decomposes the proton beam into multiple pencil beams, calculates their dose deposition separately, and then superimposes the contributions of all pencil beams to generate a three-dimensional dose distribution and output a dose matrix; and A risk visualization interface is provided for generating a color-coded risk map with dose-volume constraints based on the dose matrix.

8. The method according to claim 1, characterized in that The method further includes: designing a special prediction subnet for each impact result, and obtaining the risk probability of the impact result according to the output of the special prediction subnet; The special prediction subnet includes: an input layer, a feature fusion layer and an output layer; The input layer includes: a dose feature branch, a biomarker branch, and an image feature branch; The feature fusion layer is used to use a cross attention mechanism to perform feature fusion to obtain a fusion result; The output layer is used to activate and output the fusion result to influence the risk probability of the result occurring.

9. A result simulation device based on a generative adversarial network, characterized in that: The device comprises: A data input module, configured to input CT images and proton beam parameters into a trained generative adversarial network, wherein the generative adversarial network includes a primary generator, a fine-tuning generator, and a dual-channel discriminator; a first processing module, configured to obtain a basic tissue deformation field after encoding and decoding the CT image by the primary generator; A second processing module is configured to obtain an enhanced image and an output feature vector by subjecting the basic tissue deformation field and the Bragg peak distribution map obtained based on the proton beam parameters to feature fusion and attention weight mapping by the fine-tuning generator; a third processing module, configured to obtain an image authenticity result after the output feature vector passes through a spatial feature discrimination branch in the dual-channel discriminator, and obtain a dose evaluation result after the output feature vector passes through a dose feature discrimination branch in the dual-channel discriminator; A data output module is used to obtain an output image of the trained generative adversarial network after discriminating the enhanced image according to the image authenticity result and the dose evaluation result.

10. An electronic device, characterized in that: include: at least one processor; a memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the result simulation method based on the generative adversarial network as described in any one of claims 1 to 8 by executing the instructions stored in the memory.

Images