Iodine density map and atomic number map generation model training method and generation method

Abstract

The application relates to a training method and a generation method of an iodine density map and an atomic number map generation model, and relates to the technical field of deep learning, wherein the training method comprises the following steps: performing multi-scale feature fusion processing on final up-sampling features of different scales to obtain a first result; performing target structure convolution operation and double-path up-sampling operation on to-be-fused up-sampling features for a preset number of times to obtain a second result, and performing convolution operation on a CT image sample through a target structure to obtain a third result; performing feature merging on the first result, each second result and the third result to obtain merged features of a final scale, and obtaining iodine density generation data and atomic number generation data corresponding to the CT image sample based on the merged features; and performing supervision training on an initial network model through a preset loss function to obtain a target generation model of the iodine density map and the atomic number map. The application has the effect of reducing the generation cost of the atomic number map and the iodine density map.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, specifically to a training method and generation method for a generative model of iodine density map and atomic number map. Background Technology

[0002] Iodine density mapping visualizes the distribution of iodine contrast agents in tissues. It reflects the degree of absorption and retention of iodine contrast agents in tissues, thus indirectly providing information about tissue function, metabolic activity, or pathological conditions. Atomic number mapping typically refers to an image that infers the atomic number distribution of a substance based on its attenuation characteristics to X-rays. This type of mapping helps distinguish different materials or tissues because the atomic number (i.e., the number of protons) directly affects a substance's ability to absorb X-rays. Iodine density mapping and atomic number mapping are of great significance for tumor characterization, vascular assessment, and metal artifact correction.

[0003] Currently, traditional computed tomography (CT) imaging can only provide grayscale images based on Hounsfield units, which cannot directly reflect the physical properties (such as atomic number) or functional information (such as iodine density) of tissues. This results in limitations in clinical scenarios such as tumor characterization, vascular assessment, and metal artifact correction. While dual-energy CT (DECT) can generate atomic number maps and iodine density maps through material decomposition technology, the related equipment used is relatively expensive and the radiation dose is high, resulting in high costs for generating atomic number maps and iodine density maps. Summary of the Invention

[0004] To reduce the cost of generating atomic number maps and iodine density maps, this application provides a training method and a generation method for generating iodine density maps and atomic number maps.

[0005] The first aspect of this application provides a method for training a generative model of iodine density maps and atomic number maps, specifically including:

[0006] Acquire target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number;

[0007] The CT image samples are input into a preset initial network model, and at least one branch feature is obtained through the Transformer encoder in the initial network model;

[0008] The branch features are input into the decoder in the initial network model, and the branch features are determined as target features. The target features are convolved through a preset target structure to obtain the convolution result. The convolution result is then subjected to dual-path sampling to obtain intermediate upsampled features. The intermediate upsampled features are determined as target features, and the steps of convolving the target features through the preset target structure to obtain the convolution result are repeated to obtain the final upsampled features at a single-layer scale. Multi-scale feature fusion processing is then performed on the final upsampled features at different scales to obtain the first result.

[0009] The target structure convolution operation and dual-path sampling operation are performed on the upsampled feature to be fused a preset number of times to obtain a second result. The CT image sample is then convolved with the target structure to obtain a third result. The upsampled feature to be fused is the final upsampled feature fused with the final upsampled feature at the previous scale.

[0010] The first result, each of the second results, and the third result are combined to obtain the final scale of the combined features. Based on the combined features, the iodine density generation data and atomic number generation data corresponding to the CT image sample are obtained.

[0011] Based on the iodine density generation data, the atomic number generation data, the real iodine density data, and the real atomic number data, the initial network model is trained under supervision using a preset loss function to obtain the target generation model of the iodine density map and the atomic number map.

[0012] By employing the aforementioned technical solution, convolutional operations are performed on target features through the target structure, enabling the acquisition of global contextual features with fewer parameters and capturing more long-range information. The convolutional results are then subjected to dual-path sampling to obtain the final upsampled features at a single scale. This dual-path sampling method avoids overly smoothing of the generated results due to linear interpolation and introduces randomized branches, making the generated image closer to the real image. Furthermore, the feature information of the final upsampled features corresponding to each branch of the first, second, and third results is contributed to the final scale, improving image reconstruction quality and thus enhancing the performance of the initial network model. The initial network model is then trained using a loss function to further improve its performance. Subsequently, this target generation model can quickly and accurately generate iodine density maps and atomic number maps based on CT images without the need for expensive equipment, thus reducing the cost of generating atomic number maps and iodine density maps to some extent.

[0013] In one implementation, performing a dual-path sampling operation on the convolution result to obtain intermediate upsampled features specifically includes:

[0014] The convolution result is sequentially subjected to convolution operation, activation function processing and bilinear interpolation processing to obtain a first processing result, and the first processing result is then subjected to convolution operation to obtain a first path sampling result;

[0015] The convolution result is sequentially subjected to convolution operation, activation function processing and pixel shuffling and rearrangement processing to obtain a second processing result, and the second processing result is subjected to convolution operation to obtain a second path sampling result;

[0016] The first upsampling result and the second upsampling result are merged to obtain the intermediate upsampling feature.

[0017] In one implementation, the multi-scale feature fusion processing of the final upsampled features at different scales to obtain a first result specifically includes:

[0018] Establish jump connections between the features of each branch;

[0019] The final upsampled features at the current scale are fused with the final upsampled features at the next lower scale to obtain the initial fusion result.

[0020] Through the skip connection, at least one intermediate upsampled feature corresponding to the final upsampled feature of the upper-level scale of the current scale is obtained;

[0021] At least one feature to be fused is determined from each of the intermediate upsampled features, and each of the fused features is fused with the initial fusion result. The step of fusing the final upsampled feature at the current scale with the final upsampled feature at the next scale of the current scale is repeated to obtain the first result.

[0022] In one embodiment, the method further includes:

[0023] Substituting the iodine density generation data into a preset first calculation formula yields the corresponding first error. The first calculation formula is as follows:

[0024] ;

[0025] In the formula, Indicating iodine density generation data The iodine density generated at the point; This indicates that the actual data on iodine density is in The true value of iodine density at the point; Mask is a matrix that controls the weights of the contribution of the generation error of iodine density at different locations to the overall mean absolute error, where α and β both represent weights.

[0026] In one embodiment, the method further includes:

[0027] Substituting the atomic number generation data into a preset second calculation formula yields the corresponding second error. The second calculation formula is as follows:

[0028] ;

[0029] In the formula, This represents the set of pixel locations within the body area corresponding to a CT image sample. This represents the total number of pixels within the body area. This indicates the position of a pixel within the body's dimensions. Indicating atomic number generation data The generated atomic number value at the point; Indicating the true atomic number data The true value of the atomic number at the point.

[0030] A second aspect of this application provides a method for generating an iodine density map and an atomic number map, specifically including:

[0031] The CT image to be analyzed is obtained and the target generative model is obtained by training the generative model of iodine density map and atomic number map as described in the first aspect.

[0032] The CT image to be analyzed is input into the target generation model to obtain the corresponding iodine density map and atomic number map.

[0033] A third aspect of this application provides a training apparatus for a generative model of iodine density maps and atomic number maps, specifically comprising:

[0034] The data acquisition module is used to acquire target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number.

[0035] The feature extraction module is used to input the CT image samples into a preset initial network model, and obtain at least one branch feature through the Transformer encoder in the initial network model;

[0036] The feature fusion module is used to input the branch features into the decoder in the initial network model, determine the branch features as target features, perform convolution operation on the target features through a preset target structure to obtain the convolution result, perform dual-path sampling operation on the convolution result to obtain intermediate upsampled features, determine the intermediate upsampled features as target features, and repeat the step of performing convolution operation on the target features through a preset target structure to obtain the convolution result, to obtain the final upsampled features at a single-layer scale, and perform multi-scale feature fusion processing on the final upsampled features at different scales to obtain the first result;

[0037] The result determination module is used to perform a preset number of target structure convolution operations and dual-path sampling operations on the upsampled features to be fused to obtain a second result, and to perform a convolution operation on the CT image samples through the target structure to obtain a third result. The upsampled features to be fused are the final upsampled features fused with the final upsampled features at the previous scale.

[0038] The result generation module is used to merge features of the first result, each of the second results and the third result to obtain the merged features at the final scale, and based on the merged features, to obtain the iodine density generation data and atomic number generation data corresponding to the CT image sample;

[0039] The model training module is used to supervise the training of the initial network model based on the iodine density generation data, the atomic number generation data, the real iodine density data, and the real atomic number data, using a preset loss function, to obtain the target generation model of the iodine density map and the atomic number map.

[0040] By adopting the above technical solution, the data acquisition module obtains the target training data, the feature extraction module obtains at least one branch feature through the Transformer encoder in the initial network model, the feature fusion module determines the final upsampling feature at a single-layer scale, and performs multi-scale feature fusion on the final upsampling features at different scales to obtain the first result, the result determination module determines the second and third results, the result generation module merges the features of the first result, each of the second and third results to obtain the merged features, and determines the iodine density generation data and atomic number generation data based on the merged features, and finally the model training module performs supervised training on the initial network model through a preset loss function to obtain the target generation model of iodine density map and atomic number map.

[0041] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when loaded and executed by a processor, performs the steps of the method described in any one of the first aspects.

[0042] A fourth aspect of this application provides an electronic device, specifically comprising:

[0043] A processor, a memory, and a computer program stored in the memory and capable of running on the processor, the processor being configured to load and execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0044] In summary, this application includes at least one of the following beneficial technical effects: By performing convolution operations on target features using the target structure, global contextual features can be obtained with fewer parameters, capturing more long-range information. Then, a dual-path sampling operation is performed on the convolution result to obtain the final upsampled features at a single scale. This dual-path sampling method avoids excessive smoothing of the generated results due to linear interpolation and introduces randomized branches, making the generated image closer to the real image. Furthermore, the feature information of the final upsampled features corresponding to each branch feature is contributed to the final scale from the first result, each second result, and the third result, improving the image reconstruction quality and thus enhancing the performance of the initial network model. Then, the initial network model is trained using a loss function to further improve the model's performance. Subsequently, this target generation model can quickly and accurately generate iodine density maps and atomic number maps based on CT images without the need for expensive equipment, thus reducing the cost of generating atomic number maps and iodine density maps to a certain extent. Attached Figure Description

[0045] Figure 1 This is a schematic flowchart of a method for training a generation model of iodine density map and atomic number map provided in an embodiment of this application;

[0046] Figure 2 This is a schematic flowchart of a dual-path sampling operation provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of a jump connection provided in an embodiment of this application;

[0048] Figure 4 This is a schematic flowchart of a method for generating an iodine density map and an atomic number map provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of a model training device for generating iodine density maps and atomic number maps provided in an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of the structure of another iodine density map and atomic number map generation model training device provided in the embodiments of this application;

[0051] Figure 7 This is a schematic diagram of the structure of an iodine density map and atomic number map generation device provided in an embodiment of this application.

[0052] Figure labeling: 11. Data acquisition module; 12. Feature extraction module; 13. Feature fusion module; 14. Result determination module; 15. Result generation module; 16. Model training module; 17. First calculation module; 18. Second calculation module; 21. Information acquisition module; 22. Model prediction module. Detailed Implementation

[0053] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0054] In the description of the embodiments of this application, words such as "exemplarily," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0055] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0056] See Figure 1This application discloses a flowchart illustrating a method for training a generative model of iodine density maps and atomic number maps. This method can be implemented using a computer program or run on a training device for a generative model of iodine density maps and atomic number maps based on a von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including:

[0057] S101: Obtain target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number.

[0058] Specifically, in this embodiment, the CT image sample can be CT image data of a historical patient's brain; in other embodiments, the CT image sample can also be CT image data of other body parts of a historical patient. Iodine density reflects the functional information of a patient's body tissues, characterizing the concentration distribution of iodine contrast agent in body tissues, and is typically used to assess tissue metabolic activity or vascular permeability. Atomic number reflects the physical characteristics of a patient's body tissues, and is closely related to the tissue's X-ray attenuation coefficient. Iodine density and atomic number are important for tumor characterization, vascular assessment, and metal artifact correction. True iodine density data refers to the actual iodine density data within the patient's body area to which the CT image sample belongs, including the actual iodine density values ​​at various locations within the body area. True atomic number data refers to the actual atomic number data within the patient's body area to which the CT image sample belongs, including the actual atomic number values ​​at various locations within the body area. It should be noted that a single CT image sample is perfectly matched to its corresponding true iodine density data and true atomic number data in size, dimensions, and spatial location.

[0059] Furthermore, in the iodine density map and atomic number map generation model training method disclosed in this application embodiment, the execution entity is a server. The server is wirelessly connected to a terminal, which can be a smartphone or a personal computer. The terminal has a client related to model training installed, and the server is the backend server of the client. Specifically, it can be an independent physical server or a cluster of multiple physical servers. A feasible way to obtain target training data is as follows: the terminal sends historical patient CT image samples, along with iodine density maps and atomic number maps, to the server. After receiving the data, the server performs registration processing to register the iodine density map and atomic number map to the coordinate system of the corresponding CT image sample, thereby obtaining the true iodine density data and true atomic number data corresponding to the CT image sample.

[0060] S102: Input the CT image sample into the preset initial network model, and obtain at least one branch feature through the Transformer encoder in the initial network model.

[0061] Specifically, in this embodiment, the initial network model is an improved version of the network structure in UNETR (Unified Transformer). UNETR is a Transformer-based image segmentation model that combines the global modeling capabilities of Transformer with the symmetric structure of U-Net, including a Transformer encoder, decoder, and skip connections between them. The improvements of the initial network model compared to the UNETR model are as follows: The convolutional operations in the UNETR model are replaced by a target structure, which is a composite structure of a Mamba module with residual connections and depthwise separable convolutions. Convolutional operations using this target structure allow for the acquisition of global contextual features with fewer parameters, capturing more long-range information. The Mamba module is a sequence modeling tool based on a State Space Model (SSM). It captures long-term dependencies in sequences through linear state transition equations and combines residual connections to improve training stability and performance. Furthermore, the deconvolutional operations in the UNETR model are replaced by dual-on-sampling operations, the details of which are described below. It should be noted that the initial network model also includes a Transformer encoder, a decoder, and skip connections between them. The decoder is a convolutional neural network-based decoder. Further, CT image samples are input into the pre-defined initial network model, and the Transformer encoder in the initial network model encodes the CT image samples to extract features, obtaining at least one branch feature.

[0062] S103: Input the branch features into the decoder in the initial network model, determine the branch features as target features, perform convolution operation on the target features through the preset target structure to obtain the convolution result, and perform dual-path sampling operation on the convolution result to obtain intermediate upsampled features. Determine the intermediate upsampled features as target features, and repeat the step of performing convolution operation on the target features through the preset target structure to obtain the convolution result, to obtain the final upsampled features at a single-layer scale, and perform multi-scale feature fusion processing on the final upsampled features at different scales to obtain the first result.

[0063] Specifically, the branch features are input into the decoder of the initial network model, and these branch features are simultaneously identified as target features. A convolution operation is performed on the target features using the target structure to obtain the convolution result. Then, a bilinear upsampling operation is performed on this convolution result to obtain intermediate upsampling features. Specifically, a 3x3 convolution kernel is used to convolve the result, followed by a non-linear transformation introduced through a preset activation function, and then bilinear interpolation is performed to obtain the first processing result. The activation function is PReLU. Bilinear upsampling is a commonly used upsampling method in image processing, primarily used to smooth pixels during image magnification. Finally, a 3x3 convolution kernel is used to convolve the first processing result to obtain features at a scale higher than the convolution result, i.e., the first upsampling result. In other embodiments, convolution operations can also be performed on the convolution result using convolution kernels of other sizes, such as 1x1.

[0064] Furthermore, this convolution result is convolved using a 3x3 convolution kernel, and then a non-linear transformation is introduced through an activation function. Following this, the pixels are shuffled and rearranged to obtain the second processed result. Pixel shuffling is an upsampling technique commonly used in tasks such as image super-resolution and semantic segmentation. It transforms a low-resolution feature map into a high-resolution one by rearranging the pixels, while maintaining a balance in the number of channels. Finally, the second processed result is convolved using a 3x3 convolution kernel to obtain the second pathsampling result.

[0065] Furthermore, the features of the first upsampling result and the second upsampling result are merged to obtain the intermediate upsampling feature. See details in [link to documentation]. Figure 2 , Figure 2 In this diagram, C, H, and W represent the number of channels, height, and width, respectively. Next, the obtained intermediate upsampled features are determined as the target features. The process of performing convolution operations on these target features using a preset target structure is repeated to obtain the convolution result. That is, the target features are again convolved using the target structure, followed by dual-path sampling. This process is repeated multiple times to obtain features at a higher level, i.e., the final upsampled features at a single-level scale. It should be noted that this dual-path sampling method avoids overly smoothed results caused by linear interpolation. A randomized sorting branch is introduced to make the generated image closer to the real image.

[0066] Finally, multi-scale feature fusion processing is performed on the final upsampled features at different scales. That is, the final upsampled features at the current scale are fused with the final upsampled features at the previous scale, and so on, to obtain the first result. This is existing technology and will not be elaborated here. In other embodiments, during the multi-scale feature fusion process, skip connections are established between the features of each branch. This can be understood as establishing skip connections between different branches (links for upsampling features from different branches), thereby realizing the reuse and fusion of features between different branches. Next, the final upsampled features at the current scale are fused with the final upsampled features at the next scale to obtain the initial fusion result. It should be noted that the fusion process can be briefly described as follows: the final upsampled features at the next scale are convolved and bi-path sampled before being fused with the final upsampled features at the current scale (keeping the scale consistent) to obtain the initial fusion result. Then, by skip connections, at least one intermediate upsampled feature corresponding to the final upsampled feature of the upper scale of the current scale is quickly obtained. Then, the intermediate upsampled feature with the same scale as the current scale is selected from the intermediate upsampled features and determined as the fusion feature. Finally, the fusion features are concatenated and fused with the initial fusion result. The step of fusing the final upsampled feature of the current scale with the final upsampled feature of the next scale of the current scale is repeated to finally obtain a first result with better quality.

[0067] For example, see Figure 3 In the figure, DWConv + RVM represents the target structure for which convolution operation is performed, and DualSampling represents dual-path sampling. In this embodiment, the obtained branch features can be four, namely the branch features of branches z3, z6, z9, and z12. The scale corresponding to the four branch features is 256×32×32. In addition, there is a feature of one branch (the fifth branch), which is the feature obtained by convolution of DWConv + RVM on the original scale of the CT image sample (the third result), with a scale of 16×512×512.

[0068] Furthermore, the 256×32×32 scale branch features of z12 are convolved using DWConv + RVM and then sampled on both sides to obtain the corresponding 128×64×64 scale features. Similarly, the 256×32×32 scale branch features of z9 are also convolved using DWConv + RVM and then sampled on both sides to obtain the corresponding 128×64×64 scale features.

[0069] The 256×32×32 scale branch features of the z6 branch are subjected to DWConv + RVM convolution operation and dual-path sampling to obtain intermediate upsampled features, i.e., features of scale 128×64×64. Then, the 128×64×64 scale features are subjected to DWConv + RVM convolution operation and dual-path sampling again to obtain features of scale 64×128×128.

[0070] The 256×32×32 scale branch features of the z3 branch are subjected to DWConv + RVM convolution operation and dual-path sampling to obtain 128×64×64 scale features. The 128×64×64 scale features are then subjected to DWConv + RVM convolution operation and dual-path sampling again to obtain 64×128×128 scale features. Finally, the 64×128×128 scale features are subjected to DWConv + RVM convolution operation and dual-path sampling to obtain 32×256×256 scale features.

[0071] Furthermore, through the skip connections between branches z6 and z9, as shown by the dashed arrows, and between z9 and z3, the 128×64×64 scale features corresponding to z3 and z6 are directly reused in branch z9. These features are then fused with the 128×64×64 scale features of branch z9 itself and the 128×64×64 scale features of branch z12. After fusion, convolution is performed using DWConv+ RVM, and dual-path sampling is performed to obtain 64×128×128 scale features. Next, through the skip connection between the z6 and z3 branches (as indicated by the dashed arrow), the 64×128×128 scale features in the z3 branch are reused in the z6 branch. These features are then fused with the 64×128×128 scale features corresponding to the z9 branch and the 64×128×128 scale features of the z6 branch itself. After fusion, a DWConv + RVM convolution operation and dual-path sampling are performed to obtain 32×256×256 scale features. Further, the 32×256×256 scale features obtained from the z6 branch are fused with the 32×256×256 scale features from the z3 branch (first result). After fusion, a DWConv + RVM convolution operation and dual-path sampling are performed to obtain 16×512×512 scale features.

[0072] Furthermore, the 128×64×64 scale features in the z12 branch are subjected to DWConv + RVM convolution and dual-path sampling, repeated three times to obtain the corresponding 16×512×512 scale features; the 64×128×128 scale features in the z9 branch are subjected to DWConv + RVM convolution and dual-path sampling, repeated twice to obtain the corresponding 16×512×512 scale features; and the 32×256×256 scale features in the z6 branch are subjected to one DWConv + RVM convolution and dual-path sampling to obtain the corresponding 16×512×512 scale features. This determines the second result.

[0073] Finally, by skip connections, the 16×512×512 scale features corresponding to the three branches z6, z9, and z12 are reused in the fifth branch, and then fused with the 16×512×512 scale features in the fifth branch and the 16×512×512 scale features corresponding to the z3 branch to obtain the final fused 16×512×512 scale features.

[0074] S104: Perform a preset number of target structure convolution operations and dual-path sampling operations on the upsampled features to be fused to obtain the second result, and then perform convolution operations on the CT image samples through the target structure to obtain the third result.

[0075] Specifically, the upsampled feature to be fused is the final upsampled feature fused with the final upsampled feature at the previous scale. The scale of the upsampled feature to be fused is lower than that of the final upsampled feature at the previous scale; the higher the feature scale, the higher the spatial resolution of the feature. After the first result is determined, the upsampled feature to be fused undergoes target structure convolution and dual-path sampling operations, repeated a preset number of times to obtain the second result. The preset number of times is a positive integer greater than 1. Finally, the CT image samples are convolved with the target structure to obtain the third result. The feature scale of each second result is consistent with the feature scale of the third result. It should be noted that the feature scale of the third result is higher than that of the first result.

[0076] S105: Perform feature merging on the first result, each of the second results, and the third result to obtain the merged features at the final scale. Based on the merged features, obtain the iodine density generation data and atomic number generation data corresponding to the CT image samples.

[0077] Specifically, the first result, each of the second results, and the third result are feature merged. The process is as follows: The first result is convolved using the target structure, followed by dual-path sampling to obtain a result with the same feature scale as the third result. This result is then concatenated with each of the second and third results to obtain the merged features at the final scale. Further, the merged features are convolved using the target structure to obtain the post-operated result. This post-operated result is then convolved using a 1*1 convolution kernel, primarily transforming the channel dimensions without changing the spatial resolution. Finally, it is processed using either the Sigmoid or Tanh activation function to obtain the iodine density generation data and atomic number generation data corresponding to the CT image samples. Specifically, this includes the iodine density generation values ​​at various locations within the body, and the atomic number generation data includes the atomic number generation values ​​at various locations within the body.

[0078] S106: Based on iodine density generation data, atomic number generation data, real iodine density data, and real atomic number data, the initial network model is trained under supervision using a preset loss function to obtain the target generation model of iodine density map and atomic number map.

[0079] Specifically, after determining the iodine density generation data and atomic number generation data corresponding to the CT image samples, the initial network model is trained under supervision using a preset loss function, combining the actual iodine density data and atomic number data. One feasible implementation is as follows: The iodine density generation data, i.e., the iodine density generation values ​​at various locations within the body, is substituted into a preset first calculation formula to obtain the first error corresponding to the iodine density generation data. The first calculation formula is:

[0080] ;

[0081] In the formula, Indicating iodine density generation data The iodine density generated at the point; This indicates that the actual data on iodine density is in The true value of iodine density at the point; Mask is a matrix that controls the weights of the contribution of the generation error of iodine density at different locations to the overall mean absolute error, where α and β both represent weights. This represents the first error corresponding to the generated iodine density value, which is the calculated Mean Absolute Error (MAE). It should be noted that since the true iodine density values ​​across the body are mostly distributed between 0 and 0.6, with only a few falling between 1 and 32, the first error is not determined by directly averaging the absolute differences between the generated and true iodine density values ​​across all locations within the body. Instead, a weighted average method is used to calculate the average MAE, minimizing the error. Specifically, the weights of pixels with true iodine density values ​​greater than 1 are considered. Set α, with a value of 20, to the weight of pixels where the true iodine density is no greater than 1. Set it to β, where β has a value of 1.

[0082] Furthermore, after determining the first error corresponding to the iodine density generation data, the atomic number generation data is substituted into the preset second calculation formula to obtain the corresponding second error. The second calculation formula is as follows:

[0083] ;

[0084] In the formula, This represents the set of pixel locations within the body area corresponding to a CT image sample. This represents the total number of pixels within the body area. This indicates the position of a pixel within the body's dimensions. Indicating atomic number generation data The generated atomic number value at the point; Indicating the true atomic number data The true value of the atomic number at the point.

[0085] Finally, the determined first and second errors are substituted into the preset loss function. During training, the loss function is continuously minimized to achieve supervised training of the initial network model. The model parameters are continuously adjusted and optimized using the backpropagation gradient algorithm, ultimately yielding the target generative model for the iodine density map and atomic number map. The loss function is:

[0086] ;

[0087] In the formula, Loss represents the loss value. This indicates the first error corresponding to the iodine density generation data. This represents the second error corresponding to the atomic number generation data.

[0088] The implementation principle of the iodine density map and atomic number map generation model training method in this application is as follows: Convolution operation is performed on the target features through the target structure to obtain global context features with fewer parameters, capturing more long-range information. Then, a dual-path sampling operation is performed on the convolution result to obtain the final upsampled features at a single scale. This dual-path sampling method avoids excessive smoothing of the generated results due to linear interpolation and introduces randomized branches, making the generated image closer to the real image. Furthermore, the feature information of the final upsampled features corresponding to the first result, each second result, and the third result are contributed to the final scale, improving the image reconstruction quality and thus enhancing the performance of the initial network model. Then, the initial network model is trained using a loss function to further improve the model's performance. Subsequently, this target generation model can quickly and accurately generate iodine density maps and atomic number maps based on CT images without the need for expensive equipment, thus reducing the cost of generating atomic number maps and iodine density maps to a certain extent.

[0089] See Figure 4 This application discloses a flowchart illustrating a method for generating iodine density maps and atomic number maps. This method can be implemented using a computer program or run on a training device for generating iodine density maps and atomic number maps based on the von Neumann architecture. The computer program can be integrated into applications or run as a standalone utility application, specifically including:

[0090] S201: Obtain the CT image to be analyzed and the target generative model obtained by training the generative model of iodine density map and atomic number map in the above embodiment.

[0091] S202: Input the CT image to be analyzed into the target generation model to obtain the corresponding iodine density map and atomic number map.

[0092] Specifically, the CT image to be analyzed is one that requires atomic number and iodine density analysis based on its own characteristics. The iodine density map and atomic number map generation method disclosed in this application also uses a server as the execution entity. The server is wirelessly connected to a terminal, which has a client installed related to iodine density map and atomic number map generation. The server is the backend server for the client, and the terminal is a personal computer or tablet. Personnel send the CT image to be analyzed to the server via the terminal. After receiving the CT image, the server retrieves the target generation model from its database, trained using the iodine density map and atomic number map generation model training method described in the above embodiments. Further, the CT image to be analyzed is input into the target generation model, ultimately generating the corresponding iodine density map and atomic number map, thereby determining the atomic number map and iodine density map at a lower cost.

[0093] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0094] Please see Figure 5 This is a schematic diagram of the structure of the iodine density map and atomic number map generation model training device provided in this application embodiment. This device for generating iodine density maps and atomic number maps can be implemented as all or part of the device through software, hardware, or a combination of both. The device includes a data acquisition module 11, a feature extraction module 12, a feature fusion module 13, a result determination module 14, a result generation module 15, and a model training module 16.

[0095] Data acquisition module 11 is used to acquire target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number.

[0096] The feature extraction module 12 is used to input CT image samples into a preset initial network model and obtain at least one branch feature through the Transformer encoder in the initial network model.

[0097] The feature fusion module 13 is used to input branch features into the decoder in the initial network model, determine the branch features as target features, perform convolution operation on the target features through a preset target structure to obtain the convolution result, perform dual-path sampling operation on the convolution result to obtain intermediate upsampled features, determine the intermediate upsampled features as target features, and repeat the steps of performing convolution operation on the target features through a preset target structure to obtain the convolution result, to obtain the final upsampled features at a single-layer scale, and perform multi-scale feature fusion processing on the final upsampled features at different scales to obtain the first result;

[0098] The result determination module 14 is used to perform a preset number of target structure convolution operations and dual-path sampling operations on the upsampled features to be fused to obtain a second result, and to perform convolution operations on CT image samples through the target structure to obtain a third result. The upsampled features to be fused are the final upsampled features fused with the final upsampled features at the previous scale.

[0099] The result generation module 15 is used to merge the features of the first result, each of the second results and the third result to obtain the merged features at the final scale. Based on the merged features, the iodine density generation data and atomic number generation data corresponding to the CT image sample are obtained.

[0100] The model training module 16 is used to supervise the training of the initial network model based on iodine density generated data, atomic number generated data, real iodine density data, and real atomic number data, and to obtain the target generation model of iodine density map and atomic number map through a preset loss function.

[0101] Optional, feature fusion module 13, specifically used for:

[0102] The convolution result is sequentially subjected to convolution operation, activation function processing and bilinear interpolation processing to obtain the first processing result, and the first processing result is then subjected to convolution operation to obtain the first path sampling result;

[0103] The convolution result is sequentially subjected to convolution operation, activation function processing and pixel shuffling and rearrangement processing to obtain the second processing result. The second processing result is then subjected to convolution operation to obtain the second path sampling result.

[0104] The first upsampling result and the second upsampling result are merged to obtain the intermediate upsampling feature.

[0105] Optional, feature fusion module 13, specifically used for:

[0106] Establish jump connections between the characteristics of each branch;

[0107] The final upsampled features at the current scale are fused with the final upsampled features at the next scale to obtain the initial fusion result.

[0108] By using skip connections, at least one intermediate upsampled feature corresponding to the final upsampled feature of the upper-level scale of the current scale is obtained;

[0109] At least one feature to be fused is determined from each intermediate upsampled feature, and each feature to be fused is fused with the initial fusion result. The step of fusing the final upsampled feature at the current scale with the final upsampled feature at the next scale is repeated to obtain the first result.

[0110] Optional, such as Figure 6 As shown, the device also includes a first computing module 17, specifically used for:

[0111] Substituting the iodine density generation data into the preset first calculation formula, the corresponding first error is obtained. The first calculation formula is:

[0112] ;

[0113] In the formula, Indicating iodine density generation data The iodine density generated at the point; This indicates that the actual data on iodine density is in The true value of iodine density at the point; Mask is a matrix that controls the weights of the contribution of the generation error of iodine density at different locations to the overall mean absolute error, where α and β both represent weights.

[0114] Optionally, the device also includes a second computing module 18, specifically used for:

[0115] Substituting the generated atomic number data into the preset second calculation formula, we obtain the corresponding second error. The second calculation formula is as follows:

[0116] ;

[0117] In the formula, This represents the set of pixel locations within the body area corresponding to a CT image sample. This represents the total number of pixels within the body area. This indicates the position of a pixel within the body's dimensions. Indicating atomic number generation data The generated atomic number value at the point; Indicating the true atomic number data The true value of the atomic number at the point.

[0118] Please see Figure 7 This is a schematic diagram of the structure of the apparatus for generating iodine density maps and atomic number maps provided in an embodiment of this application. This apparatus for generating iodine density maps and atomic number maps can be implemented as all or part of a device through software, hardware, or a combination of both. The device includes an information acquisition module 21 and a model prediction module 22.

[0119] The information acquisition module 21 is used to acquire the CT image to be analyzed and the target generative model obtained by the generative model training method of the iodine density map and atomic number map in the above embodiment.

[0120] The model prediction module 22 is used to input the CT image to be analyzed into the target generation model to obtain the corresponding iodine density map and atomic number map.

[0121] It should be noted that the above-described iodine density map and atomic number map generation model training device, when executing the iodine density map and atomic number map generation model training method, is only illustrated by the division of the above-described functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the above-described iodine density map and atomic number map generation model training device and the iodine density map and atomic number map generation model training method embodiment belong to the same concept, and their implementation process is detailed in the method embodiment, which will not be repeated here.

[0122] This application also discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the iodine density map and atomic number map generation model training method and generation method described in the above embodiments.

[0123] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0124] The above-described iodine density map and atomic number map generation model training method and generation method are stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above methods.

[0125] This application also discloses an electronic device in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, it implements the above-mentioned method for training and generating the generation model of the iodine density map and atomic number map.

[0126] The electronic device can be a desktop computer, a laptop computer, or a cloud server, and includes, but is not limited to, a processor and a memory. For example, the electronic device may also include input / output devices, network access devices, and buses.

[0127] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0128] The memory can be an internal storage unit of an electronic device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the electronic device. Furthermore, the memory can be a combination of an internal storage unit and an external storage device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0129] In this electronic device, the training method and generation method of the generation model of the iodine density map and atomic number map of the above embodiments are stored in the memory of the electronic device and are loaded and executed on the processor of the electronic device for convenient use.

[0130] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for training a generative model of iodine density maps and atomic number maps, characterized in that, The method includes: Acquire target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number; The CT image samples are input into a preset initial network model, and at least one branch feature is obtained through the Transformer encoder in the initial network model; The branch features are input into the decoder in the initial network model, and the branch features are determined as target features. The target features are convolved through a preset target structure to obtain the convolution result. The convolution result is then subjected to dual-path sampling to obtain intermediate upsampled features. The intermediate upsampled features are determined as target features, and the steps of convolving the target features through the preset target structure to obtain the convolution result are repeated to obtain the final upsampled features at a single-layer scale. Multi-scale feature fusion processing is then performed on the final upsampled features at different scales to obtain the first result. The target structure convolution operation and dual-path sampling operation are performed on the upsampled feature to be fused a preset number of times to obtain a second result. The CT image sample is then convolved with the target structure to obtain a third result. The upsampled feature to be fused is the final upsampled feature fused with the final upsampled feature at the previous scale. The first result, each of the second results, and the third result are combined to obtain the final scale of the combined features. Based on the combined features, the iodine density generation data and atomic number generation data corresponding to the CT image sample are obtained. Based on the iodine density generation data, the atomic number generation data, the real iodine density data, and the real atomic number data, the initial network model is trained under supervision using a preset loss function to obtain the target generation model of the iodine density map and the atomic number map.

2. The method for training the generation model of iodine density map and atomic number map according to claim 1, characterized in that, The step of performing a dual-path sampling operation on the convolution result to obtain intermediate upsampled features specifically includes: The convolution result is sequentially subjected to convolution operation, activation function processing and bilinear interpolation processing to obtain a first processing result, and the first processing result is then subjected to convolution operation to obtain a first path sampling result; The convolution result is sequentially subjected to convolution operation, activation function processing and pixel shuffling and rearrangement processing to obtain a second processing result, and the second processing result is subjected to convolution operation to obtain a second path sampling result; The first upsampling result and the second upsampling result are merged to obtain the intermediate upsampling feature.

3. The method for training the generation model of iodine density map and atomic number map according to claim 1, characterized in that, The process of performing multi-scale feature fusion on the final upsampled features at different scales to obtain the first result specifically includes: Establish jump connections between the features of each branch; The final upsampled features at the current scale are fused with the final upsampled features at the next lower scale to obtain the initial fusion result. Through the skip connection, at least one intermediate upsampled feature corresponding to the final upsampled feature of the upper-level scale of the current scale is obtained; At least one feature to be fused is determined from each of the intermediate upsampled features, and each of the fused features is fused with the initial fusion result. The step of fusing the final upsampled feature at the current scale with the final upsampled feature at the next scale of the current scale is repeated to obtain the first result.

4. The method for training the generation model of iodine density map and atomic number map according to claim 1, characterized in that, The loss function is: ； In the formula, Loss represents the loss value. This indicates the first error corresponding to the iodine density generation data. This represents the second error corresponding to the atomic number generation data.

5. The method for training the generation model of iodine density map and atomic number map according to claim 4, characterized in that, The method further includes: Substituting the iodine density generation data into a preset first calculation formula yields the corresponding first error. The first calculation formula is as follows: ； In the formula, Indicating iodine density generation data The iodine density generated at the point; This indicates that the actual data on iodine density is in The true value of iodine density at the point; Mask is a matrix that controls the weights of the contribution of the generation error of iodine density at different locations to the overall mean absolute error, where α and β both represent weights.

6. The method for training the generation model of iodine density map and atomic number map according to claim 4, characterized in that, The method further includes: Substituting the atomic number generation data into a preset second calculation formula yields the corresponding second error. The second calculation formula is as follows: ； In the formula, This represents the set of pixel locations within the body area corresponding to a CT image sample. This represents the total number of pixels within the body area. This indicates the position of a pixel within the body's dimensions. Indicating atomic number generation data The generated atomic number value at the point; Indicating the true atomic number data The true value of the atomic number at the point.

7. A method for generating an iodine density map and an atomic number map, characterized in that, The method includes: The method involves acquiring the CT image to be analyzed and obtaining the target generative model using the generative model training method for iodine density map and atomic number map as described in any one of claims 1 to 6. The CT image to be analyzed is input into the target generation model to obtain the corresponding iodine density map and atomic number map.

8. A training device for generating models of iodine density maps and atomic number maps, characterized in that, include: The data acquisition module (11) is used to acquire target training data, which includes at least one set of CT image samples and corresponding real data of iodine density and atomic number. The feature extraction module (12) is used to input the CT image sample into a preset initial network model and obtain at least one branch feature through the Transformer encoder in the initial network model. The feature fusion module (13) is used to input the branch features into the decoder in the initial network model, determine the branch features as target features, perform convolution operation on the target features through a preset target structure to obtain the convolution result, and perform dual-path sampling operation on the convolution result to obtain intermediate upsampled features, determine the intermediate upsampled features as target features, and repeat the step of performing convolution operation on the target features through a preset target structure to obtain the convolution result, to obtain the final upsampled features at a single-layer scale, and perform multi-scale feature fusion processing on the final upsampled features at different scales to obtain the first result; The result determination module (14) is used to perform a preset number of target structure convolution operations and dual-path sampling operations on the upsampled features to be fused to obtain a second result, and to perform convolution operations on the CT image samples through the target structure to obtain a third result. The upsampled features to be fused are the final upsampled features fused with the final upsampled features at the previous scale. The result generation module (15) is used to merge the features of the first result, each of the second results and the third result to obtain the merged features of the final scale, and based on the merged features, to obtain the iodine density generation data and atomic number generation data corresponding to the CT image sample; The model training module (16) is used to supervise the training of the initial network model based on the iodine density generation data, the atomic number generation data, the iodine density real data and the atomic number real data, through a preset loss function, to obtain the target generation model of the iodine density map and the atomic number map.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it implements the method of any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor loads and executes the computer program, it implements the method of any one of claims 1-7.

Images