An electron density map generation method, system, electronic device and storage medium

Abstract

The application discloses a method and system for generating an electron density map, an electronic device and a storage medium, and relates to the field of image processing. In the method, a CT image is input into an encoder, a first local feature of an input feature is extracted by using a residual network path in the encoder, a first global feature is extracted by using a dense connection path, a first sampling feature is obtained by down-sampling after fusion, and the first sampling feature is used as input of a next layer until a deepest layer. In the deepest layer, a local feature and a global feature are extracted by using the residual network path and the dense connection path respectively, and the local feature and the global feature are fused, a second fusion feature is input into a first layer of a decoder, the input feature is up-sampled in the decoder to obtain a second sampling feature, the second sampling feature is superimposed with the first local feature of a corresponding layer of the encoder to obtain a new second fusion feature as input of a next layer until an output layer, and the second fusion feature in the output layer is subjected to convolution processing to obtain an electron density map. The technical scheme provided by the application can accurately generate an electron density map.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an electron density map generation method and system, an electronic device, and a storage medium. BACKGROUND

[0002] In the field of medical imaging and radiotherapy, electron density maps can intuitively reflect the electron density distribution of human tissues, which plays a key role in disease diagnosis, radiotherapy planning, and other aspects. It helps doctors better understand the characteristics of the lesion site, so as to develop more accurate and effective treatment plans, and has a positive and far-reaching impact on improving medical quality and patient treatment effect.

[0003] In the prior art, traditional electron density map generation methods rely on a single feature extraction path, such as using only convolutional neural networks to extract local features or only using downsampling to obtain global features, resulting in incomplete use of feature information, which makes the generated electron density map lack in detail restoration and overall structure accuracy, and is difficult to meet the needs of high-precision medical diagnosis and treatment planning.

[0004] Therefore, there is an urgent need for a method for generating an electron density map with higher accuracy. SUMMARY

[0005] The present application provides an electron density map generation method and system, an electronic device, and a storage medium, which accurately generates an electron density map from a CT (Computed Tomography) image, improving the accuracy and efficiency of electron density map generation.

[0006] In a first aspect of the present application, an electron density map generation method is provided, applied to an electron density generation model, the electron density generation model comprising an encoder and a decoder, the method comprising:

[0007] In response to the received CT image, extracting initial features of the CT image, and inputting the initial features as input features of the first layer of the encoder to the encoder;

[0008] In the first target layer of the encoder, the first local features of the input features of the first target layer are extracted through a residual network path, the first global features of the input features of the first target layer are extracted through a dense connection path, the first local features and the first global features are fused to obtain first fusion features, the first fusion features are downsampled to obtain first sampling features, and the first sampling features are input as input features of the next layer until the next layer is the deepest layer, and the first target layer is any layer in the encoder except the deepest layer;

[0009] In the deepest layer, a second local feature of the input feature of the deepest layer is extracted through a residual network path, a second global feature of the input feature of the deepest layer is extracted through a densely connected path, the second local feature and the second global feature are fused to obtain a second fusion feature, and the second fusion feature is input as an input feature of a first layer of the decoder into the decoder;

[0010] In the second target layer of the decoder, a second sampling feature is obtained by upsampling the input feature of the second target layer, a second fusion feature is obtained by superimposing the second sampling feature and the first local feature of the corresponding layer of the encoder, the second fusion feature is taken as an input feature of a next layer, until the next layer is an output layer, and the second fusion feature is subjected to convolution processing to obtain an electron density map, and the second target layer is any layer in the decoder.

[0011] Optionally, the extracting the first local feature of the input feature of the first target layer through the residual network path comprises:

[0012] The input feature is subjected to storage order adjustment and arrangement change to obtain an initial local feature sequence;

[0013] The initial local feature sequence is normalized to obtain a first local feature sequence, and the first local feature sequence is subjected to an activation function, a preset sequence modeling architecture (Mamba module) and normalization to obtain a second local feature sequence;

[0014] The initial local feature sequence and the second local feature sequence are superimposed to obtain a third local feature sequence, and the third local feature sequence is subjected to rearrangement and storage order change to obtain the first local feature.

[0015] Optionally, the first local feature sequence is subjected to an activation function, a preset sequence modeling architecture and normalization to obtain a second local feature sequence, comprising:

[0016] An input local feature sequence of the preset sequence modeling architecture is normalized to obtain a first local sub-feature sequence, and the first local sub-feature sequence is processed through a first preset semantic segmentation module to obtain a second local sub-feature sequence;

[0017] The input local feature sequence is subjected to scale transformation to obtain a third local sub-feature sequence, and the third local sub-feature sequence and the second local sub-feature sequence are superimposed to obtain a fourth local sub-feature sequence;

[0018] The fourth local sub-feature sequence is subjected to normalization and linear transformation to obtain an output local feature sequence.

[0019] Optionally, the extracting the first global feature of the input feature of the first target layer through the dense connection path comprises:

[0020] The input feature is stored in sequence adjustment and arrangement change to obtain a first global feature sequence;

[0021] In the first branch, the first global feature sequence is normalized to obtain a second global feature sequence, and the second global feature sequence is processed through a second preset semantic segmentation module to obtain a third global feature sequence;

[0022] In the second branch, the first global feature sequence is linearly transformed to obtain a fourth global feature sequence, and the fourth global feature sequence is proportionally transformed;

[0023] The third global feature sequence and the fourth global feature sequence after proportion transformation are superimposed to obtain a fifth global feature sequence, and the fifth global feature sequence is normalized, linearly changed and rearranged to obtain the first global feature.

[0024] Optionally, the processing the second global feature sequence through the second preset semantic segmentation module to obtain the third global feature sequence comprises:

[0025] In the third branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a first global sub-feature sequence, and the first global sub-feature sequence is processed through an activation function to obtain a second global sub-feature sequence;

[0026] In the fourth branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a third global sub-feature sequence, and the third global sub-feature sequence is processed through modified shape, transformed storage sequence and depth separable convolution to obtain a fourth global sub-feature sequence, the fourth global sub-feature sequence is processed through an activation function to obtain a fifth global sub-feature sequence, the fifth global sub-feature sequence is processed through a state space model to obtain a sixth global sub-feature sequence, and the sixth global sub-feature sequence is normalized to obtain a seventh global sub-feature sequence;

[0027] The second global sub-feature sequence and the seventh global sub-feature sequence are multiplied by position to obtain an eighth global sub-feature sequence;

[0028] The eighth global sub-feature sequence is linearly transformed to obtain the third global feature sequence.

[0029] Optionally, the fusing the first local feature and the first global feature to obtain a first fusion feature comprises:

[0030] In the first target layer of the encoder, multiple sets of first intermediate features of the first size obtained by the encoder at the same level are spliced ​​together along the channel dimension to form a spliced ​​feature;

[0031] A linear transformation is performed on the splicing feature to reduce the number of output channels, forming a densely connected path feature of the second size;

[0032] The input channel number is linearly mapped by a channel adaptation layer while keeping the spatial resolution of the output features unchanged. The densely connected path features are then fused with the first local features to form the first fused feature.

[0033] Optionally, the method further includes:

[0034] A dataset is constructed by collecting historical CT images and the corresponding electron density maps of the historical CT images, and the original electron density generation model is trained using the dataset;

[0035] A loss function is constructed using the average absolute error of the difference between the estimated and true values ​​within a region containing human body structures. The value of the loss function is calculated, and when the value of the loss function is less than a preset threshold, the trained original electron density generation model is used as the electron density generation model.

[0036] The calculation of the value of the loss function includes:

[0037] The value of the loss function is calculated using the following formula:

[0038] Loss = MAE (Ω) Normal )+α·MAE(Ω Air ) + β·MAE (Ω Bone );

[0039] Where Loss represents the loss function, Ω Air Ω represents the set of all pixels within the cavity region. Bone Ω represents the set of all pixels within the region where the HU value is higher than a preset threshold. Normal MAE (Ω) represents the set of all pixels within a region of the human body structure, excluding cavities and regions with HU values ​​higher than a preset threshold. Air ) represents the mean absolute error (MAE) of the difference between the estimated value and the true value of the set of all pixels within the cavity region. Bone ) represents the mean absolute error (MAE) of the set of estimated values ​​and true values ​​of all pixels in the region where the HU value is higher than a preset threshold. Normal) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the region of the human body structure except the cavity region and the region with a HU value higher than the preset threshold, and a and β are non-negative real numbers.

[0040] In a second aspect of the present application, an electron density map generation system is provided, comprising an extraction module, a fusion module, an execution module, and a generation module, wherein:

[0041] The extraction module is configured to extract initial features of the received CT image in response to the received CT image, and input the initial features as input features of the first layer of the encoder into the encoder.

[0042] The fusion module is configured to extract first local features of the input features of the first target layer through a residual network path, extract first global features of the input features of the first target layer through a dense connection path, fuse the first local features and the first global features to obtain first fusion features, down-sample the first fusion features to obtain first sampling features, and input the first sampling features as input features of the next layer until the next layer is the deepest layer, in the first target layer of the encoder.

[0043] The execution module is configured to extract second local features of the input features of the deepest layer through a residual network path, extract second global features of the input features of the deepest layer through a dense connection path, fuse the second local features and the second global features to obtain second fusion features, and input the second fusion features as input features of the first layer of the decoder into the decoder, in the deepest layer.

[0044] The generation module is configured to up-sample the input features of the second target layer to obtain second sampling features, superimpose the second sampling features and the first local features of the corresponding layer of the encoder to obtain second fusion features, and input the second fusion features as input features of the next layer until the next layer is an output layer, in the second target layer of the decoder.

[0045] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface, and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method described in any one of the above aspects.

[0046] In a fourth aspect of the present application, a computer-readable storage medium is provided, which stores instructions that, when executed, perform the method of any of the above.

[0047] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0048] 1. The local features (focusing on detail information) extracted by the residual network path and the global features (focusing on context information) extracted by the dense connection path are fused, so that the model can capture both the local anatomical structure and the global spatial relationship of the CT image, and the accuracy of the electron density map is improved. In the decoder, the up-sampled features are superimposed with the local features of the corresponding level in the encoder, so as to realize the complementarity of deep semantic information and shallow detail information, avoid the loss of details caused by down-sampling, and enhance the edge definition and texture authenticity of the electron density map.

[0049] 2. The dense connection path promotes feature reuse by directly transmitting the output of each layer to all subsequent layers, alleviates the gradient vanishing problem in deep networks, makes the model easier to train and converges faster. The concatenation operation of multi-level features in the dense connection path integrates information of different receptive fields, enhances the modeling ability of the model for complex anatomical structures, and is especially suitable for regions with blurred organ boundaries or small density differences in CT images.

[0050] 3. The residual network path solves the degradation problem in deep network training, ensuring that the model can still learn stably under deep structure. The residual path focuses on extracting local details (such as the microstructure of bones and blood vessels), and after fusion with the global features of the dense connection path, it can improve the ability of the electron density map to distinguish high-density regions (such as calcification) and low-density regions (such as alveoli).

[0051] 4. In the decoder, the up-sampled features are superimposed with the local features of the corresponding level in the encoder, ensuring the alignment of the features in the spatial dimension and avoiding artifacts or distortion caused by direct up-sampling. Through multi-level up-sampling and feature fusion, the spatial resolution of the electron density map is gradually restored, and finally a density map with the same size as the input CT image is output, meeting the needs of anatomical structure positioning in clinical diagnosis.

[0052] 5. The local feature extraction of the residual path and the global information integration of the dense connection path make the model more robust to noise (such as electronic noise) and artifacts (such as metal artifacts) in the CT image, generating a smoother density map. By fusing multi-scale features, the model can adapt to CT images under different scanning devices, parameters or patient positions, improving the generalization performance in cross-dataset testing. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1is a flowchart of a method for generating an electron density map according to an embodiment of the present application;

[0054] Figure 2 is a framework diagram of an electron density generation model according to an embodiment of the present application;

[0055] Figure 3 is an architecture diagram of a ResMamba module according to an embodiment of the present application;

[0056] Figure 4 is an architecture diagram of a Mamba module according to an embodiment of the present application;

[0057] Figure 5 is an architecture diagram of a DenseMamba module according to an embodiment of the present application;

[0058] Figure 6 is an architecture diagram of a Dense VSS module according to an embodiment of the present application;

[0059] Figure 7 is a comparison diagram of a real electron density map and an electron density map generated according to an embodiment of the present application;

[0060] Figure 8 is a comparison diagram of dose prediction results of a real electron density map and an electron density map generated according to an embodiment of the present application;

[0061] Figure 9 is a module diagram of an electron density map generation system according to an embodiment of the present application;

[0062] Figure 10 is a structural diagram of an electronic device according to an embodiment of the present application.

[0063] Legend: 901, extraction module; 902, fusion module; 903, execution module; 904, generation module; 1001, processor; 1002, communication bus; 1003, user interface; 1004, network interface; 1005, memory. DETAILED DESCRIPTION

[0064] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only some of the embodiments of the present application, not all.

[0065] In the description of the embodiments of the present application, the words "for example" or "such as" are used to represent an example, an illustration, or a description. Any embodiment or design scheme described as "for example" or "such as" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of the words "for example" or "such as" is intended to present the relevant concept in a specific manner.

[0066] In the description of the embodiments of the present application, the term "a plurality of" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly indicating the technical features indicated. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0067] The embodiments disclose a method for generating an electron density map, Figure 1 is a flowchart of a method for generating an electron density map disclosed by the embodiments of the present application, as Figure 1 shown, the method comprises the following steps:

[0068] S101, in response to the received CT image, extracting initial features of the CT image, and inputting the initial features as input features of the first layer of the encoder into the encoder;

[0069] S102, in the first target layer of the encoder, extracting first local features of the input features of the first target layer through a residual network path, extracting first global features of the input features of the first target layer through a dense connection path, fusing the first local features and the first global features to obtain first fusion features, down-sampling the first fusion features to obtain first sampling features, and inputting the first sampling features as input features of the next layer until the next layer is the deepest layer, and the first target layer is any layer in the encoder except the deepest layer;

[0070] S103, in the deepest layer, extracting second local features of the input features of the deepest layer through a residual network path, extracting second global features of the input features of the deepest layer through a dense connection path, fusing the second local features and the second global features to obtain second fusion features, and inputting the second fusion features as input features of the first layer of the decoder into the decoder;

[0071] S104. In the second target layer of the decoder, the input features of the second target layer are upsampled to obtain a second sampled feature. The second sampled feature is superimposed with the first local feature of the corresponding layer of the encoder to obtain a second fused feature. The second fused feature is used as the input feature of the next layer until the next layer is the output layer. The second fused feature is convolved to obtain an electronic density map. The second target layer is any layer in the decoder.

[0072] Figure 2 This is a schematic diagram of the framework of the electron density generation model disclosed in the embodiments of this application, combined with... Figure 2 The above steps will be explained.

[0073] The model receives raw CT image data (typically a sequence of 3D or 2D slices) from a CT scanning device. Preliminary feature extraction is performed on the CT images using convolutional layers (e.g., 3×3 convolutional kernels) to generate low-level features (such as edges and textures). Figure 2 As shown, DWConv stands for Depthwise Separable Convolution, a commonly used convolution operation in neural networks. Depthwise Separable Convolution consists of two parts: depthwise convolution and pointwise convolution. Depthwise Convolution: In depthwise convolution, each input channel is operated on using a convolution such as a 3x3 kernel, with each channel using the same convolution. Pointwise Convolution: Pointwise convolution is a 1×1 convolution operation that performs convolution on all channels of the input at each position. ResDWConv represents a depthwise separable convolution with residual connections, consisting of one depthwise convolution module and three pointwise convolution modules, and introducing residual connections. Residual connections directly add the input to the output through skip connections, helping to alleviate the gradient vanishing problem in deep networks and enhancing the stability of model training. The initial features are used as input to the first layer of the encoder, initiating a layer-by-layer feature abstraction process. The original CT image is converted into a feature representation suitable for neural network processing, laying the foundation for subsequent local and global feature extraction. The first layer of an encoder typically uses a convolutional kernel with a large receptive field to quickly capture the macroscopic structure of an image.

[0074] like Figure 2As shown, ResMamba constitutes the residual network path, extracting local features through Residual Blocks, which usually contain two convolutional layers and an identity mapping for preserving shallow details (e.g., bone boundaries, vessel branches). DenseMamba constitutes the dense connection path, extracting global features through Dense Blocks, where the output of each layer is directly connected to all subsequent layers, enabling feature reuse and capturing global context information (e.g., overall organ morphology, density distribution). Local and global features are concatenated along the channel dimension (e.g., Concatenation in Figure 2 ) or fused through weighted summation (e.g., attention mechanism), generating first fusion features containing multi-scale information. First fusion features are downsampled (e.g., Pooling in Figure 2 , such as max-pooling or stride convolution), reducing feature map size and increasing channel number, generating first sampling features as input for the next layer. The above process is repeated until the deepest layer (the last layer of the encoder). The residual path can alleviate the gradient vanishing problem in deep networks, ensuring that detailed information is preserved in deep networks. The dense path can enhance global information transmission through feature reuse, improving the model's ability to model complex structures. Feature fusion: combining local and global features to improve the model's ability to recognize different scale structures in CT images.

[0075] At the deepest layer of the encoder, second local and global features are extracted through the residual and dense paths, respectively. The second local and global features are fused to generate second fusion features, which are input to the first layer of the decoder. The deepest layer features contain the highest level of semantic information (e.g., organ class, lesion area), and the fusion of local and global features provides rich context information for the decoder. The first layer of the decoder receives the deepest layer features, starting the feature recovery process layer by layer.

[0076] The input features of the second target layer are upsampled (e.g., UpSampling in Figure 2 , such as transpose convolution or interpolation), restoring the spatial resolution of the feature map. The upsampled features are superimposed with the first local features of the corresponding layer of the encoder (transmitted through the skip connection) (e.g., Figure 2The second fusion feature is generated by repeating the upsampling and feature superposition process until the output layer. In the output layer, the feature map is mapped to the electron density map by 1x1 convolution. Upsampling: gradually restores the spatial resolution, so that the output electron density map is consistent with the size of the input CT image. The details of the shallow layer of the encoder are introduced into the decoder through the skip connection to avoid the loss of details caused by downsampling and improve the edge definition of the density map. Channel compression and pixel value mapping are realized by 1x1 convolution to generate an electron density map that meets the clinical standards.

[0077] Optionally, the first local feature of the input feature of the first target layer extracted through the residual network path (ResMamba module) comprises:

[0078] The input feature is stored in a sequence adjustment and arrangement change to obtain an initial local feature sequence;

[0079] The initial local feature sequence is normalized to obtain a first local feature sequence, and the first local feature sequence is subjected to an activation function, a preset sequence modeling architecture (Mamba module) and normalization to obtain a second local feature sequence;

[0080] The initial local feature sequence and the second local feature sequence are superimposed to obtain a third local feature sequence, and the third local feature sequence is rearranged and the storage sequence is exchanged to obtain the first local feature.

[0081] Figure 3 is the architecture schematic diagram of the ResMamba module disclosed in the embodiments of the present application, as Figure 3As shown, ResMamba converts the storage order of the feature with size (B, C, H, W) to (B, H, W, C) and then reshapes it to a feature sequence M0 (i.e., the initial local feature sequence) with size (B, N, C). Wherein, B represents Batchsize, the number of samples loaded in one batch; C represents the number of feature channels; H represents the feature height; and W represents the feature width. K0 is normalized by the first GroupNorm on N feature vectors within each Batch to obtain a feature sequence K1 (i.e., the first local feature sequence) with size (B, N, C); the normalized feature sequence K1 is processed by the first ReLU activation function to obtain a feature sequence K2 with size (B, N, C), and the activation function adds nonlinear processing to enable the network to handle nonlinear complex problems. K2 is processed by the first preset sequence modeling architecture (i.e., the Mamba module) to obtain a feature sequence K3 with size (B, N, C); K3 is normalized by the second GroupNorm to obtain a feature sequence K4 with size (B, N, C); K4 is processed by the second ReLu activation function to obtain a feature sequence K5 with size (B, N, C); K5 is processed by the second Mamba module to obtain a feature sequence K6 (i.e., the second local feature sequence) with size (B, N, C); K6 and K0 are processed by the Sum to obtain K7; K7 is reshaped to a feature with size (B, H, W, C) and then permuted to obtain a feature with size (B, C, H, W) as the feature processed by the entire ResMamba module. The ResMamba module does not change the size of the input feature, and the output feature has the same size as the input feature; however, the residual connection is introduced in the middle to stack the input K0 and the K6 processed by a series of modules.

[0082] The residual connection allows the gradient to be directly backpropagated to the shallow layer, optimizing the training stability of the deep network. The information of the original input feature is preserved, avoiding excessive loss of information in deep processing and improving the integrity of feature expression. The model is more likely to learn the identity mapping, thus converging quickly in the early training stage. Through the preset sequence modeling architecture (such as the Mamba module), the local temporal or spatial dependence in the input feature is captured. The activation function is combined to introduce nonlinearity, enhancing the feature expression ability. The normalization operation accelerates the convergence and reduces the internal covariate shift, making the model more robust to changes in input distribution. By adjusting the storage order, the local neighborhood features can be reorganized for efficient processing by subsequent Mamba modules. Rearrangement may promote information exchange between different dimensions (such as channels and space), improving feature diversity. The adjusted order may be more suitable for parallel computing or hardware acceleration (such as the memory access mode of GPU).

[0083] Optionally, the obtaining the second local feature sequence by activating the first local feature sequence, the preset sequence modeling architecture and normalization comprises:

[0084] normalizing the input local feature sequence of the preset sequence modeling architecture to obtain a first local sub-feature sequence, processing the first local sub-feature sequence by a first preset semantic segmentation module to obtain a second local sub-feature sequence;

[0085] scaling the input local feature sequence to obtain a third local sub-feature sequence, and superimposing the third local sub-feature sequence and the second local sub-feature sequence to obtain a fourth local sub-feature sequence;

[0086] normalizing and linearly transforming the fourth local sub-feature sequence to obtain an output local feature sequence.

[0087] Figure 4 is an architecture schematic diagram of the Mamba module disclosed in the embodiments of the present application, as shown in Figure 4 The input of the Mamba module is a feature sequence M0 with a size of (B, N, C in ) (i.e., the input local feature sequence). M0 is normalized by a first LayerNorm to obtain a feature sequence M1 with a size of (B, N, C in ) (i.e., the first local sub-feature sequence); M1 is processed by a VSS module (i.e., the first preset semantic segmentation module) to obtain a feature sequence M2 with a size of (B, N, C in ) (i.e., the second local sub-feature sequence); a Scale operation is a scaling operation on M0 (i.e., M0* learnable parameters) to obtain a feature sequence M3 with a size of (B, N, C in ) (i.e., the third local sub-feature sequence); M2 and M3 are processed by a superimposition module to obtain a feature sequence M4 with a size of (B, N, C in ) (i.e., the fourth local sub-feature sequence); M4 is normalized by a second LayerNorm to obtain a feature sequence M5 with a size of (B, N, C in ); M5 is linearly transformed (Linear) to obtain a feature sequence M6 with a size of (B, N, C out ) (i.e., the output local feature sequence); the Mamba module supports different channel number outputs, but in the ResMamba and DenseMamba, the Mamba module does not change the channel number, i.e., the output channel number is the same as the input channel number.

[0088] Normalization adjusts the input distribution to a standard range, mitigating internal covariate bias and accelerating model convergence. It avoids gradient vanishing or exploding due to input scale differences, which is crucial for deep networks or complex architectures (such as Mamba modules). It provides a consistent input range for subsequent VSS modules and linear transformations, improving model robustness. VSS modules may use structures such as convolutions, attention mechanisms, or gating units to focus on semantic information (such as texture, edges, and structures) in local features, improving feature discriminativity. VSS modules may balance feature complexity and computational efficiency by adjusting the number of channels (such as 1x1 convolutions). Residual paths allow gradients to be directly backpropagated to shallow layers, optimizing deep network training. It preserves original input information, preventing information loss during deep processing and improving feature integrity. If input adjustment involves scale transformations (such as upsampling / downsampling), features at different scales can be fused, enhancing the model's adaptability to multi-scale targets. Adjustment operations may reduce the number of parameters through linear transformations (such as 1x1 convolutions) while maintaining feature expressiveness. Output normalization further stabilizes the feature distribution, providing stable input for subsequent layers (such as classification heads and detection heads). Linear transformation maps features to the target space (such as the number of classification categories, detection box coordinates, etc.) or adjusts the number of channels to match the requirements of downstream tasks.

[0089] Optionally, the first global feature extracted from the input features of the first target layer through densely connected paths includes:

[0090] The first global feature sequence is obtained by adjusting and rearranging the storage order of the input features;

[0091] In the first branch, the first global feature sequence is normalized to obtain the second global feature sequence, and the second global feature sequence is processed by the second preset semantic segmentation module to obtain the third global feature sequence;

[0092] In the second branch, the first global feature sequence is linearly transformed to obtain the fourth global feature sequence, and the fourth global feature sequence is then proportionally transformed.

[0093] The third global feature sequence and the scaled-transformed fourth global feature sequence are superimposed to obtain the fifth global feature sequence. The fifth global feature sequence is then normalized, linearly transformed, and rearranged to obtain the first global feature.

[0094] Figure 5 This is a schematic diagram of the architecture of the DenseMamba module disclosed in the embodiments of this application, as shown below. Figure 5 As shown, the input features of DenseMamba are of size (B, C). in The feature Y0 of (H, W) is obtained by adjusting the storage order and changing the arrangement of Y0, resulting in a size of (B, N, C).in ) Y1 (i.e., the first global feature sequence); Y1 is processed in two ways, the left branch (i.e., the first branch) first normalizes Y1 through LayerNorm to obtain a feature sequence Y2 (i.e., the second global feature sequence) with a size of (B, N, C in ) Y2 (i.e., the second global feature sequence); Y2 is processed by a second preset semantic segmentation module (i.e., the Dense VSS module) to obtain a feature sequence Y3 (i.e., the third global feature sequence) with a size of (B, N, C out ) Y1 (i.e., the first global feature sequence); Y1 is processed in two ways, the left branch (i.e., the first branch) first normalizes Y1 through LayerNorm to obtain a feature sequence Y2 (i.e., the second global feature sequence) with a size of (B, N, C out ) Y4 (i.e., the fourth global feature sequence); Y4 is scaled (Scale, i.e., multiplied by a learnable parameter) and then added (Sum) to Y3 to obtain a feature sequence Y5 (i.e., the fifth global feature sequence) with a size of (B, N, C out ) Y1 (i.e., the first global feature sequence); Y1 is processed in two ways, the left branch (i.e., the first branch) first normalizes Y1 through LayerNorm to obtain a feature sequence Y2 (i.e., the second global feature sequence) with a size of (B, N, C out ) Y6; Y6 is linearly transformed to obtain a feature sequence Y7 with a size of (B, N, C out ) Y7; Y7 is processed through rearrangement and storage order adjustment to obtain a feature Y8 with a size of (B, C out , H, W); Y8 is the final output result of the DenseMamba. The DenseMamba module first processes the channels, and then processes the sequence after uniforming the number of expected output channels.

[0095] The original features are rearranged into a format suitable for subsequent processing by adjusting the storage order. The rearrangement changes can facilitate interactions between different dimensions or different position features, enhancing the expressive power of global features. The adjusted order can better match hardware acceleration patterns (such as memory continuous access of GPUs), reducing computational overhead. The VSS module can focus on semantic information in the global range (such as long-distance dependence, context relationship) through attention mechanisms, kernel convolution, or pyramid structures, enhancing the global discriminability of features. Input normalization alleviates internal covariate shift and speeds up the convergence of the VSS module. The VSS module can control feature complexity through channel adjustment, balancing computational efficiency and expressive power. Linear transformation maps features to a new space (such as reducing dimensionality to reduce redundancy, or increasing dimensionality to increase nonlinearity). If the linear transformation parameters are small, the basic features can be efficiently extracted as a supplement to the VSS module. By scaling or nonlinear activation, the feature distribution range can be controlled to avoid numerical instability when superimposed with the first branch features. The superposition operation realizes the dense connection of multi-level features, combining the global semantics of the VSS module and the basic features of the linear transformation, enhancing feature richness. Residual connection allows gradients to be directly backpropagated to the shallow layers of both branches, alleviating the vanishing gradient problem of deep networks. By preserving the original input features (through the linear transformation branch) and high-level semantic features (through the VSS branch), the information bottleneck of a single path is avoided.

[0096] Optionally, the processing of the second global feature sequence by the second preset semantic segmentation module to obtain a third global feature sequence comprises:

[0097] In the third branch of the second preset semantic segmentation module, the second global feature sequence is subjected to linear transformation to obtain a first global sub-feature sequence, and the first global sub-feature sequence is processed by an activation function to obtain a second global sub-feature sequence;

[0098] In the fourth branch of the second preset semantic segmentation module, the second global feature sequence is subjected to linear transformation to obtain a third global sub-feature sequence, and the third global sub-feature sequence is processed by modifying shape, transforming storage order, and depth separable convolution to obtain a fourth global sub-feature sequence, and the fourth global sub-feature sequence is processed by an activation function to obtain a fifth global sub-feature sequence, and the fifth global sub-feature sequence is processed by a state space model to obtain a sixth global sub-feature sequence, and the sixth global sub-feature sequence is subjected to normalization to obtain a seventh global sub-feature sequence;

[0099] The second global sub-feature sequence and the seventh global sub-feature sequence are multiplied position by position to obtain an eighth global sub-feature sequence;

[0100] The eighth global sub-feature sequence is subjected to linear transformation to obtain a third global feature sequence.

[0101] Figure 6 This is a schematic diagram of the architecture of the Dense VSS module disclosed in the embodiments of this application, as shown below. Figure 6 As shown, the DenseVSS input is of size (B, N, C) in The feature sequence Z0 (i.e., the second global feature sequence) is processed in two ways. Z0 undergoes a linear transformation via the left branch (i.e., the third branch) to obtain a sequence of size (B, N, C). out The feature sequence Z1 (i.e., the first global sub-feature sequence) is obtained by processing Z1 with the SiLU activation function to obtain (B, N, C) out The feature sequence Z2 (i.e., the second global sub-feature sequence) is obtained by linear transformation of Z0 through the right branch (i.e., the fourth branch), resulting in a sequence of size (B, N, C). out The feature sequence Z3 (i.e., the third global sub-feature sequence) is obtained by modifying the storage order of Z3 through shape transformation, resulting in (B, C) out After featuring H, W, etc., the sequence is processed by a depthwise separable convolution (DWConv) and then transformed back into a vector sequence, resulting in a sequence of size (B, N, C). out The feature sequence Z4 (i.e., the fourth global sub-feature sequence) is obtained by applying the SiLU activation function to Z4, resulting in a sequence of size (B, N, C). out The feature sequence Z5 (i.e., the fifth global sub-feature sequence) is processed by the SSM (state-space model) to obtain a sequence of size (B, N, C). out The feature sequence Z6 (sixth global sub-feature sequence) is obtained by normalizing Z6 using LayerNorm; Z7 (seventh global sub-feature sequence) is obtained by multiplying Z7 and Z2 position by position to obtain a sequence of size (B, N, C). out The feature sequence Z8 (eighth global sub-feature sequence) is obtained by linear transformation of Z8, resulting in a sequence of size (B, N, C). out The feature sequence Z9 (i.e., the third global feature sequence) is the final output of the DenseVSS module.

[0102] The linear transformation and subsequent operations of the second global feature sequence are performed in the third branch and the fourth branch respectively by using different processing methods, which can mine and convert feature information from different angles and levels, enrich the representation form of the feature, and help to capture more comprehensive and detailed feature patterns. The first global sub-feature sequence in the third branch and the fourth global sub-feature sequence in the fourth branch are processed by using an activation function, a nonlinear factor is introduced, the model can learn more complex nonlinear relationships in the data, the modeling ability of the model for complex features is improved, and thus the model can better adapt to various complex semantic segmentation tasks. In the fourth branch, the third global sub-feature sequence is processed by using a depth separable convolution after being modified in shape and the storage order, compared with a traditional convolution, the depth separable convolution greatly reduces the calculation amount and the parameter amount while ensuring a certain feature extraction capability, improves the calculation efficiency, and helps to prevent overfitting, so that the model can still maintain good performance in the case of limited resources. The fifth global sub-feature sequence is processed by using a state space model in the fourth branch, which can better capture dynamic changes and long-term dependencies in sequence data, is very effective for processing global feature sequences with time sequence or spatial sequence characteristics, and helps to improve the understanding and segmentation ability of the model for complex semantic structures. The sixth global sub-feature sequence is normalized, which can make the distribution of the data more stable, speed up the convergence speed of the model, avoid problems such as gradient disappearance or explosion in the training process, and improve the training stability and generalization ability of the model. The second global sub-feature sequence obtained by the third branch and the seventh global sub-feature sequence obtained by the fourth branch after a series of processing are multiplied position by position, and the effective fusion of features in different processing paths is realized. This fusion method can combine the advantages of different features, further improve the comprehensive expression ability of the features, and make the third global feature sequence obtained finally contain more rich and discriminative information.

[0103] Optionally, the fusing the first local feature and the first global feature to obtain a first fused feature comprises:

[0104] In the first target layer of the encoder, a plurality of groups of first intermediate features of a same level encoder in a first size are spliced along a channel dimension to form a spliced feature;

[0105] The spliced feature is linearly transformed to reduce the number of output channels to form a dense connection path feature in a second size;

[0106] The input channel number is linearly mapped by a channel adaptation layer, the spatial resolution of the output feature is kept unchanged, the dense connection path feature is fused with the first local feature to form the first fused feature.

[0107] In the first target layer of the encoder, a plurality of groups of first intermediate features of the first size obtained by the same level encoder are spliced along a channel dimension to form spliced features.

[0108] The encoder of the same level can generate features of different receptive fields or modalities (such as features of different convolution kernel sizes or different modal input), and through channel splicing (instead of spatial splicing), a plurality of groups of features are stacked in the channel dimension, thereby retaining the spatial resolution of each group of features while expanding the information diversity in the channel dimension. Since the splicing operation requires the spatial size (first size) of the features to be the same, it is necessary to ensure that the plurality of groups of input features have been aligned to the same size through upsampling / downsampling or shared convolutional layers. For example, in medical images, the encoder of the same level can extract local texture features (small receptive field) and global morphological features (large receptive field) of a CT image, and after splicing, a feature tensor containing multi-scale information is formed. Linear transformation (such as 1x1 convolution) is performed on the spliced features to reduce the number of output channels to form dense connection path features of the second size. The number of channels of the spliced features can be too large (such as the number of channels being doubled due to the splicing of a plurality of groups of features), and 1x1 convolution (linear transformation) is used to reduce the number of channels to a reasonable range, thereby reducing the computational complexity and avoiding the curse of dimensionality. The 1x1 convolution not only reduces the dimension, but also weights and fuses the channel features through learning weights to extract cross-channel correlation information. The second size: the linear transformation can change the spatial size of the features (such as through step convolution), and the second size refers to the features after the number of channels is reduced, and the spatial size can remain unchanged. For example: if the number of channels of the spliced features is 256, it is reduced to 64 through 1x1 convolution to form more compact dense connection path features. The input channel number is linearly mapped through the channel adaptation layer, and the spatial resolution of the output features remains unchanged, the dense connection path features are fused with the first local features to form the first fused features. The number of channels of the dense connection path features and the first local features is adjusted to match the dimensions, which facilitates subsequent fusion. It is usually 1x1 convolution, which linearly maps the number of channels of the input features (such as mapping 64 channels to 32 channels) while keeping the spatial resolution unchanged (through padding or non-step convolution). It can be element-wise addition (Add) or splicing (Concat) and then fused through convolution. But usually the spatial resolution needs to be kept unchanged, so the number of channels may need to be adjusted through 1x1 convolution after splicing. The dense connection path features contain multi-scale global information, and the first local features (such as high-resolution detail features) are fused with them after channel adaptation, realizing the complementarity of local details and global semantics. For example, if the dense connection path features are 64 channels and the spatial size is HxW, and the first local features are 32 channels and HxW, the channel adaptation layer maps the first local features to 64 channels, and then element-wise addition is performed with the dense connection path features to form 64-channel fused features.

[0109] The input channel number of the densely connected path feature is linearly mapped by a channel adaptation layer (such as a 1x1 convolution) to match the channel number of the first local feature. For example, if the densely connected path feature has 64 channels and the local feature has 32 channels, the channel adaptation layer can expand the channel number of the local feature to 64. The channel adaptation layer adjusts the stride and padding of the convolution kernel to ensure that the spatial resolution of the output feature is consistent with the input feature. For example, in a CT image, this process can avoid the loss of spatial information during feature fusion, ensuring the accuracy of the density map or segmentation result. The densely connected path feature after channel adaptation is fused (such as element-wise addition or concatenation) with the first local feature to form the first fused feature. This process combines global context information (densely connected path feature) and local detail information (first local feature), improving the expression ability of the feature. Traditional methods (such as direct concatenation or weighted sum) usually cannot dynamically adjust the channel dimension, while the channel adaptation layer achieves flexible adaptation of the channel dimension through linear mapping, avoiding the limitations of manually designed fusion strategies. Traditional methods may ignore the long-range dependencies between features, while the densely connected path feature enhances the context relevance of the feature by retaining the core part of the multi-scale information. Traditional methods may introduce loss of spatial resolution during the fusion process, while the channel adaptation layer ensures the integrity of spatial information by designing the parameters of the convolution kernel.

[0110] Optionally, the method further comprises:

[0111] Collecting historical CT images and electronic density maps corresponding to the historical CT images to construct a data set, and training an original electronic density generation model using the data set;

[0112] Constructing a loss function by the average absolute error of the difference between the estimated value and the true value in the region containing the human structure, calculating the value of the loss function, and when the value of the loss function is less than a preset threshold, taking the trained original electronic density generation model as the electronic density generation model,

[0113] The calculation of the value of the loss function comprises:

[0114] The value of the loss function is calculated by the following formula:

[0115] Loss = MAE (Ω Normal ) + a MAE (Ω Air ) + b MAE (Ω Bone );

[0116] Where Loss represents the loss function, Ω Air represents the set of all pixel points in the cavity region, Ω Bone represents the set of all pixel points in the region with HU (Hounsfield Unit) value higher than a preset threshold, and ΩNormal MAE(Ωc) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE(Ωc) = 1 |Ωc| åx∈Ωc |xc- x|, where xc represents the estimated value of the pixel point x in the cavity region, x represents the true value of the pixel point x in the cavity region, and |Ωc| represents the number of pixel points in the cavity region. Air MAE(Ωc) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE(Ωc) = 1 |Ωc| åx∈Ωc |xc- x|, where xc represents the estimated value of the pixel point x in the cavity region, x represents the true value of the pixel point x in the cavity region, and |Ωc| represents the number of pixel points in the cavity region. Bone MAE(Ωc) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE(Ωc) = 1 |Ωc| åx∈Ωc |xc- x|, where xc represents the estimated value of the pixel point x in the cavity region, x represents the true value of the pixel point x in the cavity region, and |Ωc| represents the number of pixel points in the cavity region. Normal MAE(Ωc) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE(Ωc) = 1 |Ωc| åx∈Ωc |xc- x|, where xc represents the estimated value of the pixel point x in the cavity region, x represents the true value of the pixel point x in the cavity region, and |Ωc| represents the number of pixel points in the cavity region.

[0117] A large number of historical CT scan images are collected from a clinical database, which usually contain samples of different scan parameters (such as tube voltage, slice thickness), different anatomical structures (such as head, chest, abdomen), and different pathological states (such as normal tissue, tumor, inflammation). Electron density maps corresponding to the CT images are obtained (usually generated by physical measurement or high-precision simulation) as labels for supervised learning. The electron density map reflects the attenuation ability of tissue to X-rays and is a key input for tasks such as radiotherapy dose calculation and CT value calibration. The pixel values of the CT image (usually in Hounsfield units, HU) are normalized to a specific range (such as [-1, 1] or [0, 1]) to eliminate the influence of different scanning devices or parameters. The CT image and electron density map are resampled to a uniform resolution (such as 1x1x1mm³) to ensure the consistency of the input data. The images are cropped according to the region of interest (such as the region containing human structures) or padded for small-sized images to adapt to the model input size. The dataset is divided into training, validation, and test sets (such as 70%, 20%, 10%) to ensure that the model is evaluated on unseen data. A deep learning model (such as U-Net, 3D CNN, Transformer, etc.) is selected as the original electron density generation model, which can learn complex non-linear mapping relationships from CT images. The CT images of the training set are input into the model to generate predicted electron density maps. The error between the predicted value and the true value is calculated using the loss function, and the model parameters are updated through backpropagation. The learning rate is adjusted using an optimizer (such as Adam, SGD), and the learning rate is dynamically adjusted using a learning rate scheduler (such as ReduceLROnPlateau) to accelerate convergence and avoid overfitting. Data augmentation (such as rotation, flipping, elastic deformation) is used to increase the diversity of training samples and improve the generalization ability of the model. Regularization techniques (such as Dropout, L2 regularization) are introduced to prevent model overfitting. The loss function is defined as the average of the absolute values of the difference between the predicted electron density values and the true values in the region containing human structures. Only the error in the region containing human structures is calculated to avoid interference from the background region (such as air, scanning bed) on the loss function. For example, the human structure region can be defined by threshold segmentation or mask (Mask). During training, the loss function value on the validation set is calculated regularly to monitor the convergence of the model. When the loss function value on the validation set is less than a pre-set threshold (such as 0.01 or 0.001), the model is considered to have converged and training is stopped. The trained original electron density generation model is saved as the final model for subsequent inference tasks.

[0118] The human structure is divided into three types of regions (normal tissue, cavity, and high-density bone tissue), and independent loss terms are designed for different regions according to their characteristics. The loss contribution of different regions is dynamically adjusted through weight parameters (α, β) to avoid the error of high-density regions (such as bone) or low-density regions (such as air) dominating the overall optimization direction. The Normal region refers to the region in the human structure that is not a cavity and not high-density (HU ≤ threshold) (such as muscle, fat, organs, etc.), that is, the soft tissue region in the image that needs to be accurately segmented / reconstructed. The Air region refers to the cavity region in the image (such as lung air, intestinal gas, etc., with very low HU value), which is usually the background or low-density region in the image and is prone to misclassification. The Bone region refers to the region with HU value higher than the preset threshold (such as bone, metal implant, etc., high-density structure), that is, the high-contrast region in the image that needs to be reconstructed with high precision. Mean absolute error (MAE) is not sensitive to outliers (such as noise or artifacts), and can stably optimize the model. For example, in the electron density map, if the true value of some pixel points is abnormal due to measurement error, MAE will reduce the influence of these abnormal values on the loss function. MAE directly reflects the average deviation between the predicted value and the true value, which is consistent with the physical meaning of the electron density map. For example, in radiotherapy dose calculation, a small deviation in electron density may cause significant changes in dose distribution, and MAE can quantify this deviation. The calculation process of the loss function is transparent, which is convenient for debugging and optimization.

[0119] The training experiment of converting the CT image to the electron density map was run on two NVIDIA RTXA6000 48G graphics cards in an Ubuntu 18.04 operating system, using the deep learning framework pytorch2.1 to train the electron density generation model, and the programming language was python 3.9. Figure 7 is a comparison diagram of the real electron density map and the electron density map generated by the embodiments of the present application, as Figure 7 shown, the prediction result is very close to the real image, and the HU value prediction error is within 0.5%.

[0120] The electron density map generated by the electron density generation model and the electron density map acquired by spectral CT were imported into the Eclipse treatment planning system V15.6 (Varian Medical Systems). The treatment plan was designed and optimized using Eclipse TPS V15.6, and verified by Halcyon and Truebeam accelerators. Anisotropic analysis algorithm (AAA) and phonon optimizer (PO) were used for dose calculation and plan optimization. Figure 8 is a comparison diagram of the dose prediction results of the real electron density map and the electron density map generated by the embodiments of the present application, as Figure 8As shown, the error of the generated electron density map on the dose distribution is less than 0.2% compared with the real electron density map, which has high prediction accuracy and stability.

[0121] The embodiment also discloses an electron density map generation system, Figure 9 is a module schematic diagram of an electron density map generation system disclosed by the embodiment of the application, as shown in Figure 9 As shown, the system comprises an extraction module 901, a fusion module 902, an execution module 903 and a generation module 904, wherein:

[0122] The extraction module 901 is configured to extract initial features of a received CT image in response to the CT image, and input the initial features as input features of a first layer of an encoder into the encoder.

[0123] The fusion module 902 is configured to extract first local features of input features of a first target layer in the encoder through a residual network path, extract first global features of the input features of the first target layer through a dense connection path, fuse the first local features and the first global features to obtain first fusion features, down-sample the first fusion features to obtain first sampling features, and input the first sampling features as input features of a next layer until the next layer is a deepest layer, and the first target layer is any layer of the encoder except the deepest layer.

[0124] The execution module 903 is configured to extract second local features of input features of the deepest layer through a residual network path, extract second global features of the input features of the deepest layer through a dense connection path, fuse the second local features and the second global features to obtain second fusion features, and input the second fusion features as input features of a first layer of a decoder into the decoder.

[0125] The generation module 904 is configured to up-sample input features of a second target layer of the decoder to obtain second sampling features, superimpose the second sampling features and first local features of a corresponding layer of the encoder to obtain second fusion features, input the second fusion features as input features of a next layer until the next layer is an output layer, and perform convolution processing on the second fusion features to obtain an electron density map, and the second target layer is any layer of the decoder.

[0126] Optionally, the fusion module 902 is configured to:

[0127] adjust a storage order of the input features and change arrangement to obtain an initial local feature sequence;

[0128] normalizing the initial local feature sequence to obtain a first local feature sequence, and obtaining a second local feature sequence by applying an activation function, a preset sequence modeling architecture (Mamba module), and normalization to the first local feature sequence;

[0129] stacking the initial local feature sequence and the second local feature sequence to obtain a third local feature sequence, and obtaining the first local feature by rearranging and changing the storage order of the third local feature sequence.

[0130] Optionally, the fusion module 902 is configured to:

[0131] normalizing an input local feature sequence of the preset sequence modeling architecture to obtain a first local sub-feature sequence, and processing the first local sub-feature sequence by a first preset semantic segmentation module to obtain a second local sub-feature sequence;

[0132] performing scaling transformation on the input local feature sequence to obtain a third local sub-feature sequence, and stacking the third local sub-feature sequence and the second local sub-feature sequence to obtain a fourth local sub-feature sequence;

[0133] performing normalization and linear transformation on the fourth local sub-feature sequence to obtain an output local feature sequence.

[0134] Optionally, the fusion module 902 is configured to:

[0135] adjusting the storage order and changing the arrangement of the input feature to obtain a first global feature sequence;

[0136] In the first branch, normalizing the first global feature sequence to obtain a second global feature sequence, and processing the second global feature sequence by a second preset semantic segmentation module to obtain a third global feature sequence;

[0137] In the second branch, performing linear transformation on the first global feature sequence to obtain a fourth global feature sequence, and performing scaling transformation on the fourth global feature sequence;

[0138] stacking the third global feature sequence and the fourth global feature sequence after scaling transformation to obtain a fifth global feature sequence, and obtaining the first global feature by normalizing, linearly transforming, and rearranging the fifth global feature sequence.

[0139] Optionally, the fusion module 902 is configured to:

[0140] In a third branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a first global sub-feature sequence, and the first global sub-feature sequence is processed by an activation function to obtain a second global sub-feature sequence;

[0141] In a fourth branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a third global sub-feature sequence, and the third global sub-feature sequence is processed by a modified shape, a transformed storage order and a depth separable convolution to obtain a fourth global sub-feature sequence, the fourth global sub-feature sequence is processed by an activation function to obtain a fifth global sub-feature sequence, the fifth global sub-feature sequence is processed by a state space model to obtain a sixth global sub-feature sequence, and the sixth global sub-feature sequence is normalized to obtain a seventh global sub-feature sequence;

[0142] The second global sub-feature sequence and the seventh global sub-feature sequence are multiplied position by position to obtain an eighth global sub-feature sequence;

[0143] The eighth global sub-feature sequence is linearly transformed to obtain a third global feature sequence.

[0144] Optionally, the fusion module 902 is configured to:

[0145] In the first target layer of the encoder, a plurality of groups of first intermediate features of the first size obtained by the same level encoder are spliced along the channel dimension to form spliced features;

[0146] The spliced features are linearly transformed to reduce the output channel number to form dense connection path features of a second size;

[0147] The input channel number is linearly mapped by a channel adaptation layer, the spatial resolution of the output features is kept unchanged, the dense connection path features are fused with the first local features to form the first fusion features.

[0148] Optionally, the system further includes a training module configured to:

[0149] A data set is constructed by collecting historical CT images and electronic density maps corresponding to the historical CT images, and the original electronic density generation model is trained using the data set;

[0150] A loss function is constructed by the average absolute error of the difference between the estimated value and the true value in the region containing the human structure, the value of the loss function is calculated, and when the value of the loss function is less than a preset threshold, the trained original electronic density generation model is used as the electronic density generation model.

[0151] The calculation of the value of the loss function includes:

[0152] The value of the loss function is calculated by the following formula:

[0153] Loss = MAE (Ω Normal ) + a MAE (Ω Air ) + b MAE (Ω Bone )

[0154] Wherein, Loss represents the loss function, Ω Air represents the set of all pixel points in the cavity region, Ω Bone represents the set of all pixel points in the region with HU value higher than the preset threshold, Ω Normal represents the set of all pixel points in the region of the human structure except the cavity region and the region with HU value higher than the preset threshold, MAE (Ω Air ) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE (Ω Bone ) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the region with HU value higher than the preset threshold, MAE (Ω Normal ) represents the average absolute error between the estimated value and the true value of the set of all pixel points in the region of the human structure except the cavity region and the region with HU value higher than the preset threshold, a and b are non-negative real numbers.

[0155] It should be noted that the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0156] The embodiment also discloses an electronic device, referring to Figure 10 The electronic device can include at least one processor 1001, at least one communication bus 1002, a user interface 1003, a network interface 1004, and at least one memory 1005.

[0157] The communication bus 1002 is used to realize the connection and communication between the components.

[0158] The user interface 1003 can include a display screen (Display) and a camera (Camera), and the optional user interface 1003 can also include a standard wired interface and a wireless interface.

[0159] The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0160] The processor 1001 can include one or more processing cores. The processor 1001 connects various parts within the server through various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005.

[0161] The memory 1005 can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc. The data storage area can store data related to the various method embodiments described above, etc. The memory 1005 can alternatively be at least one storage device located away from the aforementioned processor 1001. As shown, the memory 1005, as a computer storage medium, can include an operating system, a network communication module, a user interface module, and an application program of an electronic density map generation method. Figure 10

[0162] In the electronic device shown in Figure 10 In the electronic device shown in, the user interface 1003 is mainly used to provide an interface for user input and obtain data input by the user; and the processor 1001 can be used to call an application program of an electronic density map generation method stored in the memory 1005, and when executed by one or more processors 1001, make the electronic device execute the method of one or more of the above embodiments.​

Claims

1. A method of generating an electron density map, characterized by, The application is applied to an electron density generation model comprising an encoder and a decoder, and the method comprises: In response to the received CT image, initial features of the CT image are extracted and input into the encoder as input features of the first layer of the encoder; In the first target layer of the encoder, first local features of the input features of the first target layer are extracted through a residual network path, first global features of the input features of the first target layer are extracted through a dense connection path, the first local features and the first global features are fused to obtain first fusion features, the first fusion features are down-sampled to obtain first sampling features, and the first sampling features are input into the next layer as input features until the next layer is the deepest layer, and the first target layer is any layer in the encoder except the deepest layer; In the deepest layer, second local features of the input features of the deepest layer are extracted through a residual network path, second global features of the input features of the deepest layer are extracted through a dense connection path, the second local features and the second global features are fused to obtain second fusion features, and the second fusion features are input into the decoder as input features of the first layer of the decoder; In the second target layer of the decoder, the input features of the second target layer are up-sampled to obtain second sampling features, the second sampling features are superimposed with the first local features of the corresponding layer of the encoder to obtain second fusion features, the second fusion features are input into the next layer as input features until the next layer is the output layer, and the second fusion features are convoluted to obtain an electron density map, and the second target layer is any layer in the decoder.

2. The method of generating an electron density map according to claim 1, wherein, The first local features of the input features of the first target layer are extracted through a residual network path, which comprises: The input features are adjusted in storage order and rearranged to obtain an initial local feature sequence; The initial local feature sequence is normalized to obtain a first local feature sequence, and the first local feature sequence is processed through an activation function, a preset sequence modeling architecture and normalization to obtain a second local feature sequence; The initial local feature sequence and the second local feature sequence are superimposed to obtain a third local feature sequence, and the third local feature sequence is rearranged and the storage order is changed to obtain the first local features.

3. The method of generating an electron density map according to claim 2, wherein, The first local feature sequence is processed through an activation function, a preset sequence modeling architecture and normalization to obtain a second local feature sequence, which comprises: The input local feature sequence of the preset sequence modeling architecture is normalized to obtain a first local sub-feature sequence, and the first local sub-feature sequence is processed through a first preset semantic segmentation module to obtain a second local sub-feature sequence; The input local feature sequence is scaled to obtain a third local sub-feature sequence, and the third local sub-feature sequence and the second local sub-feature sequence are superimposed to obtain a fourth local sub-feature sequence; The fourth local sub-feature sequence is normalized and linearly transformed to obtain an output local feature sequence.

4. The method of generating an electron density map of claim 1, wherein, The first global feature of the input feature of the first target layer extracted through the dense connection path comprises: The input feature is stored in sequence and changed in arrangement to obtain a first global feature sequence; In the first branch, the first global feature sequence is normalized to obtain a second global feature sequence, and the second global feature sequence is processed through a second preset semantic segmentation module to obtain a third global feature sequence; In the second branch, the first global feature sequence is linearly transformed to obtain a fourth global feature sequence, and the fourth global feature sequence is proportionally transformed; The third global feature sequence and the fourth global feature sequence after proportion transformation are superimposed to obtain a fifth global feature sequence, and the fifth global feature sequence is normalized, linearly changed and rearranged to obtain the first global feature.

5. The method of generating an electron density map according to claim 4, wherein, The processing of the second global feature sequence through the second preset semantic segmentation module to obtain the third global feature sequence comprises: In the third branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a first global sub-feature sequence, and the first global sub-feature sequence is processed through an activation function to obtain a second global sub-feature sequence; In the fourth branch of the second preset semantic segmentation module, the second global feature sequence is linearly transformed to obtain a third global sub-feature sequence, and the third global sub-feature sequence is processed through modified shape, transformed storage sequence and depth separable convolution to obtain a fourth global sub-feature sequence, and the fourth global sub-feature sequence is processed through an activation function to obtain a fifth global sub-feature sequence, and the fifth global sub-feature sequence is processed through a state space model to obtain a sixth global sub-feature sequence, and the sixth global sub-feature sequence is normalized to obtain a seventh global sub-feature sequence; The second global sub-feature sequence and the seventh global sub-feature sequence are multiplied by position to obtain an eighth global sub-feature sequence; The eighth global sub-feature sequence is linearly transformed to obtain the third global feature sequence.

6. The method of generating an electron density map of claim 1, wherein, The fusion of the first local feature and the first global feature to obtain the first fusion feature comprises: In the first target layer of the encoder, a plurality of groups of first intermediate features of the first size obtained by the same level encoder are spliced along the channel dimension to form a spliced feature; The spliced feature is linearly transformed to reduce the output channel number to form a dense connection path feature of a second size; The input channel number is linearly mapped through a channel adaptation layer, and the spatial resolution of the output feature is kept unchanged, the dense connection path feature is fused with the first local feature to form the first fusion feature.

7. The method of generating an electron density map of claim 1, wherein, The method further comprises: Collecting historical CT images and electronic density map construction data corresponding to the historical CT images to construct a data set, and training an original electronic density generation model using the data set; constructing a loss function by an average absolute error of a difference between an estimated value and a true value in a region including a human structure, calculating a value of the loss function, and taking the trained original electron density generation model as the electron density generation model when the value of the loss function is less than a preset threshold, the calculating the value of the loss function comprises: calculating the value of the loss function by the following formula: Loss = MAE(Ω Normal ) + a · MAE(Ω Air ) + β · MAE(Ω Bone ) where Loss denotes a loss function, Ω Air denotes a set of all pixel points in the cavity region, Ω Bone denotes a set of all pixel points in the region with HU values higher than the preset threshold, Ω Normal denotes a set of all pixel points in the region of the human structure other than the cavity region and the region with HU values higher than the preset threshold, MAE(Ω Air ) denotes the mean absolute error between the estimated value and the true value of the set of all pixel points in the cavity region, MAE(Ω Bone ) denotes the mean absolute error between the estimated value and the true value of the set of all pixel points in the region with HU values higher than the preset threshold, MAE(Ω Normal ) denotes the mean absolute error between the estimated value and the true value of the set of all pixel points in the region of the human structure other than the cavity region and the region with HU values higher than the preset threshold, and α and β are non-negative real numbers.

8. A system for generating an electron density map, characterized by comprising an extraction module, a fusion module, an execution module, and a generation module, wherein: the extraction module is configured to extract initial features of the CT image in response to the received CT image, and input the initial features as input features of a first layer of an encoder into the encoder; the fusion module is configured to extract first local features of input features of the first target layer through a residual network path, extract first global features of the input features of the first target layer through a dense connection path, fuse the first local features and the first global features to obtain first fusion features, down-sample the first fusion features to obtain first sampling features, and input the first sampling features as input features of a next layer until the next layer is a deepest layer, in the first target layer which is any layer of the encoder except the deepest layer; the execution module is configured to extract second local features of input features of the deepest layer through a residual network path, extract second global features of the input features of the deepest layer through a dense connection path, fuse the second local features and the second global features to obtain second fusion features, and input the second fusion features as input features of a first layer of a decoder into the decoder in the deepest layer; the generation module is configured to up-sample input features of a second target layer of the decoder to obtain second sampling features, superimpose the second sampling features and first local features of a corresponding layer of the encoder to obtain second fusion features, input the second fusion features as input features of a next layer until the next layer is an output layer, and perform convolution processing on the second fusion features to obtain an electron density map, in the second target layer which is any layer of the decoder.

9. An electronic device, comprising: comprising a processor, a memory, a user interface, and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed, perform the method of any one of claims 1-7.

Images