Photon counting ct sparse angular image reconstruction method and system based on deep learning

Abstract

The application relates to a kind of deep learning-based photon counting CT sparse angle image reconstruction method and system, the system includes data collection module, residual U-shaped image recovery network model and image reconstruction module;Residual U-shaped image recovery network model includes encoding layer, bottleneck layer and decoding layer;Encoding layer is used to carry out double-path spectral modulation to input feature under multiple scales;Bottleneck layer is used to carry out polar coordinate spectral calibration and path state mixing to the feature after modulation;Decoding layer is used to carry out skip connection decoding to the feature after mixing, and carries out double-path spectral modulation under multiple scales, obtains convolution prediction residual image;Image reconstruction module is used to add convolution prediction residual image and input image to obtain reconstruction image.The application can suppress sparse angle stripe artifact while retaining edge, texture and gray continuity in photon counting Micro-CT image by multi-scale fusion of shallow layer detail feature and deep layer global feature.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and in particular to a method and system for reconstructing sparse angle images of CT based on photon counting using deep learning. Background Technology

[0002] Photon counting CT is an emerging medical imaging counting technique in the field of medical diagnostics. Because it can provide multi-spectral images and material decomposition, achieving higher image resolution and reducing radiation dose, it has significant clinical value for disease diagnosis and has therefore attracted increasing attention. To shorten scan time, improve scanning efficiency, and reduce radiation dose, a sparse angle sampling strategy is often employed, i.e., reducing the number of projection angles. This undersampling method inevitably leads to the loss of structural information and the generation of severe fringe artifacts during reconstruction, affecting subsequent image observation, structural measurement, and quantitative analysis. Summary of the Invention

[0003] The purpose of this invention is to provide a photon counting CT sparse angle image reconstruction method and system based on deep learning, so as to improve the problem of loss of structural information and severe stripe artifacts caused by undersampling during the reconstruction process.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A deep learning-based photon counting CT sparse angle image reconstruction system includes a data collection module, a residual U-shaped image restoration network model, and an image reconstruction module; the residual U-shaped image restoration network model includes an encoding layer, a bottleneck layer, and a decoding layer;

[0006] The data collection module is used to acquire photon-counted CT sparse angle images and extract initial features from them;

[0007] The coding layer is used to perform dual-path spectral modulation on the input features at multiple scales;

[0008] The bottleneck layer is used to perform polar coordinate spectrum calibration and path state mixing on the modulated features.

[0009] The decoding layer is used to perform skip-connected decoding on the mixed features and perform dual-path spectral modulation at multiple scales to obtain the convolutional prediction residual image.

[0010] The image reconstruction module is used to add the convolution prediction residual image and the input image to obtain the reconstructed image.

[0011] In a preferred embodiment, the coding layer includes encoders of three cascaded scales, and the decoding layer includes decoders of three cascaded scales. Each encoder and decoder is equipped with a dual-path spectrum modulation module for performing dual-path spectrum modulation on the input features. The decoder upsamples the features and concatenates them with the output features of the corresponding scale level encoder, reduces the dimensionality, and then performs dual-path spectrum modulation.

[0012] In a preferred embodiment, the bottleneck layer comprises four cascaded bottleneck units, wherein the first and third bottleneck units include a polar coordinate spectrum calibration module and a path state mixing module, and the second and fourth bottleneck units contain only a path state mixing module; the polar coordinate spectrum calibration module is used to perform polar coordinate spectrum calibration, and the path state mixing module is used to perform path state mixing on the calibrated polar coordinate spectrum.

[0013] A deep learning-based photon counting CT sparse angle image reconstruction method includes the following steps:

[0014] S10: Obtain photon-counted CT sparse angle images and extract initial features from them;

[0015] S20, performs dual-path spectral modulation on the initial features at multiple scales;

[0016] S30 performs polar coordinate spectrum calibration and path state mixing on the modulated features;

[0017] S40 performs skip-connected decoding on the mixed features and performs dual-path spectral modulation at multiple scales to obtain the convolutional prediction residual image;

[0018] S50, the convolutional prediction residual image is added to the input image to obtain the reconstructed image.

[0019] The dual-path spectrum modulation processing includes the following operations in sequence:

[0020] The input feature X is first normalized by channels, and then 1×1 convolution and 3×3 depth convolution are performed.

[0021] Features after depthwise convolution Main path features and gated path features ;

[0022] The gated path provides hints and connections to the main path, making the main path characteristic of... , It is the first scaling factor;

[0023] The main path undergoes adaptive spectral convolution to output features. , Dynamic convolution kernel;

[0024] The main path provides a hint connection to the gated path, causing the gated path characteristics to change. β is the second scaling factor;

[0025] The modified gated path features are convolved and merged by horizontal convolution, vertical convolution and local convolution, and then generated by convolution and activation function to produce gated weights G.

[0026] The main path output features M and the gate weights G of the gated path output are fused by element-wise multiplication, and then output after a 1x1 convolution. , ;

[0027] After introducing the compensation term S and gain compensation, the final output is... , , where g is the channel scaling parameter.

[0028] The polar coordinate spectrum calibration process includes the following operations in sequence:

[0029] First, channel normalization is performed, then reflection filling is performed, then two-dimensional Fourier transform is performed, and finally spectrum centering is performed.

[0030] Convert the spectrogram into polar coordinate spectral feature representation units;

[0031] The polar coordinate spectral feature representation unit is divided into an angle branch and a radius branch. The angle branch is processed by a 1×3 depth convolution, and the radius branch is processed by a 3×1 depth convolution. The angle branch and the radius branch after depth convolution are then fused based on weights.

[0032] The fused polar coordinate spectral feature representation units are processed by 1×1 convolution and then divided into amplitude correction feature representation units and phase correction feature representation units in the channel dimension, and then converted into a spectrum.

[0033] Calibrate the amplitude and phase;

[0034] After calibration, frequency domain reconstruction is performed to restore the Fourier arrangement, and then the spatial domain is restored, with the reflection filling area being cropped.

[0035] After a 3×3 depthwise convolution, the residual is scaled and added to the input to obtain the final output.

[0036] The processing of the path state mixing includes the following operations in sequence:

[0037] First, channel normalization is performed, then the channels are expanded to 2C, and then a 3×3 depthwise convolution is performed.

[0038] The feature channels are divided into Signal branches and Route branches. The Signal branch is expanded into a one-dimensional sequence according to four paths: row serpentine, column serpentine, reverse row, and reverse column. Then, the one-dimensional sequences obtained from each path are stacked.

[0039] Perform a 1x1 depthwise convolution on the stacked sequence and generate the parameters required for the state space model through linear projection;

[0040] Each path is selectively scanned, and the two-dimensional spatial positions obtained from the four path scans are stitched together and then fused using a 1×1 convolution.

[0041] After the route branch is activated by the SiLU function, it is multiplied element-wise with the fusion result, then passed through a 1×1 depthwise convolution, and finally scaled by the residual and added to the input to obtain the final output.

[0042] A computer program product includes computer-readable instructions that, when executed by a processor, implement the steps in the deep learning-based photon counting CT sparse angle image reconstruction method of the present invention.

[0043] A computer-readable storage medium including computer-readable instructions that, when executed by a processor, implement the steps in the deep learning-based photon counting CT sparse angle image reconstruction method of the present invention.

[0044] An electronic device includes: a memory storing program instructions; and a processor connected to the memory, executing the program instructions in the memory to implement the steps in the deep learning-based photon counting CT sparse angle image reconstruction method of the present invention.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. This invention uses a residual U-shaped network model to fuse shallow detailed features with deep global features at multiple scales, which can preserve the edge, texture and grayscale continuity in photon-counted Micro-CT images while suppressing sparse angular stripe artifacts.

[0047] 2. This invention combines the adaptive spectral convolution of the main feature path with the directional structure perception of the gated path through a dual-path spectral modulation module, enabling the network to dynamically adjust the feature enhancement method according to the spectral degradation and stripe direction in different image samples, thereby reducing the problem of insufficient adaptability of fixed convolution kernels to complex artifacts.

[0048] 3. This invention generates directional gating weights through horizontal, vertical and local gated branches, and selectively modulates the spectral enhancement features of the main path, which helps to distinguish between real tissue edges, local textures and directional stripe artifacts generated by sparse angle reconstruction, thereby reducing the risk of over-smoothing or artifact amplification.

[0049] 4. This invention uses a polar coordinate spectrum calibration module to convert deep features to the frequency domain and models the spectrum information according to the frequency radius and frequency angle. It can process low-frequency structure, mid-frequency texture, high-frequency edge and directional stripe distribution respectively, so that the spectrum energy and structural position information can be more reasonably calibrated.

[0050] 5. This invention uses a constrained residual correction method for spectral amplitude and phase to correct spectral energy anomalies while constraining the amplitude of phase perturbation, which helps to maintain the stability of image structure position and avoids structural shift or detail distortion caused by direct frequency domain enhancement.

[0051] 6. This invention uses a path state mixing module to expand two-dimensional features along four types of paths: row, column, reverse row, and reverse column, and then performs state space scanning. This enables the establishment of long-distance dependencies across regions, improving the network's ability to distinguish large-scale continuous stripe artifacts and long-range organizational structures. Attached Figure Description

[0052] Figure 1 This is a structural diagram of a deep learning-based photon counting CT sparse angle image reconstruction system provided for an embodiment.

[0053] Figure 2 This is a schematic diagram of the dual-path spectrum modulation (DRSB) module structure.

[0054] Figure 3 A schematic diagram of the PSC module structure for polar coordinate spectrum calibration.

[0055] Figure 4 A schematic diagram of the path state hybrid PSM module structure.

[0056] Figure 5 Image of a walnut reconstructed using Micro-CT at a sparse angle of 1 / 16.

[0057] Figure 6 Image of a walnut reconstructed using Micro-CT at a sparse angle of 1 / 12.

[0058] Figure 7 Image reconstruction data of a mouse from a Micro-CT image with a sparse angle of 1 / 12.

[0059] Figure 8 Image reconstruction data of a mouse from a Micro-CT image with a sparse angle of 1 / 16th.

[0060] Figure 9 The flowchart illustrates a deep learning-based photon counting CT sparse angle image reconstruction method provided for an embodiment.

[0061] Figure 10 This is a block diagram of the electronic device provided in the embodiment. Detailed Implementation

[0062] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0063] like Figure 1 As shown in this embodiment, a deep learning-based photon-counting CT sparse angle image reconstruction system is provided, including a data collection module, a residual U-shaped image restoration network model, and an image reconstruction module; the data collection module obtains photon-counting CT sparse angle images and extracts initial features from them. , X is the input image (i.e., the obtained CT sparse angle image), and X0 represents the initial features; the residual U-shaped image restoration network model outputs a convolutional prediction residual image based on the initial features; the image reconstruction module adds the convolutional prediction residual image and the input image to obtain the reconstructed image.

[0064] The residual U-shaped image restoration network model includes an encoding layer, a bottleneck layer, and a decoding layer. The encoding layer consists of three encoders at different scales (the number of scales is determined by the input image size, the spatial distribution of sparse angular artifacts, and computational resources). Figure 1 The encoder consists of Encoder1, Encoder2, and Encoder3, each stacked with a dual-route spectral block, and downsampled using Conv+PixelUnshuffle. The bottleneck layer contains multiple bottleneck units (SpectralPathBottlenecks), where polar coordinate spectral calibration is performed at designated bottleneck units, followed by path-state mixing. The decoding layer comprises three scale-level decoders (…). Figure 1 The decoder consists of three parts: Decoder1, Decoder2, and Decoder3. Each decoder is stacked with a dual-path spectrum modulation module. The decoder is upsampled by Conv+PixelShuffle and concatenated with the output features of the corresponding scale level encoder. Then, it is dimensionality reduced by 1×1 convolution before entering the dual-path spectrum modulation module.

[0065] In this embodiment, the bottleneck layer contains four cascaded bottleneck units. The first and third bottleneck units include a polar coordinate spectrum calibration module, a path state mixing module, and a local residual block. The second and fourth bottleneck units do not have a polar coordinate spectrum calibration module; they only contain a path state mixing module and a local residual block. The purpose of this arrangement is to perform spectrum calibration at intervals, avoiding the high computational overhead caused by performing Fourier transform and polar coordinate processing in each bottleneck unit. At the same time, long-range dependencies are established through continuous path state mixing, and local edges and textures are supplemented through local residual blocks.

[0066] like Figure 2 As shown, the Dual Route Spectral Block (DRSB) in this embodiment simultaneously performs local spatial feature extraction, frequency domain adaptive enhancement, directional artifact detection, and dynamic gating fusion while maintaining the same input and output dimensions. Compared to ordinary convolutional residual blocks, DRSB does not rely solely on fixed convolutional kernels to extract local information. Instead, it divides features into a main path and a gated path, jointly performing image restoration feature modeling through spectral adaptive convolution and directional gating mechanisms. The main path is responsible for generating the main restoration features with frequency domain enhancement capabilities, while the gated path is responsible for generating a spatial weight map based on the horizontal, vertical, and local structures of the image. This weight map is then used to selectively modulate the features generated by the main path.

[0067] The DRSB module primarily enables the main path to have frequency domain adaptive modeling capabilities through FDConv, making it suitable for handling the problem of abnormal spectral distribution in sparse angle CT images. It constructs orientation-aware gating through ARG (Axis Route Gate), which can capture horizontal, vertical, and local structures, and is more sensitive to stripe artifacts, edge structures, and directional degradation. Through bidirectional cue connectivity, gating, and residual gain, the main path and the gated path form a cooperative relationship, while ensuring training stability.

[0068] Specifically, the DRSB module first performs channel normalization (LayerNom) on the input features. This is to stabilize the response distribution of different channels without disrupting the spatial structure of the CT image.

[0069] This represents the current input features of the DRSB module at a certain location. This applies to DRSB modules at different network levels and locations. They are not the same. Specifically, the input to the DRSB module in Encoder1 is the shallow feature X0 obtained from the initial convolution; the input to the DRSB module in Encoder2 is the downsampled feature of the output feature of Encoder1; and the input to the DRSB module in Encoder3 is the downsampled feature of the output feature of Encoder2. The bottleneck layer output is upsampled and then enters the decoding stage. The input to the DRSB module in Decoder3 is the feature obtained by concatenating the upsampled feature of Decoder3 with the output feature of Encoder3 and then performing a 1×1 convolution to reduce the dimensionality; the input to the DRSB module in Decoder2 is the feature obtained by concatenating the upsampled feature of Decoder2 with the output feature of Encoder2 and then performing a 1×1 convolution to reduce the dimensionality; and the input to the DRSB module in Decoder1 is the feature obtained by concatenating the upsampled feature of Decoder1 with the output feature of Encoder1 and then performing a 1×1 convolution to reduce the dimensionality. If a network stage contains multiple cascaded DRSB modules, the input of the first DRSB module is the input feature of that stage, and the input of subsequent DRSB modules is the output feature of the previous DRSB module.

[0070] After normalization, 1×1 convolution (1×1 Conv) is used to expand the channels. Then it goes through a 3×3 depthwise convolution (3×3DWConv). Depth convolution extracts local spatial relationships only within each channel, with a smaller number of parameters, but it can supplement neighborhood texture, edge and detail information.

[0071] The text employs 1×1 convolutions extensively here and there, primarily to facilitate channel information exchange without altering spatial dimensions. This expands the channel count from CCC to 2C2C2C, evenly dividing the expanded features into master and gated paths, and then performing channel projection and fusion at the module's end. Because 1×1 convolutions do not introduce additional spatial neighborhood mixing, they are suitable for channel expansion, compression, and cross-channel combination.

[0072] The 3×3 convolution is used extensively here and thereafter, primarily to extract local neighborhood information within each channel, supplementing edge, texture, and local grayscale variations while maintaining low parameter and computational costs. It introduces local spatial priors before spectral modeling. 3×3 represents a trade-off between local receptive field, parameter count, and computational cost, resulting in lower parameter and computational costs. While other odd-numbered convolutional kernels can also be used, they will alter the receptive field and computational cost.

[0073] Received Dividing into two in the channel dimension, there are , , B represents batch size, C represents the number of channels, and H and W represent feature map size. These are the main path features, responsible for generating the main recovery information; It is a gated path feature, responsible for determining whether features in the main path should be enhanced or suppressed.

[0074] The gated path (Route Seed) first provides a lightweight hint link to the main path (Main Seed): ;in, It is the first learnable scaling factor, initialized to 0.001. The gated branch contains directional and structural information. By using this cue connection to provide early hints to the main path, it can help the main path focus more on areas with obvious artifact distributions. The initial value is very small, so the stability of the main path will not be disrupted in the early stages of training.

[0075] The main path then enters adaptive spectral convolution (FDConv2D), with FDC stands for Frequency Dynamic Convolution, which is an adaptive convolution that combines frequency band decomposition and spectral convolution kernels to generate dynamic routes. The dynamic convolution kernel is determined by the input feature content. Therefore, there is .

[0076] Because sparse angular artifacts often manifest as frequency domain anomalies and directional stripe artifacts, relying solely on a fixed convolution kernel is insufficient to adapt to the artifact intensity and distribution in different samples. In this embodiment, adaptive spectral convolution is used to dynamically adjust the filtering method based on the image content, thereby enhancing the model's ability to model frequency domain degradation.

[0077] Then, the main path output will inversely regulate the gated path: , where β is the second learnable scaling factor, initialized to 0.001. The purpose of this cue connection is to inform the gated path which important spectral features have been extracted by the main path, thereby generating more reasonable gate weights. That is, the gated branch does not generate weights in a fixed manner, but rather makes judgments based on the restored features of the main path.

[0078] The gated path includes an ARG (Axis Route Gate) module, which contains three branch convolutional blocks for perceiving directionality: lateral convolution, longitudinal convolution, and local convolution. The input passes through these three branch convolutional blocks, and the three feature paths are then concatenated and fused. The purpose is to capture structural information in different directions. Artifacts in sparse angular CT images often exhibit a striped or directional diffusion distribution. Lateral convolution can perceive continuous structures in the horizontal direction, longitudinal convolution can perceive continuous structures in the vertical direction, and local convolution supplements neighborhood details.

[0079] ;

[0080] k represents the kernel length for both horizontal and vertical convolutions. The value of k is determined by the following factors: the continuous range of fringe artifacts along a single direction; the feature map size; the required directional receptive field; the computational cost and number of parameters; and the experimental results on the validation set. Smaller k focuses more on local directional structure, while larger k can cover longer fringes but increases computational cost. In this embodiment, k=7 is used as an empirical trade-off between directional receptive field and computational efficiency.

[0081] The gating features output by the ARG module are then processed... And the Sigmoid activation function, have ,in G is the dynamic gating weight, ranging from 0 to 1, representing the degree to which the main path output M is retained in different spatial locations and channels.

[0082] The main path output feature M is fused with the dynamic gating weight G of the gated path output using element-wise multiplication. The purpose of this design is not simply to superimpose two features, but to allow the gated path to selectively modulate the main path. When there are obvious artifacts or critical edges in certain regions, G can enhance the corresponding response; when certain regions are invalid noise or background, G can suppress the main path output. After element-wise multiplication, and then through a... The output has .

[0083] In addition, the addition of extended features through Generated compensation items Its function is to preserve the basic information in the original features and avoid excessive suppression of certain details after gating. For this task, it helps to maintain the continuity of edges, textures and grayscale.

[0084] The final fusion features are: .

[0085] The obtained fused features are then projected back to the original number of channels and undergo a gain compensation process, as follows:

[0086] ;

[0087] Where g is a learnable channel scaling parameter, initialized to 0.001. Final output. , can be represented as .

[0088] The polar coordinate spectrum calibration module transforms features from the spatial domain to the frequency domain, models the "frequency radius" and "frequency direction" in the polar coordinate spectrum space, calibrates the spectrum and phase, and finally transforms it back to the spatial domain and adds the residuals back to the input. Figure 3 As shown. The purpose of this module is to: model the relationship between low-frequency structure, mid-frequency texture and high-frequency details using frequency radius; model directional stripe artifacts in sparse angular CT using angular direction; and repair spectral energy distribution and structural location information through residual correction of amplitude and phase, while using γ to control the correction intensity to ensure training stability.

[0089] like Figure 3 As shown, specifically, channel normalization (LayerNom) is first performed to stabilize the spectral response of different channels. Then, reflection padding is performed to mitigate spectral artifacts caused by the periodic boundary assumption of the FFT. Reflection padding makes the edges more continuous and reduces frequency domain leakage. The padding ratio is set to 0.0625 (0.0625 is an empirical configuration in the current embodiment. This ratio is used to improve boundary continuity and reduce spectral leakage caused by the periodic boundary assumption of the FFT without excessively increasing the Fourier computation size. A ratio that is too small may not adequately alleviate boundary discontinuities, while a ratio that is too large will increase computational cost and introduce more mirror regions), approximately 1 / 16 of the height and width. This is specifically expressed by the following formula:

[0090] ;

[0091] Input here This is the current input feature of the polar coordinate spectrum calibration module. According to the network structure, the bottleneck layer consists of multiple bottleneck units connected in series. In one embodiment, only the first and third bottleneck units contain the polar coordinate spectrum calibration module. The output features of the third-layer encoder are first downsampled to obtain the bottleneck layer input features, which serve as the input to the polar coordinate spectrum calibration module in the first bottleneck unit. After the first bottleneck unit outputs, the input continues to the second bottleneck unit, and the features output by the second bottleneck unit serve as the input to the polar coordinate spectrum calibration module in the third bottleneck unit. Therefore, the input features in different bottleneck units... This represents the characteristics passed to the corresponding polar coordinate spectrum calibration module at different stages.

[0092] The output of the reflection fill is then subjected to a two-dimensional Fourier transform (FFT2) and spectral centering (FFTshift). , , FFTshift moves low frequencies to the center of the spectrum and high frequencies to the periphery, with the aim of dividing the spectrum into polar coordinates according to radius and angle.

[0093] After passing through the amplitude-phase decomposition module, the spectrum is divided into amplitude A and phase. , , This can compress the dynamic range of amplitude and prevent excessive low-frequency energy from causing training instability.

[0094] The phase is not directly used as a single angle input, but instead becomes... Then pieced together The significance of this treatment is to avoid phase differences between - and A numerical jump occurred nearby.

[0095] The Polar Pooling module implements polar coordinate pooling, transforming a two-dimensional spectrogram into polar coordinate spectral feature representations organized by "frequency" and "directional angle." For sparse angular CT images, stripe artifacts often exhibit strong directionality, making angle modeling crucial. First, the Polar Pooling module establishes a polar coordinate index for the spectral plane, and for each spectral point... The relationships with radius and angle are as follows: , u represents the horizontal frequency coordinate, and v represents the vertical frequency coordinate.

[0096] Then, the spectrum is divided into radius intervals and angle intervals, so that each spectrum point falls within a certain polar coordinate spectrum region. For example, the current configuration is that the radius interval radius_bins is 4 and the angle interval angle_bins is 8, which means that the spectrum is divided into 32 polar coordinate spectrum regions, and each spectrum point exists exactly in one of the divided polar coordinate spectrum regions.

[0097] Then, the average value of the features in each polar coordinate spectral region is calculated. The average value obtained is a representative feature of the spectral region in polar coordinates. This represents the spectral characteristics of the polar coordinate spectral region b. This represents the number of features in the polar coordinate spectral region b. Represents the spectrum points The input features. The spectral region features of all polar coordinate spectral regions constitute the polar coordinate spectral feature representation unit: .

[0098] Then through Map the channel from 3C back to C, i.e. This process is equivalent to fusing the three types of spectral information—amplitude, cosine phase, and sine phase—to obtain a more compact polar coordinate spectral representation.

[0099] The polar coordinates are then divided into two branches: an angle branch and a radius branch. The angle branch is processed by a 1×3 depthwise convolution. Its function is to model the relationship between spectra in different directions; for example, spectral anomalies in the Euclid direction may correspond to sparse corner fringe artifacts. The radius branch is processed by a 3×1 depthwise convolution. The radius corresponds to the frequency range. A lower radius represents low-frequency structures, while a higher radius represents high-frequency edges and details. Radial branches are used to model the relationships between low, mid, and high frequencies.

[0100] pass The fusion weights are obtained, and then the angular feature representation units (composed of spectral features in different frequency angular directions) and the radius feature representation units (composed of spectral features in different frequency radii) are fused based on these weights to obtain the polar coordinate spectral feature representation units. The specific calculation process is as follows:

[0101] ;

[0102] This process means the model adaptively determines whether the current spectral calibration relies more on angular direction information or frequency radius information. For directional fringe artifacts, angular direction information is more important; for blurring and loss of detail, the relationship between high and low frequency radii is more important.

[0103] Subsequently, the fused polar coordinate spectral feature representation units undergo 1×1 convolution processing. It is divided into two parts in the channel dimension. , ,in This represents the amplitude correction feature representation unit. This represents the phase correction feature representation unit.

[0104] Each spectral point will take its corresponding polar coordinate spectral feature representation unit based on its polar coordinate spectral region. In other words, the polar coordinate spectral feature representation unit will have its complete spectrum retrieved by the broadcast module. The calculation process is as follows: , , .

[0105] Next, amplitude and phase calibration are performed. The original amplitude and phase are retained through bypass, and then a correction value is added. The specific calculation process is as follows:

[0106] ;

[0107] in, Indicates the amplitude correction scale. This represents the phase correction scale. The initial values ​​are set to 0.1 and 0.01 respectively. The initial value of the phase correction scale is smaller because the phase directly affects the position of the image structure, and excessive phase perturbation can easily damage the image shape.

[0108] Next, frequency domain reconstruction is performed, first restoring the log amplitude. Then, the complex spectrum is reconstructed based on the amplitude and phase to restore the original FFT arrangement:

[0109] ;

[0110] Continue restoring to the spatial domain and cropping the area filled with reflections:

[0111] ;

[0112] After another 3×3 depthwise convolution, we get , .

[0113] Finally, the residual is scaled and added back to the input to obtain the final output. , , where γ is a learnable third scaling parameter. The initial value was 0.001.

[0114] The purpose of the Path State Hybrid Module (Polar Spectrum Calibrator, abbreviated PSC) is to unfold two-dimensional image features into a sequence in multiple directions and use a state-space model to perform long-range dependency modeling, thereby solving the problem of limited receptive field in ordinary convolution.

[0115] Specifically, the first step is to perform channel normalization (LayerNom). Its function is to stabilize the numerical distribution of different channels, making subsequent state-space scans more stable. After normalization, 1×1 Conv is used to expand the channels to 2C. Then it goes through a 3×3 depthwise convolution. , This step involves supplementing local spatial information. While state-space models excel at long-range dependencies, local textures, edges, and neighborhood relationships still require convolution to enhance them.

[0116] Here The input to the path state mixing module varies depending on its location. For example, the input to the path state mixing module in the first bottleneck unit is the output of its polar coordinate spectrum calibration module, while the input to the path state mixing module in the second bottleneck unit is the output of the local residual block in the first bottleneck unit.

[0117] Then, the feature channels are divided into two paths: Signal and Route branches. S is the signal branch, used to enter the state space scan; R is the route branch, used to generate gated modulation weights.

[0118] S first unfolds the two-dimensional features into four one-dimensional sequences according to the four path sequences. Then, it performs one-dimensional local convolution and parameter projection on the one-dimensional sequences of each path. Then, it performs selective state scanning on each path. Then, it restores the scanning results to the original two-dimensional position. Finally, it splices and merges all paths.

[0119] The four paths are row serpentine, column serpentine, reverse row, and reverse column. For each path p, the two-dimensional features are unfolded into a one-dimensional sequence. ,in After performing one-dimensional local convolution and parameter projection stacking on the four paths, we have: .

[0120] Before scanning, a 1x1 depthwise convolution is applied to the sequence. , The purpose is to add short-range context to each scan sequence, so that the state-space model has both local sequence perception and long-range memory capabilities.

[0121] The parameters required to generate the state-space model through linear projection: ,in Indicates the dynamic step size. This represents the mapping from input to state. This represents the mapping from state to output. The module also maintains state decay parameters internally. The negative exponential form ensures that the state decays to a negative value, making the sequence state update more stable.

[0122] Perform a selective state scan on each path. The output is ,in Represents the hidden state at position t. This represents the t-th input token. Let A represent the dynamic step size and A represent the state decay matrix. Indicates the input projection. This represents the output projection, where D represents the jump connection scaling factor. , and Both are related to the input, so this is an input-adaptive state-space scan, rather than a fixed convolution or a fixed loop.

[0123] After each path scan is completed, the one-dimensional sequence is restored to its original two-dimensional spatial position. The four paths are obtained Then perform channel dimension splicing. Then fused via 1×1 Conv ,therefore, A single path can only propagate information in one sequence, while a four-path scan can simultaneously model lateral, vertical, forward, and reverse dependencies, reducing directional bias.

[0124] The other half of the route branch is activated by the SiLU function. Then multiply it element-wise with the state scan output. The role of this gating mechanism is to control the intensity of the state-space output. Not all regions require strong long-range modeling; for example, smooth regions may only require a weaker response, while fringe artifacts or structural boundary regions require stronger global dependency modeling.

[0125] Finally, the output projection is concatenated with the residual, and the fused result is passed through a 1×1 Conv. Then, through residual scaling, , , It is a learnable fourth channel scaling parameter, initialized to 0.001.

[0126] To train the PSDNet model, paired samples of sparse-angle PCCT images and high-quality label images can be constructed. The sparse-angle images serve as network input, while the high-quality label images serve as supervision targets. Images can be uniformly converted into single-channel grayscale tensors and normalized to the 0-1 range. During training, enhancement methods such as random cropping, flipping, or rotation can be used to improve the network's adaptability to different anatomical structures and artifact orientations.

[0127] The training loss can be composed of a weighted average of robust pixel loss, structural similarity loss, gradient consistency loss, and frequency domain consistency loss. Robust pixel loss constrains the overall grayscale error, structural similarity loss constrains local structural consistency, gradient consistency loss constrains edges and textures, and frequency domain consistency loss constrains Fourier amplitude or complex spectrum differences. The weights of each loss can be adjusted according to the data characteristics.

[0128] If the PCCT data is used for different sizes, different scanning protocols, or different projection angles, it can be adapted by adjusting the number of basic channels, the number of stacked modules, the number of polar coordinate intervals, the size of the convolution kernel, the size of the training patch, or the loss weights.

[0129] Figure 5 This is a reconstruction data image of a walnut from a Micro-CT image with a sparse angle of 1 / 16th. Figure 6 This is a reconstruction data image of a walnut from a Micro-CT image with a sparse angle of 1 / 12. Figure 7This is a reconstruction data image of a mouse from a Micro-CT image with a 1 / 12 sparse angle. Figure 8 Image reconstruction data of a mouse from a Micro-CT image with a sparse angle of 1 / 16th. Figures 5-8 In the image, from left to right, are the initial reconstruction image of the sparse angle, the reconstruction image of the method of this invention, and the original image reference image. The comparison shows that this invention can reduce radial stripes and random noise generated by sparse angle sampling, while maintaining the continuity of the target contour, internal texture, and local grayscale during the reconstruction process. Figure 5 and Figure 6 This demonstrates the reconstruction effect of the present invention on walnut images under different sparse angle ratios. Figure 7 and Figure 8 This demonstrates the reconstruction effect of the present invention on mouse images under different sparse angle ratios.

[0130] like Figure 9 As shown, based on Figure 1 The system shown implements a deep learning-based photon counting CT sparse angle image reconstruction method, which includes the following steps:

[0131] S10: Obtain photon-counted CT sparse angle images and extract initial features from them. , X is the input image, and X0 represents the initial features.

[0132] Here, a 3×3 convolution is used to extract initial features because a 3×3 convolution can simultaneously cover the central pixel and its eight neighbors, making it suitable for extracting local grayscale changes, edges, and initial textures in CT images. Especially when the stride is 1 and the padding is 1, the spatial dimensions of the input and output remain unchanged, which is convenient for subsequent residual connections and multi-scale processing. Moreover, compared with larger convolutions such as 5×5 and 7×7, the 3×3 convolution has a lower number of parameters and computational cost, and compared with the 1×1 convolution, the 3×3 convolution has a better ability to extract local spatial features.

[0133] S20, perform multi-scale dual-path spectral modulation on the initial features, for example, after three-scale hierarchical coding modulation, we have , , .

[0134] Dual-path spectral modulation involves adaptive spectral convolution on the main path, directional structure perception through a gated path, and selective modulation of the main path features using bidirectional cueing connections and dynamic gating weights to achieve local spatial feature extraction, frequency domain adaptive enhancement, directional artifact perception, and dynamic gating fusion.

[0135] The specific processing flow of dual-path spectrum modulation can be found in the relevant description in the aforementioned dual-path spectrum modulation module section, and will not be repeated here.

[0136] S30 performs polar coordinate spectrum calibration and path state mixing on the modulated features. .

[0137] Polar coordinate spectrum calibration involves modeling the "frequency radius" and "frequency direction" in polar coordinate spectrum space, then calibrating the spectral amplitude and phase, and finally converting it back to the spatial domain and adding the residuals back to the input. In other words, it involves transferring deep features into the frequency domain, aggregating tokens according to the frequency radius and frequency angle, and performing constrained residual calibration on the amplitude and phase. Path state mixing unfolds two-dimensional image features into a sequence in multiple directions and performs an input adaptive state space scan to establish long-distance spatial dependencies.

[0138] The specific processing flow for polar coordinate spectrum calibration and path state mixing can be found in the relevant descriptions in the aforementioned polar coordinate spectrum calibration module and path state mixing module sections, and will not be repeated here.

[0139] S40 performs skip-connect decoding on the mixed features, that is, concatenates the corresponding scale features of the coding layer with the decoded upsampled features through skip connections. , , The convolution prediction residual image is obtained by performing dual-path spectral modulation.

[0140] S50, the convolutional prediction residual image is added to the input image to obtain the reconstructed image, i.e. , Y represents the convolution prediction residual image, and Y represents the reconstructed image.

[0141] like Figure 10 As shown, this embodiment also provides an electronic device that may include a processor and a memory, wherein the memory is coupled to the processor. It is worth noting that this figure is exemplary, and other types of structures can be used to supplement or replace this structure to achieve data extraction, report generation, communication, or other functions.

[0142] like Figure 10 As shown, the electronic device may also include an input unit, a display unit, and a power supply. It is worth noting that the electronic device is not necessarily required to include these components. Figure 10 All components shown in the image. Furthermore, electronic devices may also include... Figure 10 For components not shown, please refer to existing technologies.

[0143] A processor, sometimes also called a controller or operating control, may include a microprocessor or other processor device and / or logic device that receives input and controls the operation of various components of an electronic device.

[0144] The memory may be one or more of the following: a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It can store information such as the processor's configuration information and the instructions executed by the processor. The processor can execute programs stored in the memory to perform information storage or processing. In one embodiment, the memory also includes a buffer memory, or buffer, to store intermediate information.

[0145] This invention also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed in an electronic device, the program product causes the electronic device to perform the operation steps included in the method of this invention.

[0146] This invention also provides a storage medium storing computer-readable instructions that cause an electronic device to perform the operation steps included in the method of this invention.

[0147] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The above specific embodiments are merely several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A deep learning-based photon counting CT sparse angle image reconstruction system, characterized in that, It includes a data collection module, a residual U-shaped image restoration network model, and an image reconstruction module; the residual U-shaped image restoration network model includes an encoding layer, a bottleneck layer, and a decoding layer; The data collection module is used to acquire photon-counted CT sparse angle images and extract initial features from them; The coding layer is used to perform dual-path spectral modulation on the input features at multiple scales; The bottleneck layer is used to perform polar coordinate spectrum calibration and path state mixing on the modulated features. The decoding layer is used to perform skip-connected decoding on the mixed features and perform dual-path spectral modulation at multiple scales to obtain the convolutional prediction residual image. The image reconstruction module is used to add the convolution prediction residual image and the input image to obtain the reconstructed image.

2. The photon counting CT sparse angle image reconstruction system based on deep learning according to claim 1, characterized in that, The coding layer includes cascaded encoders at three scales, and the decoding layer includes cascaded decoders at three scales. Each encoder and decoder is equipped with a dual-path spectrum modulation module for dual-path spectrum modulation of the input features. The decoder upsampled features are concatenated with the output features of the corresponding scale-level encoder, and then dimensionality reduced before dual-path spectral modulation is performed.

3. The photon counting CT sparse angle image reconstruction system based on deep learning according to claim 2, characterized in that, The dual-path spectrum modulation module performs dual-path spectrum modulation as follows: For input features First, channel normalization is performed, followed by 1×1 convolution and 3×3 depthwise convolution; Features after depthwise convolution Decomposition into main path features and gated path features ; The gated path provides hints and connections to the main path, making the main path characteristic of... , This is the first scaling factor; The main path undergoes adaptive spectral convolution to output features. , Dynamic convolution kernel; The main path provides a hint connection to the gated path, causing the gated path characteristics to change. β is the second scaling factor; The modified gated path features are convolved and merged by horizontal convolution, vertical convolution and local convolution, and then generated by convolution and activation function to produce gated weights G. The main path output features M after adaptive spectral convolution are fused with the gate weights G of the gated path output through element-wise multiplication, and then output after 1x1 convolution. , ; After introducing the compensation term S and gain compensation, the final output is... , , where g is the channel scaling parameter.

4. The photon counting CT sparse angle image reconstruction system based on deep learning according to claim 1, characterized in that, The bottleneck layer contains four cascaded bottleneck units. The first and third bottleneck units include a polar coordinate spectrum calibration module and a path state mixing module, while the second and fourth bottleneck units only contain a path state mixing module. The polar coordinate spectrum calibration module is used to perform polar coordinate spectrum calibration, and the path state mixing module is used to perform path state mixing on the calibrated polar coordinate spectrum.

5. The photon counting CT sparse angle image reconstruction system based on deep learning according to claim 4, characterized in that, The polar coordinate spectrum calibration module performs the following operations in sequence: First, channel normalization is performed, then reflection filling is performed, then two-dimensional Fourier transform is performed, and finally spectrum centering is performed. Convert the spectrogram into polar coordinate spectral feature representation units; The polar coordinate spectral feature representation unit is divided into an angle branch and a radius branch. The angle branch is processed by a 1×3 depth convolution, and the radius branch is processed by a 3×1 depth convolution. The angle branch and the radius branch after depth convolution are then fused based on weights. The fused polar coordinate spectral feature representation units are processed by 1×1 convolution and then divided into amplitude correction feature representation units and phase correction feature representation units in the channel dimension, and then converted into a spectrum. Calibrate the amplitude and phase; After calibration, frequency domain reconstruction is performed to restore the Fourier arrangement, and then the spatial domain is restored, with the reflection filling area being cropped. After a 3×3 depthwise convolution, the residual is scaled and added to the input to obtain the final output.

6. The photon counting CT sparse angle image reconstruction system based on deep learning according to claim 4, characterized in that, The path state mixing module performs the following operations for path state mixing: First, channel normalization is performed, then the channels are expanded to 2C, and then a 3×3 depthwise convolution is performed. The feature channels are divided into Signal branches and Route branches. The Signal branch is expanded into a one-dimensional sequence according to four paths: row serpentine, column serpentine, reverse row, and reverse column. Then, the one-dimensional sequences obtained from each path are stacked. Perform a 1x1 depthwise convolution on the stacked sequence and generate the parameters required for the state space model through linear projection; Selective state scanning is performed on each path, and the two-dimensional spatial positions obtained from the scanning of the four paths are stitched together and then fused through 1×1 convolution. After the route branch is activated by the SiLU function, it is multiplied element-wise with the fusion result, then passed through a 1×1 depthwise convolution, and finally scaled by the residual and added to the input to obtain the final output.

7. A deep learning-based photon counting CT sparse angle image reconstruction method, characterized in that, Includes the following steps: S10: Obtain photon-counted CT sparse angle images and extract initial features from them; S20, performs dual-path spectral modulation on the initial features at multiple scales; S30 performs polar coordinate spectrum calibration and path state mixing on the modulated features; S40 performs skip-connected decoding on the mixed features and performs dual-path spectral modulation at multiple scales to obtain the convolutional prediction residual image; S50, the convolutional prediction residual image is added to the input image to obtain the reconstructed image.

8. The photon counting CT sparse angle image reconstruction method based on deep learning according to claim 7, characterized in that, The dual-path spectrum modulation processing includes the following operations in sequence: The input feature X is first normalized by channels, and then 1×1 convolution and 3×3 depth convolution are performed. Features after depthwise convolution Main path features and gated path features ; The gated path provides hints and connections to the main path, making the main path characteristic of... , It is the first scaling factor; The main path undergoes adaptive spectral convolution to output features. , Dynamic convolution kernel; The main path provides a hint connection to the gated path, causing the gated path characteristics to change. β is the second scaling factor; The modified gated path features are convolved and merged by horizontal convolution, vertical convolution and local convolution, and then generated by convolution and activation function to produce gated weights G. The main path output features M and the gate weights G of the gated path output are fused by element-wise multiplication, and then output after a 1x1 convolution. , ; After introducing the compensation term S and gain compensation, the final output is... , , where g is the channel scaling parameter.

9. The photon counting CT sparse angle image reconstruction method based on deep learning according to claim 7, characterized in that, The polar coordinate spectrum calibration process includes the following operations in sequence: First, channel normalization is performed, then reflection filling is performed, then two-dimensional Fourier transform is performed, and finally spectrum centering is performed. Convert the spectrogram into polar coordinate spectral feature representation units; The polar coordinate spectral feature representation unit is divided into an angle branch and a radius branch. The angle branch is processed by a 1×3 depth convolution, and the radius branch is processed by a 3×1 depth convolution. The angle branch and the radius branch after depth convolution are then fused based on weights. The fused polar coordinate spectral feature representation units are processed by 1×1 convolution and then divided into amplitude correction feature representation units and phase correction feature representation units in the channel dimension, and then converted into a spectrum. Calibrate the amplitude and phase; After calibration, frequency domain reconstruction is performed to restore the Fourier arrangement, and then the spatial domain is restored, with the reflection filling area being cropped. After a 3×3 depthwise convolution, the residual is scaled and added to the input to obtain the final output.

10. The photon counting CT sparse angle image reconstruction method based on deep learning according to claim 7, characterized in that, The processing of the path state mixing includes the following operations in sequence: First, channel normalization is performed, then the channels are expanded to 2C, and then a 3×3 depthwise convolution is performed. The feature channels are divided into Signal branches and Route branches. The Signal branch is expanded into a one-dimensional sequence according to four paths: row serpentine, column serpentine, reverse row, and reverse column. Then, the one-dimensional sequences obtained from each path are stacked. Perform a 1x1 depthwise convolution on the stacked sequence and generate the parameters required for the state space model through linear projection; Each path is selectively scanned, and the two-dimensional spatial positions obtained from the four path scans are stitched together and then fused using a 1×1 convolution. After the route branch is activated by the SiLU function, it is multiplied element-wise with the fusion result, then passed through a 1×1 depthwise convolution, and finally scaled by the residual and added to the input to obtain the final output.

Images