A deep learning-based multi-material basis substance decomposition method

Abstract

The application is a multi-material base substance decomposition method based on deep learning, and the steps are as follows: step one: according to the three-element base substance group list with priority, the part of the dual-energy CT image data set is decomposed pixel by pixel to obtain a series of corresponding multi-base substance decomposition images; step two: the base substance decomposition image obtained in step one is used as the label of decomposition, and the base substance decomposition network is used for supervised learning; step three: based on the pre-training network weight of step two, the decomposition network is self-supervised trained using all dual-energy CT image data to improve the base substance decomposition effect of the network. The application designs the corresponding network structure, loss function and training strategy according to the characteristics of multi-base substance decomposition, solves the problem of lack of true value in base substance decomposition, and guarantees the accuracy and reliability of the decomposition result. The application can generate the distribution information of different substances, can accurately evaluate the chemical composition of the measured tissue, and makes up for the defect that the conventional CT image cannot distinguish similar density substances.

Description

Technical Field

[0001] This invention relates to a deep learning-based method for multi-material-based matter decomposition, belonging to the fields of medical image processing, pattern recognition, and computer vision. It mainly involves the principles of dual-energy CT imaging, matter decomposition, self-supervised learning, and deep convolutional networks, and has broad application prospects in related fields such as material composition analysis and virtual imaging. Background Technology

[0002] Dual-energy CT primarily uses two types of X-rays with different energies to scan the same object. This effectively eliminates the influence of tissue density, allows for a reasonable assessment of the tissue's chemical composition, and provides information on the distribution of different substances.

[0003] One of the most important applications of dual-energy CT is the analysis of matrix materials. Matrix material images can show the distribution of different substances, providing information that conventional CT images cannot display, which has significant clinical value. However, dual-energy CT can only produce images of two matrix materials in its physical principle, while in clinical applications, doctors usually need to obtain the content distribution of multiple substances, thus requiring the production of images of multiple matrix materials.

[0004] Regarding the decomposition of multi-matrix materials, scholars have conducted certain research, and the following methods are currently common: Kuijk et al. added the assumption of volume conservation to the decomposition of two-matrix materials, thereby achieving the decomposition of three-matrix materials (see reference, Kuijk, J. Grashuis, J. Steenbeek, et al, "Evaluation of postprocessing dual-energy methods in quantitative computedtomography: Part 1. Theoretical considerations". Investigative Radiology, 876-881, 1990). Building upon this, others achieved the decomposition of various matrix materials. They proposed a priority list of ternary matrix material groups, and by iterating over each pixel of the image, they solved for the optimal ternary matrix material group corresponding to each pixel, thereby achieving matrix material decomposition (see reference, Mendenza, Lamb, Sakhani, A flexible method for multi-material decomposition of dual-energy CT images. IEEE Transactions on Medical Imaging, 99-116, 2013. (PR S.)). P. Lamb, D. V.S.A.H.A., “A flexible method for multi-material decomposition of dual-energy CT images”. IEEE Transactions on Medical Imaging, 99-116, 2013). However, the above methods directly solve matrix equations, and this direct inversion method easily amplifies noise in the image and produces more artifacts. To address this shortcoming, Long et al. proposed a method based on statistical image reconstruction. This method obtains an accurate matrix material image by minimizing the penalized likelihood cost function and uses an edge-preserving regularization term to reduce the interference of image noise (see reference, Y. Long, J.A. Fessler, “Multi-material decomposition using statistical image reconstruction for spectral CT”. IEEE Transactions on Medical Imaging, 1614-1626, 2014). Xue et al. designed an iterative method for multi-material matrix decomposition using dual-energy CT images. They utilized material sparsity as a constraint and added least squares data fidelity terms, total variation terms, and volume conservation constraints to form the decomposition objective function. Among them, the total variation constraints can effectively suppress image noise (see reference, Xue, Qin, Luo, et al., Multi-Material Decomposition for Single Energy CT Using aterial Sparsity Constraint. IEEE Transactions on Medical Imaging, 1303-1318, 2021).

[0005] In recent years, deep learning has achieved success in many computer vision tasks due to the powerful fitting ability of networks. Many scholars have adopted deep learning methods to solve the problem of matrix decomposition in dual-energy CT. Xu et al. proposed a deep neural network structure composed of a fully convolutional network and a fully connected network. The fully convolutional network extracts features from the input dual-energy CT image, while the fully connected network calculates the decomposition coefficients of the matrix based on the obtained feature vectors. Experiments have shown that the network has high decomposition accuracy and noise resistance (see reference, Xu, Yan, Zhang et al, “Image decomposition algorithm for dual-energy computed tomography via fully convolutional network”. Computational and Mathematical Methods in Medicine, 1-9, 2018). Zhang et al. designed a two-input, two-output cross-network structure based on the characteristics of the matrix decomposition formula. The network takes dual-energy CT images as input and two base material decomposition images as output. The cross structure can realize information exchange in the decomposition process of the two base materials and plays an important role (see reference, W. Zhang, H. Zhang, L. Wang, et al., “Image domain dual material decomposition for dual-energy CT using butterfly network”. Medical Physics, 2037-2051, 2019).

[0006] However, overall, research on multi-matrix decomposition based on dual-energy CT is still in its early stages. The main challenges lie in three aspects: First, dual-energy CT equipment is complex and expensive, and its clinical application is not as widespread as general CT equipment, making it difficult to obtain large-scale dual-energy CT datasets. Second, theoretically, the matrix decomposition process based on dual-energy CT can only decompose two matrix substances, while practical applications usually require the decomposition of three or more matrix substances. Third, because the specific content distribution of matrix substances such as muscle, fat, and bone in the human body is difficult to measure, the matrix decomposition process lacks corresponding ground truth values, making it impossible to quantitatively evaluate the decomposition results. Existing methods mainly include traditional optimization methods and deep learning-based decomposition methods. Traditional optimization methods are computationally intensive and prone to amplifying noise, while deep learning-based methods can only decompose two matrix substances. This invention proposes a deep learning-based multi-material matrix decomposition method, designing corresponding network structures, loss functions, and training strategies based on the characteristics of the matrix decomposition process. This method first generates matrix labels based on a priority list of ternary matrix groups, then performs supervised pre-training based on these labels to accelerate network convergence, and finally performs self-supervised training based on the loss function to further improve the decomposition results of the network. Summary of the Invention

[0007] To address the aforementioned problems, the present invention aims to provide a multi-material-based matter decomposition method based on deep learning. This method first generates matter labels based on a priority list of ternary matter groups, then performs supervised pre-training based on these labels to accelerate network convergence, and finally performs self-supervised training based on a loss function to further improve the network's decomposition results. The technical solution and overall concept of this invention are mainly reflected in the following three aspects:

[0008] 1) Decompose some dual-energy CT images pixel by pixel according to the priority list of ternary matrix groups to obtain matrix decomposition images, which are used as labels for pre-training.

[0009] 2) Construct a basic material decomposition network and use the labels obtained from the decomposition in step one for supervised learning to accelerate the convergence of the network;

[0010] 3) Design a loss function for self-supervised training, fine-tune and optimize the weight parameters of the network, and finally perform accurate multi-material-based decomposition of dual-energy CT.

[0011] This invention relates to a deep learning-based method for decomposing multi-material-based substances, the specific steps of which are as follows:

[0012] Step 1: Decompose a portion of the dual-energy CT image dataset pixel by pixel according to the priority list of ternary matrix groups to obtain a series of corresponding multi-matrix decomposed images;

[0013] A subset of data was randomly selected from the dual-energy CT dataset. The number and types of matrix materials to be decomposed were determined, and a priority list of ternary matrix material groups was constructed based on the differences in their attenuation characteristics. Then, matrix material decomposition was performed on each pixel of the CT image according to the ternary matrix material group order, ultimately outputting multiple matrix material images. The specific process is as follows:

[0014] S11. Construct a list of ternary matrix groups. Select the number and types of matrix materials to be decomposed, list all possible ternary matrix groups, and then find the linear decay coefficients of the selected matrix materials at different energies from the NIST database. For each ternary matrix group, calculate the differences in decay characteristics among the matrix materials in the group. Rank the ternary matrix groups according to the differences in decay characteristics, thus outputting a priority list of ternary matrix groups.

[0015] S12. Perform multi-matrix decomposition. For each pixel of the CT image, perform matrix decomposition sequentially according to the ternary matrix groups. Stop solving when a physically meaningful solution exists; this solution is the result of the matrix decomposition for that pixel. If none of the ternary matrix groups can yield a solution with all values ​​in the range of 0-1, select the solution whose sum of absolute values ​​is closest to 1. After statistically analyzing the matrix decomposition results for each pixel, obtain the matrix decomposition image for the entire image.

[0016] Step 2: Use the base material decomposition image obtained in Step 1 as the decomposition label, and perform supervised learning using the base material decomposition network.

[0017] A matrix decomposition network was constructed, using dual-energy CT images as input and the matrix decomposition images obtained in step one as labels for pre-training. The decomposition network encodes depth features into its input using an encoder, followed by decoding using a decoder. A co-learning module was introduced to fuse information from CT images of different energies, resulting in a preliminary decomposition of the input dual-energy CT image. The network parameters were then optimized based on the mean squared error loss function and multi-scale perception loss function between the decomposition results and the labels, ultimately outputting the pre-trained decomposition network parameters. The specific process is as follows:

[0018] S21. A base material decomposition network is constructed. The network mainly consists of two deep convolutional encoders and one deep convolutional decoder. Its input is a portion of the dual-energy CT images to be decomposed, and its output is the preliminary decomposition result of the input images. The two deep convolutional encoders in the network use the ConvNeXt network as the basic framework to extract multi-scale convolutional features from the input dual-energy CT images. As the network depth increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information, while deep features contain more global semantic information. To improve the model's performance and enable the network to focus on more important regions during the decomposition process, an attention module is also introduced into the encoder. This module takes the feature map generated by the encoder as input and outputs the importance of each channel in the feature space. The deep convolutional decoder in the network fuses and decodes the multi-scale convolutional features extracted by the encoder to obtain the preliminary decomposition result of the input image. Furthermore, based on the characteristics of the input dual-energy CT images, a co-learning module is designed to fuse information from CT images of different energies. This module uses the feature maps of different energies output by the two convolutionals to generate fusion weights and outputs the fused feature map.

[0019] S22. The matrix decomposition network constructed in S21 is trained and optimized using the multi-matrix decomposition images obtained in step one and the corresponding dual-energy CT images. First, the network parameters are initialized using random initialization. Then, the decomposition loss function L between the network's decomposition results on the input images and the corresponding labels is calculated. The decomposition loss function L is the sum of the mean squared error loss function and the multi-scale perception loss function. The expression for the mean squared error loss function is: Where x t (p)(t=1,...,N,p=1,...,M) represents the decomposition coefficient of the t-th base material generated by the network for the p-th pixel, y t (p)(t=1,...,N,p=1,...,M) represents the decomposition coefficient of the t-th base material in the label for the p-th pixel, where M represents the number of pixels and N represents the types of base materials decomposed. The expression for the multi-scale perceptual loss function is as follows: in Let xi represent the output at the s-th scale of the pre-trained network, and yi = 1, ..., N represent the result and label of the decomposed network, respectively. N represents the number of base material images, and S represents the number of scales. A stochastic gradient descent optimizer is used to optimize the network, with an initial learning rate set to 10. -2 Furthermore, momentum update and weight decay mechanisms are introduced, and the loss function is optimized by adjusting the network weight values ​​through gradient backpropagation. The final output is the pre-trained decomposed network parameters.

[0020] Step 3: Based on the pre-trained network weights from Step 2, the decomposition network is trained using all dual-energy CT image data in a self-supervised manner to improve the network's matrix decomposition effect.

[0021] The decomposition network was fine-tuned using all dual-energy CT image data. A loss function was constructed based on the relationship between the base material image and the dual-energy CT image to enable self-supervised learning of the network. Edge-preserving and material sparsity terms were introduced to optimize the decomposition network parameters. Specifically, the base material decomposition network was first initialized with the pre-trained parameters obtained in step two. Then, the decomposition network was self-supervised trained based on the loss function L. The loss function L is derived from the self-supervised loss function Li. self Edge Preservation Function L edge With material sparsity function L sparsity It consists of three components. The expression for the self-supervised loss function is: Matrix A consists of the linear attenuation coefficients of the matrix material at different energies, x represents the solved matrix material decomposition image, and μ is the vector of linear attenuation coefficients of the tissue under test at different energies, which can be obtained from dual-energy CT images. This loss function requires that the dual-energy CT image reconstructed from the matrix material image generated by the network be as close as possible to the input dual-energy CT image, ensuring the accuracy of the matrix material image decomposed by the network, thus solving the problem of lacking ground truth labels in matrix material decomposition. The expression for the edge-preserving function is:

[0022] in This represents solving for the edge of the image in the k-direction, x i (i = , ..., N) represents the decomposition diagram of the i-th base substance, Y j (j = L, H) represents the input low-energy and high-energy CT images, N represents the amount of matrix material, and M represents the total number of image pixels. This loss function ensures that the matrix-decomposed image and the original dual-energy CT image share common anatomical edges. The expression for the material sparsity function is: Where X i (i = 1, ..., N) represents the decomposition image of the i-th matrix material, where N represents the number of matrix materials and M represents the total number of pixels in the image. Minimizing this function makes the matrix material composition sparse for each pixel. The initial learning rate is set to 10. -4 Furthermore, momentum update and weight decay mechanisms are introduced, and the loss function is optimized by adjusting network weights through gradient backpropagation. The final output is the accurate matrix decomposition result of the input dual-energy CT image.

[0023] The process of a deep learning-based multi-material matrix decomposition method is as follows: Figure 1As shown, this method first generates matrix material labels based on a priority list of ternary matrix material groups, then performs supervised pre-training based on these labels to accelerate network convergence, and finally performs self-supervised training based on the loss function to further improve the network's decomposition results, obtaining multiple accurate matrix material decomposition images.

[0024] The advantages and effects of this invention are as follows: This invention proposes a multi-material matrix decomposition method based on deep learning. In clinical practice, it is often required to decompose three or more matrix substances. However, based on dual-energy CT, matrix decomposition can theoretically only uniquely decompose two matrix substances, thus multi-matrix decomposition is an ill-posed problem. Furthermore, the specific content distribution of matrix substances is difficult to determine, so the matrix decomposition process lacks a corresponding true value. Based on this, this invention constructs a multi-matrix decomposition network. This method first uses an easily implemented direct inversion method to obtain a subset of labels, then uses these labels to pre-train the network, and finally performs self-supervised training based on a loss function to improve decomposition performance. This method, by minimizing the loss function, can find a reasonable set of solutions to the ill-posed problem, solving the problem of the lack of a true value in matrix decomposition. The generated matrix images have clear edges, low noise, and the types of matrix substances to be decomposed can be arbitrarily set, effectively meeting practical needs. Attached Figure Description

[0025] Figure 1 This is a flowchart of the overall process for a deep learning-based multi-material-based matter decomposition method.

[0026] Figure 2 Flowchart for supervised pre-training label acquisition method.

[0027] Figure 3 This is a schematic diagram of the decomposition network of the base material.

[0028] Figure 4 This is a schematic diagram of a self-supervised training method.

[0029] Figures 5a-5n The diagram illustrates the results of each step in the method of this invention when the four base substances are decomposed.

[0030] Figures 6a-6q The diagram illustrates the results of each step in the method of this invention when the five basic substances are decomposed. Detailed Implementation

[0031] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0032] This invention is a deep learning-based method for decomposing multi-material-based substances, the overall process of which is as follows: Figure 1 As shown, the specific implementation details for each part are as follows:

[0033] Step 1: Decompose a portion of the dual-energy CT image dataset pixel-by-pixel based on a priority list of ternary matrix components to obtain a series of corresponding multi-matrix decomposed images.

[0034] S11、 Figure 2 This outlines the specific workflow for the base material decomposition method. First, a list of ternary base material groups needs to be constructed. The number and types of base materials to be decomposed are selected, and all possible ternary base material groups are listed. Then, the linear decay coefficients of the selected base materials at different energies are retrieved from the NIST database. For each ternary base material group, the differences in decay characteristics between the base materials within the group are calculated. In practice, the degree of linear independence of the vector groups is used as the evaluation criterion for these differences. Where β1, β2, and β3 represent the vectors corresponding to the three matrix substances in the ternary matrix, and these vectors are composed of the linear decay coefficients of the matrix substances at different energies. |G(β1, β2, β3)| is the determinant of the Gram matrix composed of β1, β2, and β3, and |||| represents the vector length. The larger the ω, the higher the linear independence of β1, β2, and β3, and the higher their ranking in the list. The ternary matrix groups are arranged according to the calculated ω values, thus outputting a priority list of ternary matrix groups.

[0035] S12. Perform multi-matrix decomposition. For each pixel of the CT image, perform matrix decomposition sequentially according to the ternary matrix group arrangement. Specifically, solve the following system of linear equations. Where μ iL With μ iH (i = 1, 2, 3) represent the linear attenuation coefficients of the i-th matrix material under low-energy and high-energy X-ray irradiation, respectively. α ip (i = 1, 2, 3) represents the decomposition coefficient of the i-th matrix material, which is the target value and should theoretically be within the range of 0-1. p represents the p-th pixel being decomposed. The solution process stops when a solution exists that satisfies the range of 0-1; this solution is the result of matrix material decomposition for that pixel. If none of the ternary matrix material groups can yield a solution where all values ​​are within the range of 0-1, then the solution whose sum of absolute values ​​is closest to 1 is selected. After statistically analyzing the matrix material decomposition for each pixel, the matrix material decomposition image of the entire image can be obtained.

[0036] Step 2: Using the base material decomposition image obtained in Step 1 as the decomposition label, perform supervised learning using the base material decomposition network.

[0037] S21. Construct a basic material decomposition network, the structure of which is shown in the diagram below. Figure 3As shown, the network mainly consists of two deep convolutional encoders and one deep convolutional decoder, with each encoder and decoder containing the same number of sub-encoders and sub-decoders. The network input is a subset of dual-energy CT images to be decomposed, and the output is the preliminary decomposition result of the input images. The sub-encoders in the network use the ConvNeXt network as the basic framework, mainly composed of convolutional layers, normalization layers, and nonlinear layers, used to extract multi-scale convolutional features from the input dual-energy CT images. As the number of network layers increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information, while deep features contain more global semantic information. To improve the model's performance and enable the network to focus on more important regions during the decomposition process, this invention also introduces an attention module in the encoder. This module takes the feature map generated by the encoder as input and outputs the importance of each channel in the feature space. The deep convolutional decoder in the network fuses and decodes the multi-scale convolutional features extracted by the encoder to obtain the preliminary decomposition result of the input image. In addition, based on the characteristics of the input dual-energy CT images, a co-learning module is also designed to fuse information from CT images of different energies. This module uses feature maps of different energies output by two convolutional layers to generate fusion weights and outputs the fused feature map.

[0038] S22. The matrix decomposition network constructed in S21 is trained and optimized using the multi-matrix decomposition images obtained in step one and the corresponding dual-energy CT images. First, the network parameters are initialized using random initialization. Then, the decomposition loss function L between the network's decomposition results on the input images and the corresponding labels is calculated. The decomposition loss function L is the sum of the mean squared error loss function and the multi-scale perception loss function. The expression for the mean squared error loss function is: Where x t (p) = (t = 1, ..., N, p = 1, ..., M) represents the decomposition coefficient of the t-th base material generated by the network for the p-th pixel. t (p)(t=1,...,N,p=1,...,M) represents the decomposition coefficient of the t-th base material in the label for the p-th pixel, where M represents the number of pixels and N represents the types of base materials decomposed. The expression for the multi-scale perceptual loss function is as follows: in Let xi represent the output at the s-th scale of the pre-trained network, and yi = 1, ..., N represent the result and label of the decomposed network, respectively. N represents the number of base material images, and S represents the number of scales. A stochastic gradient descent optimizer is used to optimize the network, with an initial learning rate set to 10. -2Furthermore, momentum update and weight decay mechanisms are introduced, and the loss function is optimized by adjusting the network weight values ​​through gradient backpropagation. The final output is the pre-trained decomposed network parameters.

[0039] Step 3: Based on the pre-trained network weights from Step 2, perform self-supervised training on the decomposition network using all dual-energy CT image data to improve the network's matrix decomposition performance.

[0040] The decomposition network was fine-tuned using all dual-energy CT image data. The network training method was as follows: Figure 4 As shown, a loss function is constructed based on the relationship between the base material image and the dual-energy CT image to enable the network to perform self-supervised learning. An edge-preserving term and a material sparsity term are introduced to optimize the decomposition network parameters. Specifically, the parameters of the base material decomposition network are first initialized using the pre-trained parameters obtained in step two. Then, the decomposition network is self-supervised trained based on the loss function L. The loss function L is derived from the self-supervised loss function Li. self Edge Preservation Function L edge With material sparsity function L sparsity It consists of three components. The expression for the self-supervised loss function is: Matrix A consists of the linear attenuation coefficients of the matrix material at different energies, x represents the solved matrix material decomposition image, and μ is the vector of linear attenuation coefficients of the tissue under test at different energies, which can be obtained from dual-energy CT images. This loss function requires that the dual-energy CT image reconstructed from the matrix material image generated by the network be as close as possible to the input dual-energy CT image, ensuring the accuracy of the matrix material image decomposed by the network, thus solving the problem of lacking ground truth labels in matrix material decomposition. The expression for the edge-preserving function is as follows: in X represents the edge of the image in the k-direction. i (i = 1, ..., N) represents the decomposition diagram of the i-th base substance, Y j (j = L, H) represents the input low-energy and high-energy CT images, N represents the amount of matrix material, and M represents the total number of image pixels. This loss function ensures that the matrix-decomposed image and the original dual-energy CT image share common anatomical edges. The expression for the material sparsity function is: Where X i (i = 1, ..., N) represents the decomposition image of the i-th matrix material, where N represents the number of matrix materials and M represents the total number of pixels in the image. Minimizing this function makes the matrix material composition sparse for each pixel. The initial learning rate is set to 10. -4 Furthermore, momentum update and weight decay mechanisms are introduced, and the loss function is optimized by adjusting network weights through gradient backpropagation. The final output is the accurate matrix decomposition result of the input dual-energy CT image.

[0041] To visually demonstrate the effects of the present invention, Figures 5a-5n This invention demonstrates the visualization results of matrix material decomposition on dual-energy CT image data, with the network decomposing the matrix into four types of matrix materials. Figure 5a b represents a pair of dual-energy CT images, which serve as inputs to the overall method. Figures 5c-5f The images of the base materials obtained from step one decomposition represent the distribution of the four base materials from left to right, serving as labels for subsequent supervised training. Figure 5g-5j The results generated after supervised pre-training of the network in step two show that the network's ability to process labels... Figures 5c-5f After learning, input Figure 5a b underwent relatively accurate decomposition of the base material. Figure 5k-5n The output of the network after self-supervised training in step three shows that, through further fine-tuning and optimization, the network generated images of four base materials with clear anatomical structures and accurate distribution. Figures 6a-6q This demonstrates the results of the present invention when decomposing dual-energy CT image data into five basic substances, wherein... Figure 6a , 6b To input dual-energy CT images, Figure 6c-6g Images of the five basic substances obtained from step one's decomposition. Figure 6h-6l These are the distribution results of the five basic materials generated after step two pre-training. Figure 6m-6q This is the final output of step three. It can be seen that the method of this invention has good performance, producing images that conform to prior knowledge, with clear edges, low noise, and the ability to accurately decompose any type and quantity of matrix material. This invention designs a corresponding network structure based on the characteristics of the physical process of matrix material decomposition and constructs a loss function and training method to address the shortcomings of multi-matrix decomposition. Since multi-matrix decomposition is ill-posed, this invention first uses a traditional method to generate labels, and then performs supervised pre-training on the network based on the labels to accelerate network convergence. Furthermore, because matrix material decomposition lacks ground truth values, this invention designs a self-supervised training method based on the loss function. Experimental results show that this invention can accurately decompose multi-matrix materials in dual-energy CT, and can be closely integrated with applications in material composition analysis, virtual imaging, and other related fields, possessing broad market prospects and application value.

Claims

1. A multi-material-based matter decomposition method based on deep learning, characterized in that: The specific steps are as follows: Step 1: Decompose a portion of the dual-energy CT image dataset pixel by pixel according to the priority list of ternary matrix groups to obtain a series of corresponding multi-matrix decomposed images; Step 2: Using the base material decomposition image obtained in Step 1 as the decomposition label, supervised learning is performed using the base material decomposition network. The base material decomposition network is constructed, with dual-energy CT images as input and the base material decomposition image obtained in Step 1 as the label for pre-training. The decomposition network encodes depth features into the network input using an encoder, then decodes the depth features using a decoder, and introduces a common learning module to fuse information from CT images of different energies, obtaining a preliminary decomposition of the input dual-energy CT image. Based on the mean squared error loss function and multi-scale perception loss function between the network decomposition result and the label, the decomposition network parameters are optimized, and finally, the pre-trained decomposition network parameters are output. Step 3: Use all dual-energy CT image data to perform self-supervised training on the decomposition network to improve the network's matrix decomposition effect; use all dual-energy CT image data to fine-tune the decomposition network; construct a loss function based on the relationship between the matrix image and the dual-energy CT image to enable the network to perform self-supervised learning, and introduce edge preservation term and material sparsity term to optimize the decomposition network parameters. Specifically, in step two: S21. Construct a matrix decomposition network consisting of two deep convolutional encoders and one deep convolutional decoder. The input is a subset of dual-energy CT images to be decomposed, and the output is the preliminary decomposition result of the input images. The two deep convolutional encoders use the ConvNeXt network as their basic framework to extract multi-scale convolutional features from the input dual-energy CT images. As the network depth increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information, while deep features contain more global semantic information. Simultaneously, an attention module is introduced into the encoder. This module takes the feature map generated by the encoder as input and outputs the importance of each channel in the feature space. The deep convolutional decoder fuses and decodes the multi-scale convolutional features extracted by the encoder to obtain the preliminary decomposition result of the input images. A co-learning module is also designed to fuse information from CT images of different energies. This module uses the feature maps from the two convolutionals belonging to different energies to generate fusion weights and outputs the fused feature map. S22. The matrix decomposition network constructed in S21 is trained and optimized using the multi-matrix decomposition images obtained in step one and the corresponding dual-energy CT images. First, the network parameters of the matrix decomposition network are initialized, and the initialization parameters are obtained by random initialization. Then, the decomposition loss function between the network's decomposition results of the input images and the corresponding labels is calculated. Decompose the loss function The sum of the mean squared error loss function and the multi-scale sensing loss function is given by: ,in Indicates for the first The nth pixel, generated by the network The decomposition coefficient of the seed substrate. Indicates for the first The pixel in the label The decomposition coefficient of the seed substrate. Indicates the number of pixels. The base material types representing the decomposition are shown in the following expression: [Expression of the multi-scale sensing loss function]. ,in The first one representing the pre-trained network The output corresponding to each scale and These represent the results and labels of the decomposed network, respectively. Indicates the number of base material images. The scale represents the number of scales; the network is optimized using a stochastic gradient descent optimizer, with an initial learning rate of 10. -2 Furthermore, a momentum update and weight decay mechanism is introduced, and the network weight values ​​are adjusted through gradient backpropagation to optimize the loss function; the final output is the pre-trained decomposed network parameters.

2. The method for multi-material-based matter decomposition based on deep learning according to claim 1, characterized in that: In step one, a portion of data is randomly selected from the dual-energy CT dataset, and the number and types of matrix materials to be decomposed are determined. A priority list of ternary matrix material groups is constructed based on the differences in the attenuation characteristics of the matrix materials. Then, the matrix materials are decomposed sequentially for each pixel of the CT image according to the order of the ternary matrix material groups, and finally, multiple matrix material images are output.

3. A method for multi-material-based matter decomposition based on deep learning according to claim 1 or 2, characterized in that: In step one, specifically: S11. Construct a list of ternary matrix groups; select the number and types of matrix materials to be decomposed, list all possible ternary matrix groups, and then find the linear decay coefficients of the selected matrix materials at different energies from the NIST library; for each ternary matrix group, calculate the difference in decay characteristics between the matrix materials in the group; arrange the ternary matrix groups according to the difference in decay characteristics, and output a list of ternary matrix groups with priority. S12. Perform multi-matrix decomposition; decompose each pixel of the CT image sequentially according to the ternary matrix group arrangement. Stop solving when there is a solution with physical meaning. This solution is the result of matrix decomposition for that pixel. If it is impossible to obtain a solution with all values ​​in the range of 0-1 using all ternary matrix groups, select the solution whose sum of absolute values ​​is closest to 1. After statistically analyzing the matrix decomposition of each pixel, obtain the matrix decomposition image of the entire image.

4. The method for multi-material-based matter decomposition based on deep learning according to claim 1, characterized in that: In step three, specifically: First, the parameters of the matrix decomposition network are initialized using the pre-trained parameters obtained in step two. Then, based on the loss function... Self-supervised training of the decomposed network; loss function By self-supervised loss function Edge Preservation Function With material sparsity function It consists of three components; the expression for the self-supervised loss function is: , where the matrix It is composed of the linear decay coefficients of the base material at different energies. This represents the decomposition image of the base material obtained from the solution. It is the linear attenuation coefficient vector of the tissue under test at different energies, which is obtained from dual-energy CT images. This loss function requires that the dual-energy CT image reconstructed from the matrix image generated by the network should be as close as possible to the input dual-energy CT image, thus ensuring the accuracy of the matrix image decomposed by the network and solving the problem of lack of true value labels in matrix decomposition.

5. The method for decomposing multi-material-based substances based on deep learning according to claim 4, characterized in that: The expression for the edge-preserving function is: ,in, The image representing the solution is in The edge of the direction, Representing the Decomposition image of seed substrate. Representing the input low-energy and high-energy CT images, Indicates the amount of base material. This represents the total number of pixels in the image; this loss function ensures that the base material decomposed image and the original dual-energy CT image share common anatomical edges; the expression for the material sparsity function is: ,in Representing the Decomposition image of seed substrate. Indicates the amount of base material. This represents the total number of pixels in the image; minimizing this function makes the matrix components corresponding to each pixel sparse; the initial learning rate is set to 10. -4 Furthermore, a momentum update and weight decay mechanism is introduced, and the loss function is optimized by adjusting the network weights through gradient backpropagation; the final output is the accurate matrix decomposition result of the input dual-energy CT image.

Images