Image processing method and system and storage medium

Abstract

The embodiment of the invention provides an image processing method and system and a storage medium, and the method comprises the steps: carrying out the energy spectrum calibration through a reference material, and obtaining an energy spectrum response function of each pixel; based on the energy spectrum response function and the energy spectrum measurement value of the target object, obtaining a reference material decomposition result image corresponding to the target object through a decomposition model; and based on the reference material decomposition result image, obtaining a target material decomposition result image in the target object.

Description

Technical Field

[0001] This specification relates to the field of medical imaging, and particularly to an image processing method, system, and storage medium. Background Art

[0002] Computed Tomography (CT) is a widely used medical imaging technology that can be used to distinguish different materials. Among them, Photon-Counting CT (PCCT) has greater potential due to its unique energy spectrum measurement mechanism. PCCT uses Photon-Counting Detectors (PCDs) for energy spectrum measurement, and the energy of each incident X-ray photon is measured individually and accumulated into the total count of the corresponding energy channel. The usual material decomposition algorithm uses numerical polynomials to fit the line integral of the corresponding material decomposition from the direct measurement values. However, due to the non-uniformity and instability between detector pixels, spectral calibration needs to be performed frequently, and special processing is required to adapt to different spectral responses. Spectral CT image reconstruction and material decomposition based on a model can be performed, where the energy spectrum model can adapt to variable energy spectrum responses, but the current model-based methods can only be applied to specific systems and usually use computationally intensive iterative algorithms, which are difficult to apply clinically.

[0003] Therefore, it is desirable to provide an image processing method, system, and storage medium to improve the adaptability of the energy spectrum model to different energy spectrum systems, thereby improving the efficiency and quality of material decomposition. Summary of the Invention

[0004] One embodiment of this specification provides an image processing method. The method includes: performing spectral calibration using a reference material to obtain the spectral response function of each pixel; based on the spectral response function and the spectral measurement value of the target object, obtaining the reference material decomposition result image corresponding to the target object through a decomposition model; and based on the reference material decomposition result image, obtaining the target material decomposition result image in the target object.

[0005] In some embodiments, the first raw data of the spectral scan of the reference material can be obtained; the first calibration data is determined based on the first raw data; and the spectral response function is obtained based on the first calibration data and the energy spectrum model.

[0006] In some embodiments, the energy spectrum model can be related to at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold.

[0007] In some embodiments, the spectral response function, the spectral measurement value, and the negative logarithm of the spectral measurement value may be input into the decomposition model to obtain the reference material decomposition result image.

[0008] In some embodiments, image domain material decomposition may be performed based on the reference material decomposition result image to obtain the target material decomposition result image.

[0009] In some embodiments, the decomposition model may include a multi-layer perceptron network.

[0010] In some embodiments, the decomposition model may be trained through the following steps: based on the spectral response function and the spectral model, multiple training samples are obtained through data augmentation; the decomposition model is trained based on the multiple training samples.

[0011] In some embodiments, each of the multiple training samples may include a first input parameter, a second input parameter, and a third input parameter. The obtaining of multiple training samples through data augmentation may include: randomly generating an integration path as the first input parameter; randomly selecting one from all the spectral response functions as the second input parameter; based on the first input parameter and the second input parameter, simulating the spectral measurement value through the spectral model as the third input parameter.

[0012] One embodiment of this specification provides an image processing system, including a spectral response acquisition module, a first result acquisition module, and a second result acquisition module; the spectral response acquisition module is configured to perform spectral calibration using a reference material to obtain the spectral response function of each pixel; the first result acquisition module is configured to obtain the reference material decomposition result image corresponding to the target object through a decomposition model based on the spectral response function and the spectral measurement value of the target object; the second result acquisition module is configured to obtain the target material decomposition result image in the target object based on the reference material decomposition result image.

[0013] One embodiment of this specification provides a computer-readable storage medium, where the storage medium stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the image processing method. Description of the Drawings

[0014] This specification will be further described by way of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0015] Figure 1Schematic diagram of the application scenario of the image processing system according to some embodiments of this specification;

[0016] Figure 2 Schematic diagram of the image processing system according to some embodiments of this specification;

[0017] Figure 3 Exemplary flowchart of the image processing method according to some embodiments of this specification;

[0018] Figure 4 Schematic diagram of the image processing method according to some embodiments of this specification;

[0019] Figure 5 Schematic diagram of the image processing method according to some embodiments of this specification;

[0020] Figure 6 Schematic diagram of the image processing method according to some embodiments of this specification;

[0021] Figure 7 Schematic diagram of the material decomposition result of simulated energy spectrum measurement using a single energy spectrum response according to some embodiments of this specification;

[0022] Figure 8 Schematic diagram of the 70 / 120 keV virtual monoenergetic images (VMIs) of the water phantom according to some embodiments of this specification;

[0023] Figure 9 Schematic diagram of the VMIs of the Gammex phantom and the distribution maps of iodine / calcium according to some embodiments of this specification;

[0024] Figure 10 Schematic diagram of the 70 keV VMI of the Gammex phantom obtained using the polynomial fitting method according to some embodiments of this specification. Detailed implementation manners

[0025] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0026] It should be understood that the "system", "device", "unit" and / or "module" used herein is a way to distinguish different components, elements, parts, sections or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0027] As shown in this specification and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the previous or subsequent operations are not necessarily executed precisely in sequence. On the contrary, the steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0029] Figure 1 It is a schematic diagram of the application scenario of the image processing system shown in some embodiments of this specification.

[0030] In this specification, the image processing system 100 is simply referred to as the system 100. As Figure 1 shown, in some embodiments, the system 100 may include a medical imaging device 110, a first computing device 120, a second computing device 130, a user terminal 140, a storage device 150, and a network 160.

[0031] The medical imaging device 110 may refer to a device that uses different media to reproduce the internal structure of a target object (such as the human body) as an image. In some embodiments, the medical imaging device 110 may be any device that can distinguish different materials. For example, spectral CT such as PCCT, monoenergetic CT, etc. The medical imaging device 110 provided above is only for illustrative purposes and not for limiting its scope. In some embodiments, the medical imaging device 110 may perform spectral measurements on a target object containing multiple materials (such as a water phantom, a Gammex phantom, etc.) and send the spectral measurement data (such as photon count values, etc.) to other components of the system 100 (such as the first computing device 120, the second computing device 130, the storage device 150). In some embodiments, the medical imaging device 110 may exchange data and / or information with other components in the system 100 through the network 160.

[0032] The first computing device 120 and the second computing device 130 are systems with computing and processing capabilities, which may include various computers, such as servers and personal computers, or may also be computing platforms composed of multiple computers connected in various structures. In some embodiments, the first computing device 120 and the second computing device 130 may be the same device or different devices.

[0033] One or more sub-processing devices (e.g., single-core processing devices or multi-core multi-chip processing devices) may be included in the first computing device 120 and the second computing device 130, and the processing device may execute program instructions. By way of example only, the processing device may include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), or other types of integrated circuits.

[0034] The first computing device 120 may process information and data related to energy spectrum measurement values. In some embodiments, the first computing device 120 may execute an image processing method as shown in some embodiments of this specification to obtain at least one processing result, for example, an energy spectrum response function, a material decomposition result image, etc. In some embodiments, the first computing device 120 may include a decomposition model based on machine learning (e.g., a multi-layer perceptron (MLP) (also known as a multi-layer perceptron, multi-layer perceptron, etc.)), and the first computing device 120 may obtain a material decomposition result image through the decomposition model. In some embodiments, the first computing device 120 may obtain a trained decomposition model from the second computing device 130. In some embodiments, the first computing device 120 may exchange information and data through the network 160 and / or other components in the system 100 (e.g., the medical imaging device 110, the second computing device 130, the user terminal 140, the storage device 150). In some embodiments, the first computing device 120 may be directly connected to the second computing device 130 and exchange information and / or data.

[0035] The second computing device 130 can be used for model training. In some embodiments, the second computing device 130 can execute the training method of the decomposition model as shown in some embodiments of this specification to obtain a trained decomposition model. In some embodiments, the second computing device 130 can generate training samples through Data Augmentation for training the decomposition model. In some embodiments, the second computing device 130 can obtain spectral measurement data from the medical imaging device 110 as the training data of the model.

[0036] The user terminal 140 can receive and / or display the processing results of medical images. In some embodiments, the user terminal 140 can receive spectral measurement data from the first computing device 120 and obtain the material decomposition result in the target object based on this spectral measurement data. In some embodiments, the user terminal 140 can instruct the first computing device 120 to execute the image processing method as shown in some embodiments of this specification. In some embodiments, the user terminal 140 can control the medical imaging device 110 to obtain the spectral measurement data of the target object. In some embodiments, the user terminal 140 can be a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, a desktop computer, or any other device with input and / or output functions, or any combination thereof.

[0037] The storage device 150 can store the data or information generated by other devices. In some embodiments, the storage device 150 can store the spectral measurement data collected by the medical imaging device 110. In some embodiments, the storage device 150 can store the data and / or information processed by the first computing device 120 and / or the second computing device 130, such as a trained decomposition model, a spectral response function, a material decomposition result image, etc. The storage device 150 can include one or more storage components, and each storage component can be an independent device or a part of other devices. The storage device can be local or implemented through the cloud.

[0038] The network 160 can connect the components of the system and / or connect the system to the external resource part. The network 160 enables communication between the components and between the system and other parts outside the system, facilitating the exchange of data and / or information. In some embodiments, one or more components in the system 100 (such as the medical imaging device 110, the first computing device 120, the second computing device 130, the user terminal 140, the storage device 150) can send data and / or information to other components through the network 160. In some embodiments, the network 160 can be any one or more of a wired network or a wireless network.

[0039] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various changes and modifications can be made under the guidance of the content of this specification. The features, structures, methods, and other features of the exemplary embodiments described in this specification can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, the first computing device 120 and / or the second computing device 130 can be based on a cloud computing platform, such as a public cloud, a private cloud, a community cloud, and a hybrid cloud, etc. However, these changes and modifications will not depart from the scope of this specification.

[0040] Figure 2 is a schematic diagram of an image processing system according to some embodiments of this specification.

[0041] As Figure 2 shown, the image processing system 200 can include an energy spectrum response acquisition module 210, a first result acquisition module 220, and a second result acquisition module 230. In some embodiments, the energy spectrum response acquisition module 210, the first result acquisition module 220, and the second result acquisition module 230 can be implemented by the first computing device 120.

[0042] In some embodiments, the energy spectrum response acquisition module 210 can be used to perform energy spectrum calibration using a reference material to obtain the energy spectrum response function of each pixel.

[0043] In some embodiments, the energy spectrum response acquisition module 210 can acquire the first raw data of the energy spectrum scan of the reference material; determine the first calibration data (CalibrationData) based on the first raw data; and obtain the energy spectrum response function based on the first calibration data and the energy spectrum model.

[0044] In some embodiments, the energy spectrum model can be related to at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector (PCD) energy channel threshold, etc.

[0045] In some embodiments, the first result acquisition module 220 can be used to obtain the reference material decomposition result image corresponding to the target object through a decomposition model based on the energy spectrum response function and the energy spectrum measurement value of the target object.

[0046] In some embodiments, the decomposition model can include various machine learning models such as a multi-layer perceptron (MLP).

[0047] In some embodiments, the image processing system 200 may further include a model training module 240. The model training module 240 may be used to train the decomposition model through the following steps: obtaining a plurality of training samples through data augmentation based on the energy spectrum response function and the energy spectrum model; and training the decomposition model based on the plurality of training samples. In some embodiments, the model training module 240 may be implemented by the second computing device 130.

[0048] In some embodiments, each of the plurality of training samples for training the decomposition model may include a first input parameter, a second input parameter, and a third input parameter. The model training module 240 may randomly generate an integration path as the first input parameter; randomly select one from all the energy spectrum response functions as the second input parameter; and simulate an energy spectrum measurement value as the third input parameter based on the first input parameter and the second input parameter through the energy spectrum model.

[0049] In some embodiments, the second result acquisition module 230 may be used to obtain the decomposition result image of the target material in the target object based on the reference material decomposition result image.

[0050] In some embodiments, the second result acquisition module 230 may perform material decomposition in the image domain based on the reference material decomposition result image to obtain the decomposition result image of the target material.

[0051] Figure 3 is an exemplary flowchart of the image processing method shown in some embodiments of the present specification.

[0052] As Figure 3 shown, the process 300 includes the following steps. In some embodiments, the process 300 may be executed by the first computing device 120.

[0053] Step 310, performing energy spectrum calibration using a reference material to obtain the energy spectrum response function of each pixel. In some embodiments, step 310 may be executed by the energy spectrum response acquisition module 210.

[0054] The target object is an object scanned by a medical imaging device. For example, a human body, a phantom, etc. In some embodiments, the target object may include at least one element and / or material. For example, solids / liquids of various metal elements / non-metal elements, various compounds, etc. In some embodiments, the target object may be a Gammex phantom including at least one solid material.

[0055] A reference material refers to a material used to obtain parameters of a medical imaging device, which may include at least two materials, for example, aluminium (Al) and polymethyl methacrylate (PMMA) plates. In some embodiments, the parameters of the medical imaging device may include the energy spectrum response function of each pixel in its detector. In some embodiments, the reference material may be included in the target object.

[0056] A target material refers to a material whose properties are to be determined, which may include one or more materials, for example, one or any combination of water, iodine (I), calcium (Ca), etc. In some embodiments, the target material may be included in the target object.

[0057] In some embodiments, the first computing device 120 may perform energy spectrum calibration on a medical imaging device (e.g., medical imaging device 110) using the reference material to obtain the energy spectrum response function of each pixel in the detector of the medical imaging device. In the following of this specification, the medical imaging device is described as a PCCT, and its detector is a PCD.

[0058] In some embodiments, the first computing device 120 may use the medical imaging device to perform an energy spectrum scan on the reference material to obtain the raw data of the energy spectrum scan of the reference material as the first raw data. Among them, the first raw data may include energy spectrum measurement values, for example, photon count values in the PCCT. Since the energy spectrum measurement of the detector is based on the pixel units (simply referred to as pixels) of the detector, the energy spectrum measurement values also correspond to each pixel.

[0059] In some embodiments, during energy spectrum calibration, the first computing device 120 may determine the first calibration data based on the first raw data. Specifically, the first computing device 120 may use two or more known materials (e.g., Al and PMMA plates) as reference materials, and use combinations of different thicknesses of these materials to generate energy spectrum measurement values of the corresponding material line integrals, and use the line integrals of these materials and the corresponding energy spectrum measurement values as the first calibration data. In some embodiments, when using the reference material to obtain the first calibration data, the energy spectrum scan may also be referred to as calibration scans.

[0060] In some embodiments, the first computing device 120 may also obtain the first raw data and / or the first calibration data by other means, for example, obtaining from a storage device (e.g., storage device 150).

[0061] In some embodiments, the first computing device 120 may obtain the energy spectrum response function based on the first calibration data and the energy spectrum model. Among them, each pixel has a corresponding energy spectrum response function.

[0062] An energy spectrum model is a model used to represent the relationship between energy spectrum measurement values, materials, and medical imaging devices. In some embodiments, the energy spectrum model can be represented by the following formula: y i = f(x1 i , x2 i , x2 i ,..., xn i ) (1) where i represents the pixel number; y i represents the energy spectrum measurement value of the i-th pixel in the detector (e.g., PCD) of a medical imaging device (e.g., PCCT); n≥2, and f(x1 i , x2 i , x2 i ,..., xn i ) represents that y i is a function of multiple parameters x1 i , x2 i , x2 i ,..., xn i . In some embodiments, the parameters x1 i , x2 i , x2 i ,..., xn i can include parameters related to materials and parameters related to medical imaging devices. Among them, the parameters related to materials can include material line integrals, etc., and the parameters related to medical devices can include energy spectrum response functions, tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.

[0063] In some embodiments, the energy spectrum model can be related to the energy spectrum response function and the material line integral. Based on formula (1), the energy spectrum measurement value can be expressed as a function of two parameters, the energy spectrum response function and the material line integral. The energy spectrum model can be represented by the following formula: y i = S i exp{-μL i} (2) where S i represents the energy spectrum response function of the i-th pixel in the detector (e.g., PCD) of a medical imaging device (e.g., PCCT); exp represents the exponential function with the natural constant e as the base; L i and μ both correspond to a certain material, L i represents the material line integral of the i-th pixel, and μ represents the energy-related attenuation coefficient of the material. When the material line integrals of each pixel are equal (e.g., the material thickness is uniform), the material line integral can be represented by L.

[0064] In some embodiments, the energy spectrum model may also be related to at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.

[0065] In some embodiments, based on formula (1), the energy spectrum measurement value can be expressed as a function of three parameters: the energy spectrum response function, the material line integral, and the tube current. The energy spectrum model can be expressed by the following formula: Where S p，i represents the energy spectrum response function; I represents the tube current; α is a function of (I, L, S p，i ); exp represents the exponential function with the natural constant e as the base; τ is the counting dead time of the PCD; the meanings of the remaining parameters are the same as those in formula (1). The energy spectrum model in formula (3) includes a measure of the dependence on the incident count rate and can avoid the influence of the pulse pile-up effect.

[0066] In some embodiments, the first computing device 120 may obtain the energy spectrum response function based on the first calibration data and the energy spectrum model using various methods (such as the Maximum Likelihood Expectation Maximization (MLEM) algorithm, etc.). In some embodiments, based on formula (2), the energy spectrum response function can be obtained through the following formula: Where represents the maximum likelihood estimate of the energy spectrum response function S i of the i-th pixel; arg min f(x) represents a subset of the domain, and any element in this subset can make the function f(x) take the minimum value; y i (L) represents the energy spectrum measurement value of the i-th pixel filtered by the material with a known line integral; represents the negative logarithm of the energy spectrum measurement value, which can be represented by Ly, that is In the following of this specification, Ly will be used to represent the negative logarithm of the energy spectrum measurement value.

[0067] In some embodiments of this specification, by integrating the parameter information of the medical imaging system (such as the energy spectrum response function, tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.) into a unified energy spectrum model, it is possible to comprehensively utilize the calibration data obtained under different scanning protocols and use the unified energy spectrum model to parameterize different scanning protocols to calibrate the PCCT system, thus overcoming the problem that traditional calibration methods can only calibrate each specific scanning protocol separately.

[0068] Step 320: Based on the energy spectrum response function and the energy spectrum measurement values of the target object, obtain the reference material decomposition result image corresponding to the target object through the decomposition model. In some embodiments, step 320 may be executed by the first result acquisition module 220.

[0069] The reference material decomposition result image refers to an image based on the material decomposition result (also known as the substance decomposition result) of the reference material. For example, a 70 / 120 keV virtual monoenergetic image (VMIs) reconstructed based on the decomposition result of the reference material, etc. In some embodiments, the first computing device 120 may obtain the reference material decomposition result image corresponding to the target object based on the energy spectrum response function and the energy spectrum measurement values of the target object.

[0070] The decomposition model is a model used to obtain the substance decomposition result of the material. Among them, the substance decomposition result can be represented by an image or parameter values, etc. For example, the substance decomposition result can be represented in the form of material line integral, virtual monoenergetic image (VMIs), substance concentration map, etc.

[0071] In some embodiments, the input of the decomposition model may include energy spectrum measurement values and various parameters related to the medical imaging device (for example, energy spectrum response function, tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.), and the output may be parameters related to the material (for example, material line integral).

[0072] In some embodiments, the decomposition model may include various machine learning models, such as a multi-layer perceptron network (MLP), a neural network model (Neural Network, NN), etc. In some embodiments, the decomposition model may be a multi-layer perceptron network.

[0073] Take Figure 5 and 6 as an example to illustrate the structure of the decomposition model. As Figure 5 and 6 shown, the MLP decomposition networks 510 and 610 are decomposition models, both of which are three-layer perceptron networks. Among them, the first layer and the last layer may each contain m (for example, 64, 128, 256, etc.) nodes, and the middle layer may contain 2m nodes; a Leaky ReLU (an activation function in deep learning) activation layer is used for full connection (Fully Connected, FC) between each layer, where m is an integer.

[0074] In some embodiments, the decomposition model can be obtained through training. Regarding how to train the decomposition model and more content related to the decomposition model, reference can be made to the relevant description in Figure 4 , and details will not be elaborated here.

[0075] In some embodiments, the spectral measurement value of the target object can be obtained in various ways. For example, by performing spectral scanning on the target object using a medical imaging device, obtaining from a storage device, etc. Among them, the target object may include target materials. For example, the target object is a Gammex head phantom including at least two unknown materials. In some embodiments, the first computing device 120 can perform spectral scanning on the target object containing the target material and the reference material together, so as to obtain the spectral measurement value of the target object. For example, during spectral scanning, an object containing a reference material (such as Al and PMMA plates) can be stacked with the target object (such as a Gammex head phantom with solid water, different concentrations of iodine and calcium inserts).

[0076] In some embodiments, the first computing device 120 can input the spectral response function obtained in step 310, the spectral measurement value of the target object, and the negative logarithm of the spectral measurement value into the decomposition model, and obtain the output material line integral as the material decomposition result. For example, Figure 5 in the three-layer MLP decomposition network 510, the input is the known spectral response function combination S, the spectral measurement value y, and the negative logarithm Ly of the spectral measurement value, and the output is the estimated decomposition result which is the maximum likelihood estimate of the material line integral L. Another example, Figure 6 in the three-layer MLP decomposition network 610, based on the known tube current I, the input is the known spectral response function combination S, the spectral measurement value y, and the negative logarithm Ly of the spectral measurement value, and the output is the estimated decomposition result

[0077] In some embodiments, the first computing device 120 can obtain the projection of the virtual monoenergetic image (VMIs) at 70 / 120 keV based on the obtained material line integral. Taking the reference materials including aluminum and PMMA as an example, based on formula (2), the projection of the virtual monoenergetic image (VMIs) at a specified energy level (such as 70 / 120 keV) can be shown as the following formula: l VMI (E)=μ Al (E)l Al +μ PMMA (E)l PMMA (5) where l VMI (E) represents the projection value of the virtual monoenergetic image at energy level E; l Al represents the aluminum line integral, μ Al (E) represents the attenuation coefficient of aluminum at energy level E; l PMMA represents the PMMA line integral, μ PMMA (E) represents the attenuation coefficient of PMMA at energy level E.

[0078] In some embodiments, the first computing device 120 may reconstruct the virtual monoenergetic image based on the projection of the virtual monoenergetic image at a specified energy level, and use the reconstructed virtual monoenergetic image as the reference material decomposition result image corresponding to the target object. For example, Figure 8 Shown are the virtual monoenergetic images (VMIs) of a water phantom at 70 keV and 120 keV. Among them, the left image is the virtual monoenergetic image (VMI) of the water phantom at 70 keV, and the right image is the virtual monoenergetic image (VMI) of the water phantom at 120 keV. Another example, Figure 9 In, subgraphs (a) and (b) are respectively the virtual monoenergetic images of a Gammex phantom at 70 keV and 120 keV. Figure 8 and 9 The virtual monoenergetic images in are all reference material decomposition result images obtained by reconstructing the image based on the projection of the virtual monoenergetic image at a specified energy level.

[0079] In some embodiments, various methods can be used for reconstruction, such as the Filtered Backprojection (FBP) method, machine learning model reconstruction, etc. In some embodiments, the parameters involved in the reconstruction may include slice thickness, in-plane voxel size, etc. For example, the slice thickness in the reconstruction can be set to 5 mm, and the in-plane voxel size can be set to 0.47 mm.

[0080] Compared with the virtual monoenergetic images obtained by other methods (such as the polynomial fitting method, etc.), the image quality of the virtual monoenergetic images obtained in the above manner has been significantly improved. For example, as Figure 10 shown, the virtual monoenergetic image of a Gammex phantom at 70 keV, which is obtained by the polynomial fitting method. Compared with Figure 9 subgraphs (a) and (b) in, it can be seen that Figure 9 the virtual monoenergetic image in has good uniformity, while Figure 10 the virtual monoenergetic image obtained by the polynomial fitting method in has a large number of circular artifacts and poor uniformity.

[0081] Step 330, based on the reference material decomposition result image, obtain the target material decomposition result image in the target object. In some embodiments, step 330 may be executed by the second result acquisition module 230.

[0082] The target material decomposition image is an image that can represent the material decomposition result of the target material. For example, a substance concentration map, etc. In some embodiments, the first computing device 120 may perform material decomposition in the image domain based on the acquired reference material decomposition result image, and use the substance concentration map obtained after material decomposition as the target material decomposition result image in the target object.

[0083] Taking the target object as a Gammex head phantom containing solid water, iodine with different concentrations, and calcium inserts as an example, how to perform material decomposition in the image domain will be described below. The first computing device 120 may perform material decomposition in the image domain on this phantom based on virtual monoenergetic images (VMIs) at 70 / 120 keV. Assuming that all voxels have the same solid water concentration, the following formula holds:

[0084] where E1 and E2 represent two different energy levels (for example, 70 keV and 120 keV); μ VMI (E) represents the virtual monoenergetic image at energy level E; μ W (E), μ J (E), μ Ca (E) represent the attenuation coefficients of solid water, iodine, and calcium at energy level E respectively; ρ W 、ρ I 、ρ Ca represent the solid water, iodine, and calcium concentration maps obtained after material decomposition in the image domain respectively. In some embodiments, it may be assumed that ρ W = 1 g / mL to solve the dual-energy three-base decomposition problem. In some embodiments, the attenuation coefficients μ W (E), μ I (E), μ Ga (E) of solid water / iodine / calcium can be calibrated through Gammex inserts.

[0085] Figure 9 is a schematic diagram of the VMIs of the Gammex phantom and the distribution maps of iodine / calcium shown in some embodiments of this specification. Figure 9 In, subgraphs (c) and (d) are the distribution maps of iodine (Iodine) and calcium (Calcium) in the Gammex phantom obtained based on the virtual monoenergetic images of subgraphs (a) and (b) respectively. Among them, the dots in subgraphs (c) and (d) represent the inserts of iodine and calcium in the Gammex phantom, and the higher the brightness, the higher the concentration of the corresponding element.

[0086] In some embodiments, the first computing device 120 may use an image to simultaneously represent the material decomposition results of the reference material and the target material. For example, Figure 7The figure shows the results of material decomposition for simulated spectral measurements using a single spectral response. Among them, subfigure (a) shows the results of training the network with a single energy spectrum, while subfigure (b) shows the results of training the network with multiple energy spectra. The test energy spectrum was not used in either single-spectrum or multi-spectrum training. The points within region 710 represent the material decomposition results of the reference material (PMMA), and the points within region 720 represent the material decomposition results of the target material. It can be seen that the points in the two regions in subfigure (a) do not coincide, while the points in the two regions in subfigure (b) basically coincide, indicating that the single-energy-spectrum training network cannot be applied to different energy spectra, while the multi-energy-spectrum training network can obtain highly accurate approximate material decomposition results.

[0087] In some embodiments of the present specification, through an energy spectrum model that includes the energy spectrum parameters (e.g., energy spectrum response function) of a medical imaging device and integrating it into a machine learning model, the material decomposition of substances is performed through the machine learning model, which can adapt to different energy spectrum responses and provide accurate material decomposition results for various energy spectrum CT systems; by including other parameter information of the medical imaging device (e.g., at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.) in the energy spectrum model, the machine learning model can overcome the influence of other adverse factors, thereby further improving the quality of material decomposition. For example, by adding tube current, the energy spectrum model includes a measure of the dependence on the incident count rate to address the adverse effects of pulse pile-up effects.

[0088] Figure 4 It is a schematic diagram of an image processing method shown in some embodiments of the present specification.

[0089] In some embodiments, the first computing device 120 can implement the image processing method shown in process 300 by executing some steps in process 400 to obtain the decomposition result of the target material of the target object based on the decomposition model. In some embodiments, the second computing device 130 can obtain the decomposition model through training by executing some steps in process 400.

[0090] As Figure 4 shown, the first computing device 120 can collect data on the target object 410 through a medical imaging device to obtain the photon count value 440 as the energy spectrum measurement value. Among them, the target object 410 may include a reference material 411. The first computing device 120 can perform energy spectrum calibration on the reference material 411 and obtain the energy spectrum response function 430 based on the energy spectrum calibration result and the energy spectrum model 420. For example, as Figure 5 shown, the energy spectrum calibration process may include: performing a calibration scan on the reference material through a medical imaging device to obtain calibration data (y i , L i ), where i represents the pixel number, yi represents the photon count value of the i-th pixel, L i represents the material integration path of the i-th pixel (also known as the integration path); the energy spectrum response (i.e., the energy spectrum response function) Si is obtained by the MLEM iterative reconstruction method based on calibration data. For another example, as Figure 6 shown, the energy spectrum calibration process may include performing calibration scans on a reference material at different tube currents by a medical imaging device to obtain calibration data (I, y i , L i ), where I represents the tube current, and y i , L i , and the meaning of Si is the same as that in Figure 5 . For more content on how to obtain the energy spectrum response function, reference can be made to the relevant description in step 310, which will not be elaborated here.

[0091] After obtaining the energy spectrum response function 430, the first computing device 120 can input the photon count value 440 and the energy spectrum response function 430 into the decomposition model 460 to obtain the output material decomposition result 470; based on the material decomposition result 470, image reconstruction is performed to obtain the virtual monoenergetic image 480 as the reference material decomposition result image corresponding to the target object 410; image domain material decomposition is performed on the virtual monoenergetic image 480 to obtain the material decomposition concentration map 490 as the target material decomposition result image in the target object 410. In some embodiments, on the basis of inputting the photon count value 440 and the energy spectrum response function 430 into the decomposition model 460, the negative logarithm of the photon count value 440 can also be input into the decomposition model 460 as an input parameter. In some embodiments, the input parameters of the decomposition model 460 may also include others, such as at least one of the tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc. For more content on how to obtain the reference material decomposition result image and the target material decomposition result image, reference can be made to the relevant descriptions in steps 320 and 330, which will not be elaborated here.

[0092] Hereinafter, all are based on Figure 4The medical imaging device in [the example] is a PCCT. Data acquisition was performed on an engineering prototype PCCT system equipped with a CdZnTe-PCD, obtaining two energy bins with energy thresholds set at 30 / 65 keV. In the standard pixel acquisition mode, after binning in the two-dimensional plane and the axial direction, the effective pixel size was 1 mm. A low-dose scenario was acquired using an X-ray tube current of 20 mA to minimize pulse pile-up effects. During the acquisition, a head dish compensator was used to maximize spectral variations, and Al and PMMA plates of different thicknesses were used as reference materials to obtain spectral calibration data. The energy-dependent attenuation coefficients of each material used the standard values recorded in the NIST XCOM database. The target objects could be a water phantom with a diameter of 180 mm and a Gammex head phantom with various concentrations of iodine and calcium inserts. After obtaining the material decomposition results through the decomposition model, the projections of 70 / 120 keV virtual monoenergetic images (VMIs) could be calculated based on this, and reconstructed using the standard filtered back projection (FBP) method, where the slice thickness was 5 mm and the in-plane side length of the voxel was 0.47 mm. Then, based on the 70 / 120 keV virtual monoenergetic images, image-domain material decomposition of solid water / iodine / calcium was performed, where it was assumed that all voxels had the same solid water concentration. Additionally, the attenuation coefficients of solid water / iodine / calcium could be calibrated using the Gammex inserts.

[0093] As Figure 4 shown, the decomposition model 460 can be trained.

[0094] In some embodiments, the second computing device 130 can obtain training samples 450 through data augmentation based on the spectral response function 430 and the spectral model 420, where the training samples 450 can include multiple ones.

[0095] In some embodiments, each training sample can include a first input parameter, a second input parameter, and a third input parameter. The second computing device 130 can randomly generate an integration path as the first input parameter; randomly select one from all the spectral response functions (each spectral response function corresponds to each pixel) as the second input parameter; and based on the first input parameter and the second input parameter, simulate a spectral measurement value through the spectral model as the third input parameter. For example, as Figure 5 shown, after obtaining the spectral response Si through spectral calibration, the data augmentation process includes: randomly generating an integration path L as the first input parameter, randomly selecting one S i from all the spectral response functions obtained through spectral calibration as the second input parameter, and based on L and S i , simulating a spectral measurement value y through the spectral model.

[0096] In some embodiments, the second input parameter may be the average energy spectrum response function of all energy spectrum response functions, i.e., the average of all energy spectrum response functions.

[0097] In some embodiments, after generating the training sample 450, the second computing device 130 may input the training sample 450 into the decomposition model 460 to train it to obtain a trained decomposition model 460.

[0098] In some embodiments, the second computing device 130 may use the second input parameter and the third input parameter as model inputs, and the first input parameter as the gold standard to train the decomposition model. As Figure 5 and 6 shown, the energy spectrum measurement value y and the energy spectrum response function S obtained by data augmentation can be input into the decomposition model (i.e., the MLP decomposition network 510, the MLP decomposition network 610), where S can be the average energy spectrum response function or can include all spectral response functions, and the integration path L is used as the gold standard for calculating the loss function (Loss).

[0099] The following is an example of the training process. During the training process, the Adam optimizer can be used, and the learning rate is set to 10 5 , to minimize the l-1 norm of the difference between L and the true value. The training dataset (S, y, L) is simulated in each epoch. For each data point, an energy spectrum response function S is randomly selected from the energy spectrum calibration results i , and a batch of material line integrals L are randomly sampled. By inserting the randomly sampled (S, y, L) into the general energy spectrum model (Equation (1)), the corresponding energy spectrum measurement value y is calculated. The training can include 10,000 epochs, and each epoch uses 1,000 simulated data points. The reference materials are Al and PMMA plates, where the range of the aluminum line integral is [0, 50] mm, and the range of the PMMA line integral is [0, 300] mm. To test the influence of different training methods on energy spectrum harmonization, two networks can be trained, one using only the average energy spectrum response function and the other using all energy spectrum response functions.

[0100] In some embodiments, on the basis of using the second input parameter and the third input parameter as model inputs, the second computing device 130 may further use the negative logarithm of the energy spectrum measurement value (the third input parameter) as the fourth input parameter. Then the second computing device 130 may use the second input parameter, the third input parameter, and the fourth input parameter as model inputs, and the first input parameter as the gold standard to train the model. As Figure 5As shown, during the data augmentation process, the negative logarithm Ly of the energy spectrum measurement value y obtained through simulation can be obtained. This is used as the fourth input parameter and, together with the energy spectrum measurement value y and the energy spectrum response function S, is input into the MLP decomposition network 510, while the integration path L serves as the gold standard to train the MLP decomposition network 510.

[0101] In some embodiments, on the basis of using the second input parameter, the third input parameter, and the fourth input parameter as model inputs, the second computing device 130 can further input a fifth input parameter into the decomposition model. The fifth input parameter may include at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc. Then, the second computing device 130 can use the second input parameter, the third input parameter, the fourth input parameter, and the fifth input parameter as model inputs and the first input parameter as the gold standard to train the model. As Figure 6 shown, the tube current I obtained during the data augmentation process can be used as the fifth input parameter and, together with the energy spectrum measurement value y, the energy spectrum response function S, the negative logarithm Ly of the energy spectrum measurement value y, and the tube current I, is input into the MLP decomposition network 610, while the integration path L serves as the gold standard to train the MLP decomposition network 610.

[0102] In some embodiments, during the process of obtaining multiple training samples through data augmentation, the second computing device 130 can randomly generate the fifth input parameter. For example, it can randomly generate at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc. After generating the fifth input parameter, the second computing device 130 can simulate the energy spectrum measurement value based on the first input parameter, the second input parameter, and the fifth input parameter through the energy spectrum model. As Figure 6 shown, during the data augmentation process, the tube current I can be randomly generated, and then based on L, S i and I, the energy spectrum measurement value y is simulated through the energy spectrum model.

[0103] In some embodiments, the parameters of the energy spectrum model used during the data augmentation process (such as energy spectrum response function, tube current, PCD counting dead time, etc.) may include multiple sets of parameters from multiple different PCCT systems, which can enhance the generalization ability of the decomposition network and improve the adaptability to different PCCT systems.

[0104] In some embodiments, other methods can be used instead of the data augmentation process, such as Monte Carlo simulation, etc.

[0105] In some embodiments of this specification, by training a decomposition model based on parameters related to the medical imaging device itself, such as all energy spectrum response functions, the trained decomposition model has good adaptability to different energy spectra, can obtain highly accurate material decomposition results, and at the same time, the trained model can parameterize different scanning protocols to calibrate the PCCT system.

[0106] It should be noted that the above descriptions of processes 300 and 400 are only for illustration and explanation, and do not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to processes 300 and 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, data such as energy spectrum measurement values and material decomposition results (i.e., material line integrals) in process 300 can also be used as training samples for the decomposition model.

[0107] The beneficial effects that the embodiments of this specification may bring include but are not limited to: (1) Through an energy spectrum model that includes the energy spectrum parameters of the medical imaging device (e.g., energy spectrum response function), integrating it into a machine learning model, and using the machine learning model to perform material decomposition on substances, it can adapt to different energy spectrum responses and provide accurate material decomposition results for various energy spectrum CT systems; (2) By including other parameter information of the medical imaging device (e.g., at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold, etc.) in the energy spectrum model, the machine learning model can overcome the influence of other adverse factors, thereby further improving the quality of material decomposition; (3) By training a decomposition model based on parameters related to the medical imaging device itself, such as all energy spectrum response functions, the trained decomposition model has good adaptability to different energy spectra, can obtain highly accurate material decomposition results, and at the same time, the trained model can parameterize different scanning protocols to calibrate the PCCT system. It should be noted that the beneficial effects that different embodiments may bring are different. In different embodiments, the beneficial effects that may be produced can be any one or several combinations of the above, or any other beneficial effects that may be obtained.

[0108] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are proposed in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0109] Meanwhile, this specification uses specific terms to describe the embodiments of this specification. Terms such as "an embodiment", "one embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0110] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numerical and alphabetical characters, or the use of other names in this specification are not used to limit the order of the processes and methods in this specification. Although some currently useful embodiments of the invention are discussed through various examples in the above disclosure, it should be understood that such details only serve the purpose of illustration. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0111] Similarly, it should be noted that, in order to simplify the expression of the disclosure in this specification and thus help the understanding of one or more embodiments of the invention, in the previous description of the embodiments of this specification, sometimes multiple features are grouped into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above.

[0112] In some embodiments, numbers are used to describe the components and the quantity of attributes. It should be understood that such numbers used for the description of embodiments are modified by the modifiers "about", "approximate", or "substantially" in some examples. Unless otherwise stated, "about", "approximate", or "substantially" indicate that the said numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values can change according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used to confirm the breadth of the scope in some embodiments of this specification are approximate values, in specific embodiments, such numerical settings are made as precise as possible within the feasible range.

[0113] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated by reference into this specification. This excludes application history files that are inconsistent with or conflict with the content of this specification, as well as files that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0114] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. An image analysis method, comprising: Performing energy spectrum calibration using a reference material to obtain an energy spectrum response function for each pixel; Based on the energy spectrum response function and the energy spectrum measurement value of the target object, obtaining a reference material decomposition result image corresponding to the target object through a decomposition model; Based on the reference material decomposition result image, obtaining a target material decomposition result image in the target object.

2. The method according to claim 1, wherein the performing energy spectrum calibration using a reference material to obtain an energy spectrum response function for each pixel comprises: Obtaining first raw data of an energy spectrum scan of the reference material; Determining first calibration data based on the first raw data; Obtaining the energy spectrum response function based on the first calibration data and an energy spectrum model.

3. The method according to claim 2, wherein the energy spectrum model is related to at least one of tube current, tube voltage, filter material, filter shape, and photon counting detector energy channel threshold.

4. The method according to claim 1, wherein the obtaining, based on the energy spectrum response function and the energy spectrum measurement value of the target object, a reference material decomposition result image corresponding to the target object through a decomposition model comprises: Inputting the energy spectrum response function, the energy spectrum measurement value, and the negative logarithm of the energy spectrum measurement value into the decomposition model to obtain the reference material decomposition result image.

5. The method according to claim 1, wherein the obtaining, based on the reference material decomposition result image, a target material decomposition result image in the target object comprises: Performing image domain material decomposition based on the reference material decomposition result image to obtain the target material decomposition result image.

6. The method according to claim 1, wherein the decomposition model comprises a multi-layer perceptron network.

7. The method according to claim 1, wherein the decomposition model is trained through the following steps: Based on the energy spectrum response function and the energy spectrum model, obtaining a plurality of training samples through data augmentation; Training the decomposition model based on the plurality of training samples.

8. The method according to claim 7, wherein each of the plurality of training samples comprises a first input parameter, a second input parameter, and a third input parameter, and the obtaining a plurality of training samples through data augmentation comprises: Randomly generating an integration path as the first input parameter; Randomly selecting one from all the energy spectrum response functions as the second input parameter; Based on the first input parameter and the second input parameter, simulating the energy spectrum measurement value through the energy spectrum model as the third input parameter.

9. An image processing system, comprising an energy spectrum response acquisition module, a first result acquisition module, and a second result acquisition module; The energy spectrum response acquisition module is configured to perform energy spectrum calibration using a reference material to obtain an energy spectrum response function for each pixel; The first result acquisition module is configured to obtain a reference material decomposition result image corresponding to the target object through a decomposition model based on the energy spectrum response function and the energy spectrum measurement value of the target object; The second result acquisition module is configured to obtain a target material decomposition result image in the target object based on the reference material decomposition result image.

10. A computer-readable storage medium storing computer instructions, which, when read by a computer, cause the computer to execute the method according to any one of claims 1 to 8.

Images