Medical information processing method and medical information processing apparatus

Abstract

The medical information processing method of the embodiment includes the steps of acquiring an X-ray CT image (I1) and spectral information at the time of imaging of the X-ray CT image (I1), segmenting the X-ray CT image (I1) by each substance to acquire distribution data (D11, D12, D13) of substances in the X-ray CT image (I1), performing forward projection processing on the distribution data (D11, D12, D13) based on the spectral information and the attenuation coefficient of each substance, acquiring a plurality of forward projection data (P11, P12, P13) of each of the substances, and generating a learning completed model (M1) by machine learning based on the plurality of forward projection data (P11, P12, P13) and raw data (R1) used in the generation of the X-ray CT image (I1).

Description

Technical Field

[0001] The embodiments disclosed in this specification and accompanying drawings relate to medical information processing methods and medical information processing devices. Background Technology

[0002] X-ray computed tomography (CT) images are widely used in clinical settings. The imaging conditions of X-ray CT images are appropriately adjusted according to the purpose of interpretation.

[0003] For example, the imaging conditions for X-ray CT images can be set by specifying the spectrum of X-rays used in the imaging. For example, during the imaging of an X-ray CT image, an X-ray CT image can be obtained by irradiating the subject with X-rays having a spectrum corresponding to the set X-ray energy value and reconstructing projection data based on the X-ray detection results.

[0004] Here, after obtaining X-ray CT images, users such as physicians sometimes wish to refer to X-ray CT images acquired using other X-ray spectra. For example, for purposes such as diagnosing the subject, users sometimes determine that it is preferable to refer to images with higher contrast or less noise. However, if X-ray CT images are acquired again using other X-ray spectra, the radiation dose to the subject increases.

[0005] Existing technical documents:

[0006] Patent documents:

[0007] Patent Document 1: Japanese Patent Application Publication No. 2021-10727

[0008] Patent Document 2: Japanese Patent Application Publication No. 2007-202700

[0009] Patent Document 3: Japanese Patent Application Publication No. 2020-201244 Summary of the Invention

[0010] The problem that the invention aims to solve

[0011] One of the problems to be solved by the embodiments disclosed in this specification and accompanying drawings is that it is possible to obtain X-ray CT images corresponding to spectral information of X-rays different from those used during imaging. However, the problems to be solved by the embodiments disclosed in this specification and accompanying drawings are not limited to the above-mentioned problems. Problems corresponding to the effects of the various configurations shown in the embodiments described below can also be identified as other problems.

[0012] Methods for solving problems

[0013] The medical information processing method of the implementation includes the following steps: obtaining an X-ray CT image and spectral information of the X-ray CT image at the time of imaging; segmenting the X-ray CT image according to each substance to obtain the distribution data of the substances in the X-ray CT image; performing forward projection processing on the distribution data based on the spectral information and the attenuation coefficient of each substance to obtain multiple forward projection data for each substance; and generating a learned model through machine learning based on the multiple forward projection data and the original data used in the generation of the X-ray CT image. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating an example of the configuration of the medical information processing device according to the first embodiment.

[0015] Figure 2 This is a block diagram illustrating an example of the configuration of the medical information processing system according to the first embodiment.

[0016] Figure 3 This is a diagram used to illustrate the learning phase involved in the first embodiment.

[0017] Figure 4 This is a diagram used to illustrate the utilization stage involved in the first embodiment.

[0018] Figure 5A This is a flowchart illustrating the learning phase process performed by the medical information processing device according to the first embodiment.

[0019] Figure 5B This is a flowchart illustrating the learning phase process performed by the medical information processing device according to the first embodiment.

[0020] Figure 6 This is a flowchart illustrating the processing of the utilization phase performed by the medical information processing device according to the first embodiment.

[0021] Figure 7 This is a display example according to the first embodiment.

[0022] Figure 8 This is a diagram used to illustrate the learning phase involved in the third embodiment.

[0023] Figure 9 This is a diagram used to illustrate the utilization stage involved in the third embodiment.

[0024] Figure 10A This is a flowchart illustrating the learning phase processing performed by the medical information processing device according to the third embodiment.

[0025] Figure 10BThis is a flowchart illustrating the learning phase processing performed by the medical information processing device according to the third embodiment.

[0026] Figure 11 This is a diagram used to illustrate the learning phase involved in the fourth embodiment.

[0027] Figure 12 This diagram illustrates an example of a method for obtaining learning data according to the fourth embodiment.

[0028] Figure 13 This is a diagram used to illustrate the utilization stage involved in the fourth embodiment.

[0029] Figure 14A This is a flowchart illustrating the learning phase processing performed by the medical information processing device according to the fourth embodiment.

[0030] Figure 14B This is a flowchart illustrating the learning phase processing performed by the medical information processing device according to the fourth embodiment.

[0031] Figure 15 This is a flowchart illustrating the processing of the utilization phase performed by the medical information processing device according to the fourth embodiment. Detailed Implementation

[0032] The embodiments of the medical information processing method and the medical information processing device will be described in detail below with reference to the accompanying drawings.

[0033] (First Embodiment)

[0034] First, let me explain Figure 1 The medical information processing device 1 shown above generates a learning completion model M1, described later, which enables the acquisition of X-ray CT images corresponding to spectral information of X-rays different from those used during imaging. In other words, the medical information processing device 1 generates a learning completion model M1 for conversion processing to other X-ray energies (kVp). The medical information processing device 1 includes, for example, a processing circuit 11 and a memory 12.

[0035] The processing circuit 11 executes the acquisition function 111 and the learning function 112, thereby controlling the overall operation of the medical information processing device 1. The acquisition function 111 is an example of an acquisition unit. The learning function 112 is an example of a learning unit.

[0036] For example, processing circuit 11 reads from memory 12 and executes the program corresponding to acquisition function 111, thereby acquiring various learning data as described later. Additionally, processing circuit 11 reads from memory 12 and executes the program corresponding to learning function 112, thereby performing machine learning using the learning data acquired by acquisition function 111, and generating a completed learning model M1. Detailed processing performed by acquisition function 111 and learning function 112 will be described later.

[0037] exist Figure 1 In the medical information processing device 1 shown, each processing function is stored in the memory 12 as a program that can be executed by a computer. The processing circuit 11 is a processor that reads out the program from the memory 12 and executes the program to realize the function corresponding to each program. In other words, the processing circuit 11 in the state of having read out the program has the function corresponding to the read-out program.

[0038] In addition, Figure 1 The description indicates that the acquisition function 111 and the learning function 112 are implemented by a single processing circuit 11. However, multiple independent processors can also be combined to form the processing circuit 11, and each processor can execute the program to implement the functions. In addition, the various processing functions of the processing circuit 11 can also be appropriately distributed or integrated into a single or multiple processing circuits.

[0039] The memory 12 is implemented, for example, by semiconductor memory elements such as RAM (Random Access Memory), flash memory, hard disk, optical disk, etc. For example, the memory 12 stores programs that enable the circuitry included in the medical information processing device 1 to perform its functions. In addition, the memory 12 stores various learning data, which will be described later.

[0040] Next, an explanation Figure 2 The medical information processing system shown includes, for example, an X-ray CT device 2 and a medical information processing device 3. The medical information processing device 3 uses the X-ray CT image taken by the X-ray CT device 2 and the learned completion model M1 generated by the medical information processing device 1 to obtain an X-ray CT image corresponding to the spectral information of X-rays different from those taken by the X-ray CT device 2.

[0041] The X-ray CT apparatus 2, for example, has an X-ray tube and an X-ray detector positioned opposite the subject. The X-ray CT apparatus 2 uses the X-ray detector to detect X-rays that have irradiated and transmitted through the X-ray tube onto the subject, thereby acquiring projection data (raw data). In addition, the X-ray CT apparatus 2 can reconstruct the acquired projection data according to each X-ray irradiation angle (viewing angle) to obtain X-ray CT images.

[0042] The medical information processing device 3 includes, for example, a memory 31, a display 32, an input interface 33, and a processing circuit 34.

[0043] The memory 31 is implemented, for example, by semiconductor memory elements such as RAM (Random Access Memory), flash memory, hard disk, optical disk, etc. For example, the memory 31 stores programs that enable the circuitry included in the medical information processing device 3 to perform its functions. In addition, the memory 31 stores the learning completion model M1 generated by the medical information processing device 1.

[0044] Display 32 displays various information. For example, display 32 displays various X-ray CT images. Additionally, display 32 may display a GUI (Graphical User Interface) for receiving various instructions or settings from the user via input interface 33. For example, display 32 may be a liquid crystal display (LCD) or a CRT (Cathode Ray Tube) display. Display 32 may be a desktop type or may be a tablet computer terminal or the like capable of wireless communication with the main body of the medical information processing device 3.

[0045] Input interface 33 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs them to processing circuit 34. For example, input interface 33 can be implemented using a mouse, keyboard, trackball, switch, button, joystick, touchpad for input operations via touch operation surface, touchscreen integrating the display screen and touchpad, contactless input circuit using optical sensors, voice input circuit, etc. Furthermore, input interface 33 can also be composed of a tablet computer terminal capable of wireless communication with the main body of medical information processing device 3. Additionally, input interface 33 can also be a circuit that accepts input operations from the user through motion capture. For example, input interface 33 can accept user body movements or gaze as input operations by processing signals obtained via a tracker or images captured from the user. Furthermore, input interface 33 is not limited to having physical operating components such as a mouse or keyboard. For example, processing circuits that accept electrical signals corresponding to input operations from external input devices separately from medical information processing device 3 and output these electrical signals to processing circuit 34 are also included in the example of input interface 33.

[0046] The processing circuit 34 performs the acquisition function 341, the image processing function 342, and the display control function 343, thereby controlling the overall operation of the medical information processing device 3.

[0047] For example, processing circuit 34 reads from memory 31 and executes the program corresponding to acquisition function 341, thereby acquiring multiple orthogonal projection data as described later. Additionally, processing circuit 34 reads from memory 31 and executes the program corresponding to image processing function 342, thereby inputting the multiple orthogonal projection data acquired by acquisition function 341 into the learned model M1, thus acquiring data corresponding to arbitrary spectral information. Furthermore, processing circuit 34 reads from memory 31 and executes the program corresponding to display control function 343, thereby causing display 32 to display X-ray CT images based on the data acquired by image processing function 342. Detailed processing performed by acquisition function 341, image processing function 342, and display control function 343 will be described later.

[0048] exist Figure 1 In the medical information processing device 1 shown, each processing function is stored in the memory 31 as a program that can be executed by a computer. The processing circuit 34 is a processor that reads out the program from the memory 31 and executes the program to realize the function corresponding to each program. In other words, the processing circuit 34, which has read out the state of the program, has the function corresponding to the read-out program.

[0049] In addition, Figure 1 The description states that the acquisition function 341, image processing function 342, and display control function 343 are implemented by a single processing circuit 34. However, multiple independent processors can also be combined to form the processing circuit 34, and each processor can execute programs to implement the functions. In addition, the various processing functions of the processing circuit 34 can also be appropriately distributed or integrated into a single or multiple processing circuits.

[0050] In addition, such as Figure 2 As shown, the X-ray CT unit 2 and the medical information processing unit 3 are interconnected via a network NW. The network NW can be either a local network closed within the facility or a network connected to the Internet. Furthermore, communication between the X-ray CT unit 2 and the medical information processing unit 3 can be conducted either through other devices such as image storage devices, or directly without using other devices. An example of such an image storage device is, for instance, a PACS (Picture Archiving and Communication System) server.

[0051] The above describes the medical information processing device 1 and the medical information processing system including the medical information processing device 3. Based on this configuration, the medical information processing device 1 generates and learns to complete the model M1, thereby enabling the medical information processing device 3 to obtain X-ray CT images corresponding to arbitrary X-ray spectral information that differs from that during imaging.

[0052] use Figure 3 This describes the learning process performed by the medical information processing device 1 to generate model M1. Figure 3 This is a diagram used to illustrate the learning phase involved in the first embodiment.

[0053] First, an image is taken of any subject, and raw data R1 is collected. Raw data R1 can be, for example, a sine wave. Raw data R1 can be obtained from... Figure 2 The X-ray CT device 2 shown can acquire the image, but other X-ray CT devices can also be used. Next, reconstruction processing based on the original data R1 is performed to generate the X-ray CT image I1. The acquisition function 111 acquires the X-ray CT image I1.

[0054] The reconstruction processing of the X-ray CT image I1 can be performed either by the acquisition function 111 or in other devices. For example, the X-ray CT device 2 takes a picture of the subject to acquire raw data R1, and reconstructs the X-ray CT image I1 based on the acquired raw data R1; the acquisition function 111 acquires the X-ray CT image I1 reconstructed by the X-ray CT device 2. Alternatively, the X-ray CT device 2 takes a picture of the subject to acquire raw data R1, and the acquisition function 111 reconstructs the X-ray CT image I1 based on the raw data R1 acquired by the X-ray CT device 2.

[0055] The method for reconstructing the X-ray CT image I1 is not particularly limited. Examples include FBP (Filtered Back Projection), successive approximation reconstruction, and successive approximation application reconstruction. Alternatively, the X-ray CT image I1 can also be reconstructed using machine learning methods. For instance, the X-ray CT image I1 can also be reconstructed using DLR (Deep Learning Reconstruction).

[0056] Next, the acquisition function 111 segments the acquired X-ray CT image I1 according to each substance, and obtains the distribution data of the substances in the X-ray CT image I1. For example, the acquisition function 111 segments the X-ray CT image I1 according to each internal organ.

[0057] The segmentation method is not particularly limited. Examples include Otsu binarization based on CT values, region expansion, Snake's method, graph cut, and mean shift. Alternatively, the X-ray CT image I1 can be displayed, and segmentation can be performed manually by allowing the user to specify the extent of each internal organ.

[0058] Alternatively, function 111 can also utilize machine learning methods to segment the X-ray CT image I1. For example, by using any X-ray CT image as input data and the results of manual segmentation of that X-ray CT image by a physician or other professional as output data, machine learning can be performed to generate a learned model that is capable of segmenting the input X-ray CT image. By inputting the X-ray CT image I1 into such a learned model, function 111 can obtain the distribution data of substances in the X-ray CT image I1.

[0059] In addition, Figure 3 In the diagram, it represents the case where the X-ray CT image I1 is segmented according to each substance to obtain three distribution data (distribution data D11, distribution data D12, and distribution data D13). However, Figure 3 This is just one example; the amount of distributed data obtained through segmentation is arbitrary.

[0060] Additionally, the acquisition function 111 acquires spectral information during the imaging of the X-ray CT image I1. For example, the acquisition function 111 acquires spectral information of the X-rays irradiated from the X-ray tube of the X-ray CT apparatus 2 during the imaging of the X-ray CT image I1. The spectral information is, for example, information that establishes a correspondence between X-ray intensity and each wavelength (X-ray energy). The spectral information can also be added to the X-ray CT image I1 as supplementary information.

[0061] More specifically, the spectral information can be determined for each X-ray tube in accordance with the setting of the X-ray energy during the imaging of the X-ray CT image I1. For example, if "120 kVp" is set as the X-ray energy, X-rays whose spectrum extends in the range below "120 kVp" are irradiated.

[0062] Furthermore, since the shape of the spectrum varies individually for each X-ray tube, spectral information can be determined with greater accuracy by calibrating each X-ray tube in advance. For example, such calibration can be performed using a spectrophotometer. That is, by measuring the X-ray spectrum in advance using a spectrophotometer for each X-ray tube and for each setting of X-ray energy, the spectral information for the X-ray CT image I1 can be determined with greater accuracy. For example, a detector using high-purity germanium can be used as such a spectrophotometer.

[0063] Furthermore, the acquisition function 111 can acquire various data, such as X-ray CT images or spectral information, via the network NW, or via a storage medium. Additionally, the acquisition function 111 stores the acquired data in the memory 12.

[0064] Next, based on the spectral information of the X-ray CT image I1 taken during imaging and the attenuation coefficient of each substance, function 111 performs orthogonal projection processing on the distribution data of the substances in the X-ray CT image I1 to obtain multiple orthogonal projection data for each substance. For example, in Figure 3 In the case shown, function 111 performs forward projection processing on the distribution data D11 to obtain forward projection data P11, performs forward projection processing on the distribution data D12 to obtain forward projection data P12, and performs forward projection processing on the distribution data D13 to obtain forward projection data P13.

[0065] That is, when projecting forward from a specific direction, the distribution data of the substances in the X-ray CT image I1 can be used to determine the length distribution of each substance along the path of forward projection. In other words, the path length of each substance can be calculated based on the distribution data of the substances in the X-ray CT image I1. Then, using the path length of each substance and its attenuation coefficient, the X-ray absorption can be calculated, and forward projection data can be generated. Furthermore, the attenuation coefficient of each substance can be obtained, for example, from literature values ​​such as those from NIST (National Institute of Standards and Technology).

[0066] More specifically, the distribution data is generated through segmentation, assuming a single substance, and therefore a single attenuation coefficient is assigned to each distribution data point. For example, an attenuation coefficient μ1 is assigned to distribution data D11, an attenuation coefficient μ2 to distribution data D12, and an attenuation coefficient μ3 to distribution data D13. However, the attenuation coefficient is dependent on the X-ray energy, and X-rays emitted from the X-ray tube have a spectral extension and are multicolored. Therefore, function 111 is obtained by integrating the attenuation coefficient μ1 as a function of X-ray energy and multiplying it by the path length, thus obtaining the forward projection data.

[0067] Next, learning function 112 performs machine learning using the various data acquired by acquisition function 111 as learning data. Specifically, learning function 112 performs machine learning based on... Figure 3 Machine learning of multiple forward projection data and the raw data R1 used in the generation of X-ray CT image I1 is used to generate a learning-completed model M1 for conversion processing to other X-ray energies (kVp).

[0068] exist Figure 3The example illustrates the case where the learned model M1 is constructed using a neural network (NN). A neural network is a network with a hierarchical structure where adjacent layers are linked together, and information propagates from the input layer to the output layer. A neural network typically consists of an input layer, multiple intermediate layers (hidden layers), and an output layer.

[0069] For example, the learning function 112 inputs multiple forward projection data obtained by the acquisition function 111 based on the X-ray CT image I1 as input data into the neural network. Additionally, the learning function 112 inputs the raw data R1 used in the reconstruction of the X-ray CT image I1 as output data into the neural network. That is, the learning function 112 inputs the raw data R1 actually acquired during the imaging of the X-ray CT image I1 as correct data into the neural network.

[0070] Here, the multiple forward projection data inputs are obtained by forward projection processing of the distribution data segmented by each substance based on spectral information and the attenuation coefficient of each substance. Furthermore, the sum of these multiple forward projection data, along with the original data R1 upon which they are based, becomes data containing information about multiple substances. However, errors sometimes arise between this sum of multiple forward projection data and the original data R1. For example, when generating the forward projection data, the attenuation coefficient of each substance is set to a specific value (literature value, etc.), but the attenuation coefficient of each substance sometimes varies individually for each subject. Additionally, for example, when the attenuation coefficient for a subject's liver is set to a specific value, the value of the attenuation coefficient may deviate for each location within the liver. Furthermore, errors may also arise due to the precision of the segmentation.

[0071] Learning function 112 enables the neural network to learn in order to minimize the error between the sum of multiple forward projection data as input data and the original data R1 as output data, thereby generating a learned model M1. For example, the relationship between the sum of multiple forward projection data and the original data R1 can be represented by the following equation (1). Equation (1) can be set for each point on the sine graph. That is, equation (1) can be set for each ray (projection position and projection angle) of the forward projection.

[0072] [Formula 1]

[0073] ∫μ1(E)dE*L1'*c1+∫μ2(E)dE*L2*c2+∫μ3(E)dE*L3'*c3=(RawDatα)…(1)

[0074] In equation (1), the right-hand side (Raw Data) represents the pixel value at a point on the original data R1. The left-hand side “∫μ1(E)dE*L1’” represents the pixel value at a point in the forward-projected data P11 corresponding to the right-hand side (Raw Data). The left-hand side “∫μ2(E)dE*L2’” represents the pixel value at a point in the forward-projected data P12 corresponding to the right-hand side (Raw Data). The left-hand side “∫μ3(E)dE*L3’” represents the pixel value at a point in the forward-projected data P13 corresponding to the right-hand side (Raw Data).

[0075] More specifically, "μ1" is the attenuation coefficient of a single substance corresponding to the forward projection data P11, "μ2" is the attenuation coefficient of a single substance corresponding to the forward projection data P12, and "μ3" is the attenuation coefficient of a single substance corresponding to the forward projection data P13. As mentioned above, these attenuation coefficients are integrated as a function of the X-ray energy "E".

[0076] “L1’” is the path length obtained by projecting forward using distribution data D11, assuming a single substance. That is, “L1’” is a known value based on distribution data D11. Similarly, “L2’” is the path length obtained by projecting forward using distribution data D12, assuming a single substance, and “L3’” is the path length obtained by projecting forward using distribution data D13, assuming a single substance.

[0077] “c1”, “c2”, and “c3” are coefficients multiplied by each term to make the equality in equation (1) true. That is, as mentioned above, an error arises between the sum of multiple forward projection data and the original data R1 due to the setting of the attenuation coefficient or segmentation processing. In equation (1), the error is absorbed by multiplying by coefficients such as “c1”, “c2”, and “c3” to make the equality true. In other words, equation (1) is the numerical expression obtained by rewriting the virtual path lengths “L1”, “L2”, and “L3” when equation (2) is true as “L1’*c1”, “L2’*c2”, and “L3’*c3” respectively.

[0078] [Formula 2]

[0079] ∫μ1(E)dE*L1+∫μ2(E)dE*L2+∫μ3(E)dE*L3=(RawData)…(2)

[0080] Therefore, for example, when the neural network learns by using the three terms "∫μ1(E)dE*L1'", "∫μ2(E)dE*L2'", and "∫μ3(E)dE*L3'" on the left side of equation (1) as input data and the right side (Raw Data) as output data, the neural network can learn coefficients such as "c1", "c2", and "c3". Furthermore, equation (1) can be set multiple times for each point on the sine wave. Typically, it is impossible to solve for coefficients such as "c1", "c2", and "c3" as a unique solution; therefore, the neural network learns coefficients to minimize the error between the left and right sides in multiple equations (1).

[0081] By performing the aforementioned machine learning, learning function 112 can generate a learned model M1 that has learned coefficients such as "c1", "c2", and "c3". That is, learning function 112 enables the neural network to learn the coefficients multiplied by each forward projection data point to minimize the error between the sum of multiple forward projection data points and the original data R1, thereby generating the learned model M1. The generation process of the learned model M1 by learning function 112 can be performed offline.

[0082] Furthermore, it was explained that in Equation (1), the terms of “∫μ1(E)dE*L1’”, “∫μ2(E)dE*L2’”, and “∫μ3(E)dE*L3’” are multiplied by coefficients such as “c1”, “c2”, and “c3” to absorb errors. However, the implementation is not limited to this. That is, the mathematical processing used to absorb errors is not limited to multiplying by coefficients.

[0083] For example, instead of multiplying each of the forward projected data shown on the left side of equation (1) by a coefficient, an exponent can be set or an additional term can be added. Through such arbitrary mathematical processing or combinations thereof, the error between the sum of multiple forward projected data and the original data R1 can be absorbed. In this case, the neural network learns the mathematical processing or combination thereof that minimizes the error.

[0084] Next, use Figure 4 This section describes the utilization phase of model M1 after learning. First, photography is performed on any subject to collect raw data R2. Raw data R2 can be collected from the same subject as raw data R1, or from other subjects. Furthermore, raw data R2 can be obtained from... Figure 2 The X-ray CT device 2 shown can acquire the image, but other X-ray CT devices can also be used. Next, reconstruction processing based on the original data R2 is performed to generate the X-ray CT image I2. The acquisition function 341 acquires this X-ray CT image I1.

[0085] The reconstruction of the X-ray CT image I2 can be performed either by the acquisition function 341 or by other devices. For example, the X-ray CT device 2 takes a photograph of the subject to acquire projection data, and reconstructs the X-ray CT image I2 based on the acquired projection data; the acquisition function 341 then acquires the X-ray CT image I2 reconstructed by the X-ray CT device 2. Alternatively, the X-ray CT device 2 takes a photograph of the subject to acquire projection data, and the acquisition function 341 reconstructs the X-ray CT image I2 based on the projection data acquired by the X-ray CT device 2. Furthermore, the method for reconstructing the X-ray CT image I2 is not particularly limited.

[0086] Next, function 341 segments the acquired X-ray CT image I2 for each substance, obtaining the distribution data of the substances in X-ray CT image I2. For example, function 341 segments the X-ray CT image I2 for each internal organ. The segmentation method is not particularly limited; machine learning methods can also be used for segmentation. Figure 4 In the figure, it represents the case where the X-ray CT image I2 is segmented according to each substance to obtain three distribution data (distribution data D21, distribution data D22 and distribution data D23).

[0087] Furthermore, based on arbitrary spectral information and the attenuation coefficient of each substance, function 341 performs orthogonal projection processing on the distribution data of substances in X-ray CT image I2 to obtain multiple orthogonal projection data for each substance. For example, in Figure 4 In the case shown, function 341 performs forward projection processing on distributed data D21 to obtain forward projection data P21, performs forward projection processing on distributed data D22 to obtain forward projection data P22, and performs forward projection processing on distributed data D23 to obtain forward projection data P23.

[0088] Here, any spectral information can differ from the spectral information obtained during the imaging of the X-ray CT image I2. For example, in Figure 4 In the example, X-ray CT image I2 is an image taken using X-rays at "120 kVp". In contrast, the arbitrary spectral information set to obtain orthographic projection data is "135 kVp".

[0089] For example, arbitrary spectral information can be set based on the X-ray energy value input by the user. For example, the user operates the GUI displayed on the monitor 32 and inputs the desired X-ray energy value. For example, if the imaging conditions of X-ray CT image I2 are "120 kVp" and the user wants to reference an image with higher contrast than X-ray CT image I2, the user can input an X-ray energy value lower than "120 kVp". Conversely, if the user wants to reference an image with less noise than X-ray CT image I2, the user can input an X-ray energy value higher than "120 kVp".

[0090] Then, image processing function 342 inputs multiple forward projection data of each substance based on X-ray CT image I2 into the learned model M1 to obtain raw data R3. Figure 3 In the case shown, the original data R3 corresponds to the arbitrary spectral information "135kVp".

[0091] For example, in the learned model M1, the coefficients “c1”, “c2”, and “c3” shown in equation (1) are learned. These coefficients can be understood as path length coefficients, which have low dependence on X-ray energy (kVp). Therefore, with any input spectral information, the learned coefficients “c1”, “c2”, and “c3” can be applied, and the original data R3 can be generated based on multiple forward projection data. That is, the learned model M1 can perform conversion processing to other X-ray energies (kVp).

[0092] Furthermore, the image processing function 342 can also generate an X-ray CT image corresponding to the arbitrary spectral information based on the raw data R3 corresponding to that arbitrary spectral information. The display control function 343 enables the display 32 to display the X-ray CT image corresponding to the arbitrary spectral information.

[0093] Image processing function 342 can also perform material identification processing using multiple X-ray CT images corresponding to multiple spectral information obtained by using the learned model M1. The following describes the case where X-ray CT images corresponding to high energy and X-ray CT images corresponding to low energy are obtained using the learned model M1. For example, image processing function 342 can decompose the X-ray CT images corresponding to high energy and low energy for each reference material and generate reference material images for each reference material. Furthermore, image processing function 342 can also further generate various images such as monochromatic X-ray images, density images, and effective atomic number images based on the generated reference material images.

[0094] use Figure 5A and Figure 5B Explanation in Figure 3The medical information processing device 1 performs a series of processes during the learning phase as described in the text. Figure 5A and Figure 5B This is a flowchart illustrating the learning phase processing performed by the medical information processing device 1 according to the first embodiment.

[0095] First, the acquisition function 111 acquires raw data (step S111) and reconstructs an X-ray CT image based on the raw data (step S112). Alternatively, the acquisition function 111 can replace steps S111 and S112 to acquire an X-ray CT image reconstructed by another device. Next, the acquisition function 111 segments the acquired X-ray CT image by each substance and acquires the distribution data of the substances (step S113). Next, the acquisition function 111 performs forward projection processing on the distribution data of the substances to acquire multiple forward projection data for each substance (step S114). Then, the acquisition function 111 associates the raw data from step S111 with the multiple forward projection data from step S114 as a set of learning data and stores it in the memory 12 (step S115). For example, the acquisition function 111... Figure 3 The original data R1 shown is associated with multiple forward projection data (forward projection data P11, forward projection data P12 and forward projection data P13) as a set of learning data and stored in memory 12.

[0096] Function 111 executes each time new raw data is obtained. Figure 5A The series of steps shown. As a result, the learning data of step S115 is stored in memory 12. Learning function 112 retrieves multiple sets of learning data from memory 12 (step S121), and generates a completed learning model M1 by performing machine learning using the multiple sets of learning data (step S122).

[0097] Next, use Figure 6 Explanation in Figure 4 The medical information processing device 3 performs a series of processes during the utilization phase as described in the text. Figure 6 This is a flowchart illustrating the processing of the utilization phase performed by the medical information processing device 3 according to the first embodiment.

[0098] First, function 341 acquires raw data (step S211), and reconstructs an X-ray CT image based on the raw data (step S212). For example, in step S212, function 341 acquires raw data based on the raw data. Figure 4The original data R2 shown is used to reconstruct the X-ray CT image I2. Alternatively, function 341 can replace steps S211 and S212 to obtain the X-ray CT image I2 reconstructed by another device. X-ray CT image I2 is an example of a first CT image of the subject obtained by performing a CT scan corresponding to the first spectral information.

[0099] Next, the acquisition function 341 segments the acquired X-ray CT images (X-ray CT image I2, etc., the first CT image) for each substance to obtain the distribution data of the substances (step S213). That is, the acquisition function 341 applies computer segmentation processing to the first CT image to obtain the distribution data of multiple substances.

[0100] Next, image processing function 342 performs forward projection processing on the distribution data of the substances based on arbitrary spectral information and the attenuation coefficient of each substance, obtaining multiple forward projection data for each substance (step S214). For example, image processing function 342 as follows: Figure 4 As shown, based on arbitrary spectral information "135kVp", forward projection processing is performed on distribution data D21 to obtain forward projection data P21, forward projection processing is performed on distribution data D22 to obtain forward projection data P22, and forward projection processing is performed on distribution data D23 to obtain forward projection data P23. That is, image processing function 342 can obtain multiple forward projection data by performing forward projection processing on each distribution data of multiple substances based on the second spectral information.

[0101] Then, image processing function 342 inputs the multiple forward projection data from step S214 into the learned model M1 to obtain the original data corresponding to arbitrary spectral information (step S215). For example, image processing function 342 as follows: Figure 4 As shown, forward projection data P21, forward projection data P22, and forward projection data P23 are input into the learned model M1 to obtain the original data R3 corresponding to any spectral information "135kVp". That is, the image processing function 342 can obtain the original data corresponding to the second spectral information by applying the learned model M1 to multiple forward projection data.

[0102] Furthermore, the image processing function 342 generates an X-ray CT image corresponding to the arbitrary spectral information based on the raw data obtained in step S215 (step S216). That is, the image processing function 342 reconstructs a second CT image corresponding to the second spectral information based on the raw data obtained in step S215. The X-ray CT image corresponding to the arbitrary spectral information is output for display or analysis by the user (step S217). For example, the display control function 343 causes the display 32 to display the X-ray CT image corresponding to the arbitrary spectral information. Alternatively, for example, the X-ray CT image corresponding to the arbitrary spectral information is transmitted via a network NW to any display device (not shown) and displayed on that display device. Alternatively, for example, the analysis result based on the X-ray CT image corresponding to the arbitrary spectral information is displayed on the display 32 or any display device. Examples of such analysis results include the aforementioned reference material image, monochromatic X-ray image, density image, effective atomic number image, etc.

[0103] pass Figure 6 Steps S213 to S216 involve performing conversion processing based on the distribution data of multiple substances. For example, in... Figure 4 In the example, conversion processing based on distribution data D21, D22, and D23 corresponding to "120kVp" is performed to generate original data R3 corresponding to "135kVp". Then, based on this distribution data D21, an X-ray CT image corresponding to "135kVp" is reconstructed. That is, the medical information processing device 3 obtains the first CT image of the subject obtained by performing a CT scan corresponding to the first spectral information. Computer segmentation processing is applied to the first CT image to obtain distribution data for multiple substances. By performing conversion processing based on the distribution data of multiple substances, a second CT image corresponding to the second spectral information can be obtained.

[0104] The learning completion model M1 described above can also be generated under various conditions. For example, the learning completion model M1 can be generated based on each condition, such as the patient's location, age, or physique, as shown in the X-ray CT image I1 used during learning. For example, the acquisition function 111 can also generate multiple learning completion models M1, such as one learned using an image of the "head," one learned using an image of the "chest," and one learned using an image of the "abdomen." In this case, the image processing function 342 uses the learning completion model M1 acquired corresponding to the location of the subject in the X-ray CT image I2 to acquire the raw data R3 corresponding to any spectral information. Thus, the learning completion model M1 can take into account the influence of the location of the subject or patient information, and perform conversion processing to other X-ray energies (kVp) with higher accuracy.

[0105] Alternatively, in the generation process of the learning-completed model M1 described above, information such as the location of the radiographed object or patient information can also be used as learning data. In this case, in addition to the multiple forward projection data based on the X-ray CT image I2 mentioned above, the image processing function 342 also inputs information such as the location of the radiographed object or patient information into the learning-completed model M1. As a result, the learning-completed model M1 can take into account the influence of the location of the radiographed object or patient information, and perform the conversion processing to other X-ray energies (kVp) with higher accuracy.

[0106] Next, use Figure 7 The display example in step S217 is explained below. Furthermore, the display in step S217 can also be performed on other display devices different from the medical information processing device 3, but... Figure 7 The description illustrates an example of how display control function 343 causes display 32 to display an image. Furthermore, the image displayed on display 32 is not limited to X-ray CT images; it can also be a resolution result based on the X-ray CT image (e.g., a reference material image, a monochromatic X-ray image, a density image, an effective atomic number image, etc.). Figure 7 The illustration shows an example of an X-ray CT image.

[0107] For example, according to Figure 6 The flowchart shows how to obtain the first CT image reconstructed in step S212 corresponding to the first spectral information and the second CT image reconstructed in step S216 corresponding to the second spectral information. The display control function 343 enables the display 32 to display the first CT image and the second CT image. Furthermore, if there are previously acquired CT images (hereinafter referred to as past images) for the same subject, the display control function 343 can also enable the display 32 to display not only the first CT image and the second CT image, but also the past images.

[0108] exist Figure 7 In this example, CT image I11 is shown as the first CT image, CT image I12 is shown as a past image, and CT image I13 is shown as the second CT image. Furthermore, the specific display method regarding layout or UI (User Interface) can be appropriately changed. For example, in... Figure 7 In this example, three display areas are set up and CT images I11, I12, and I13 are displayed in an arranged manner. However, for example, CT images I11, I12, and I13 can also be switched and displayed in one display area.

[0109] CT image I11 and CT image I12 are CT images acquired from the same subject at different dates and times. For example, when examining a subject who has suffered a systemic injury, sometimes a small-area CT scan, including the injured area, is performed again after a large-area CT scan. Additionally, when developing a treatment plan or observing the course of treatment, CT scans are sometimes repeatedly performed on the area containing the lesion to evaluate the progression of the lesion or the treatment effect. In cases where multiple CT scans are performed on the same subject, the display control function 343 can acquire CT image I12, which is a past image, in addition to CT image I11, and display it on the display 32.

[0110] CT image I13 is a CT image corresponding to "135kVp" generated by converting CT image I11, which corresponds to "120kVp". Display control function 343 can also display unconverted CT images, such as CT image I11 or CT image I12, and converted CT images, such as CT image I13, in a recognizable manner. For example, display control function 343... Figure 7 As shown, a "Converted" display can be added to the CT image I13 based on the conversion process. This allows users such as physicians to make diagnoses based on the understanding that the CT image I13 is a converted CT image. For example, in Figure 7 The image shows a CT image I12 with the same X-ray energy as CT image I13, which allows for diagnosis by taking into account both the latest but converted CT image I13 and the older but unconverted CT image I12.

[0111] exist Figure 4 The diagram represents a single learning completion model M1 (neural network NN), but multiple learning completion models M1 can also be used for transformation processing.

[0112] For example, the medical information processing device 1 generates a learning completion model M1 for each substance. For instance, the medical information processing device 1 generates learning completion models M1 corresponding to attenuation coefficient μ1, M1 corresponding to attenuation coefficient μ2, and M1 corresponding to attenuation coefficient μ3, respectively. In this case, during the conversion process, multiple forward projection data are obtained by performing forward projection processing based on the second spectral information on the distribution data of multiple substances. The learning completion model is then applied to these multiple forward projection data to obtain processed multiple forward projection data. These processed multiple projection datasets are then synthesized to obtain synthetic forward projection data. Based on this synthetic forward projection data, the second CT image can be reconstructed.

[0113] For example, image processing function 342 is for Figure 4 The distributed data D21 to D23 shown are subjected to forward projection processing based on the second spectral information to obtain multiple forward projection data P21 to P23. Next, image processing function 342 applies the learning completion model M1 corresponding to the attenuation coefficient μ1 to the forward projection data P21 to obtain processed forward projection data P21. Similarly, image processing function 342 applies the learning completion model M1 corresponding to the attenuation coefficient μ2 to the forward projection data P22 to obtain processed forward projection data P22. Furthermore, image processing function 342 applies the learning completion model M1 corresponding to the attenuation coefficient μ3 to the forward projection data P23 to obtain processed forward projection data P23. Next, image processing function 342 synthesizes the processed forward projection data P21, P22, and P23 to obtain synthesized forward projection data. Then, image processing function 342 reconstructs the second CT image based on this synthesized forward projection data.

[0114] (Second Implementation)

[0115] In the second embodiment, the case where a learning-completed model M1 is generated in accordance with the X-ray energy (kVp) of the X-ray CT image I1 is described.

[0116] For example, to obtain function 111, etc. Figure 3 As shown, an X-ray CT image I1 taken at "120kVp" is obtained, and multiple orthographic projection data for each substance are acquired. In addition, the learning function 112 generates a learning-completed model M1 corresponding to "120kVp" through machine learning based on these multiple orthographic projection data and the original data R1 used in the generation of the X-ray CT image I1.

[0117] Additionally, function 111 acquires an X-ray CT image I1 taken at "100 kVp" and obtains multiple orthographic projection data for each substance. Furthermore, function 112 generates a learned model M1 corresponding to "100 kVp" through machine learning based on these multiple orthographic projection data and the original data R1 used in the generation of the X-ray CT image I1. Similarly, function 112 generates learned models M1 corresponding to various X-ray energies.

[0118] Then, after obtaining the X-ray CT image I2, the image processing function 342 uses the learned model M1, which is obtained in accordance with the spectral information at the time of the X-ray CT image I2's imaging, to obtain the raw data R3 corresponding to any spectral information. For example, if the X-ray CT image I2 is an image taken at "120kVp", the image processing function 342 uses the learned model M1 corresponding to "120kVp" to obtain the raw data R3.

[0119] As mentioned above, although the coefficients “c1”, “c2”, and “c3” learned in the completed learning model M1 have low dependence on X-ray energy (kVp), the situation where the influence of X-ray energy cannot be ignored is still considered. In contrast, by generating the completed learning model M1 corresponding to various X-ray energies in advance, the influence of X-ray energy during imaging can be reduced, and the conversion to other X-ray energies (kVp) can be performed with higher accuracy.

[0120] Alternatively, instead of generating separate learning models M1 corresponding to various X-ray energies, the X-ray energy at the time of X-ray CT image I1 can be used as learning data. In this case, in addition to the multiple forward projection data based on X-ray CT image I2 mentioned above, image processing function 342 also inputs the X-ray energy at the time of X-ray CT image I2 at the time of X-ray CT image I2 into the learning model M1. As a result, the learning model M1 can take into account the influence of the X-ray energy at the time of imaging and perform conversion processing to other X-ray energies (kVp) with higher accuracy.

[0121] (Third Implementation)

[0122] In the first embodiment described above, an example was described of obtaining an X-ray CT image (the second CT image) corresponding to spectral information of X-rays different from that at the time of imaging, using a learning-completed model M1 that accepts input from multiple forward projection data of each substance and outputs raw data. In contrast, in the third embodiment, an example was described of obtaining the second CT image using a learning-completed model M2 that accepts input from the CT image and outputs distribution data of each substance. In other words, in the third embodiment, an example was described of obtaining the second CT image using a learning-completed model M2 that performs computer segmentation processing. The differences from the above embodiments will be explained below; descriptions of configurations identical to those in the above embodiments will be marked with the same reference numerals and will omit further explanation.

[0123] use Figure 8 This describes the learning process performed by the medical information processing device 1 to generate model M2. Figure 8 This is a diagram used to illustrate the learning phase involved in the third embodiment.

[0124] First, radiography is performed on any subject to acquire raw data R4. Next, reconstruction processing based on the raw data R4 is performed to generate an X-ray CT image I4. The acquisition function 111 acquires this X-ray CT image I4. The reconstruction processing of the X-ray CT image I4 can be performed either by the acquisition function 111 or in other devices.

[0125] Next, function 111 segments the acquired X-ray CT image I4 by each substance, obtaining the distribution data of the substances in X-ray CT image I4. For example, function 111 segments the X-ray CT image I4 by each internal organ. For example, in Figure 8 In the process, function 111 is used to segment the X-ray CT image I4 according to each substance, and obtain distribution data D44, distribution data D45 and distribution data D46.

[0126] Here, the acquisition function 111 segments the X-ray CT image I4 with high precision using manual or semi-manual methods. For example, the acquisition function 111 displays the X-ray CT image I4 and accepts input from users such as physicians who have referenced the X-ray CT image I4, thereby acquiring the distribution data of substances in the X-ray CT image I4. Furthermore, for example, the acquisition function 111 segments the X-ray CT image I4 using any method based on CT values, such as Otsu binarization, region dilation, Snake's method, graph cut, or mean shift, and then displays the segmentation results to the user. Then, the acquisition function 111 receives corrections from the user regarding the segmentation results, thereby acquiring the distribution data of substances in the X-ray CT image I4.

[0127] Next, learning function 112 performs machine learning using learning data consisting of a combination of X-ray CT image I4 and distribution data D44, distribution data D45 and distribution data D46, thereby generating a completed learning model M2.

[0128] exist Figure 8 In the example of the learned model M2, neural networks NN1, NN2, and NN3 are represented. These multiple learned models M2 are generated for each substance. Specifically, learning function 112 performs machine learning using learning data to generate neural network NN1, which consists of a combination of X-ray CT image I4 and distribution data D44 obtained by segmenting the substance corresponding to the attenuation coefficient μ1 from X-ray CT image I4. Furthermore, learning function 112 performs machine learning using learning data to generate neural network NN2, which consists of a combination of X-ray CT image I4 and distribution data D45 obtained by segmenting the substance corresponding to the attenuation coefficient μ2 from X-ray CT image I4. Finally, learning function 112 performs machine learning using learning data to generate neural network NN3, which consists of a combination of X-ray CT image I4 and distribution data D46 obtained by segmenting the substance corresponding to the attenuation coefficient μ3 from X-ray CT image I4.

[0129] Next, using Figure 9 This section describes the utilization phase of model M2 after learning. First, photography is performed on any subject to collect raw data R5. Raw data R5 can be collected from the same subject as raw data R4, or from other subjects. Furthermore, raw data R5 can be obtained from... Figure 2 The X-ray CT device 2 shown can acquire the image, but other X-ray CT devices can also be used. Next, a reconstruction process based on the original data R5 is performed to generate the X-ray CT image I5. The acquisition function 341 acquires the X-ray CT image I5. The reconstruction of the X-ray CT image I5 can be performed either by the acquisition function 341 or by other devices.

[0130] Next, the acquisition function 341 uses the learned completion model M2 to segment the acquired X-ray CT image I5 by each substance, thereby obtaining the distribution data of the substances in the X-ray CT image I5. For example, the acquisition function 341 inputs the X-ray CT image I4 into the neural network NN1 to obtain the distribution data D51 of the substances corresponding to the attenuation coefficient μ1. Additionally, the acquisition function 341 inputs the X-ray CT image I4 into the neural network NN2 to obtain the distribution data D52 of the substances corresponding to the attenuation coefficient μ2. Furthermore, the acquisition function 341 inputs the X-ray CT image I4 into the neural network NN3 to obtain the distribution data D53 of the substances corresponding to the attenuation coefficient μ3.

[0131] Next, image processing function 342, based on arbitrary spectral information and the attenuation coefficient of each substance, performs forward projection processing on each distribution data to obtain multiple forward projection data for each substance. For example, in Figure 9 In the case shown, image processing function 342 performs forward projection processing on distribution data D51 to obtain forward projection data P51, performs forward projection processing on distribution data D52 to obtain forward projection data P52, and performs forward projection processing on distribution data D53 to obtain forward projection data P53.

[0132] Here, any spectral information can differ from the spectral information at the time of X-ray CT image I5 imaging. For example, in Figure 9 In the image, X-ray CT image I5 is an image taken using X-rays at "120 kVp". In contrast, the arbitrary spectral information set to obtain orthographic projection data is "135 kVp". Furthermore, image processing function 342 synthesizes multiple orthographic projection data for each substance to obtain the raw data R6. Figure 9 In the case shown, the original data R6 corresponds to the arbitrary spectral information "135kVp".

[0133] exist Figure 9 In this process, based on the distribution data corresponding to the spectral information "120kVp" at the time of imaging, forward projection data corresponding to any spectral information "135kVp" is obtained. Here, forward projection processing is also considered for the unsegmented X-ray CT image I5, obtaining the original data corresponding to "135kVp". However, in this case, the calculation becomes complex because forward projection processing is performed on the X-ray CT image I5, which contains a mixture of various substances.

[0134] In contrast, Figure 9 In the example, by performing forward projection processing based on segmentation of each substance, the calculations can be simplified and the accuracy of the conversion processing to other X-ray energies (kVp) can be improved. Furthermore, in Figure 9In this study, by utilizing the learned model M2 to perform high-precision segmentation, the premise that the segmented distribution data is a single substance becomes reliable, further improving the accuracy of the conversion process.

[0135] Next, use Figure 10A and Figure 10B Explanation in Figure 8 The medical information processing device 1 performs a series of processes during the learning phase as described in the text. Figure 10A and Figure 10B This is a flowchart illustrating the learning phase processing performed by the medical information processing device 1 according to the third embodiment.

[0136] First, the acquisition function 111 acquires raw data (step S311), and reconstructs an X-ray CT image based on the raw data (step S312). Alternatively, the acquisition function 111 can replace steps S311 and S312 to acquire an X-ray CT image reconstructed by another device. Next, the acquisition function 111 segments the acquired X-ray CT image by each substance, acquiring distribution data for each substance (step S313). Here, the acquisition function 111 segments the X-ray CT image with high precision using a manual or semi-manual method. Then, the acquisition function 111 associates the X-ray CT image from step S312 with the distribution data for each substance from step S313 as a set of learning data and stores it in the memory 12 (step S314).

[0137] Function 111 executes each time new raw data is obtained. Figure 10A The series of steps shown. As a result, the learning data of step S314 is stored in memory 12. Learning function 112 retrieves multiple sets of learning data from memory 12 (step S321), and generates a learning completion model M2 for each substance by performing machine learning using the multiple sets of learning data (step S322).

[0138] Next, the explanation is in Figure 9 The third embodiment describes a series of processes performed by the medical information processing device 3 during the utilization phase. Furthermore, the utilization phase described in the third embodiment is the same as the utilization phase described in the first embodiment, as shown in the flowchart. Therefore, using... Figure 6 The utilization stage involved in the third embodiment is explained.

[0139] First, function 341 acquires raw data (step S211), and reconstructs an X-ray CT image based on the raw data (step S212). For example, in step S212, function 341 acquires raw data based on the raw data. Figure 4The original data R2 shown is used to reconstruct the X-ray CT image I2. Alternatively, function 341 can replace steps S211 and S212 to obtain the X-ray CT image I2 reconstructed by other devices.

[0140] Next, the acquisition function 341 segments the acquired X-ray CT image for each substance to obtain the distribution data of the substances (step S213). That is, the acquisition function 341 applies computer segmentation processing to the first CT image to obtain the distribution data of multiple substances. Here, the acquisition function 341 performs computer segmentation processing based on a learning completion model M2 using machine learning. In other words, in the computer segmentation processing according to the third embodiment, the distribution data of multiple substances is obtained from the first CT image based on the learning completion model M2.

[0141] Next, image processing function 342 performs forward projection processing on the distribution data of the substances based on arbitrary spectral information and the attenuation coefficient of each substance, obtaining multiple forward projection data for each substance (step S214). For example, image processing function 342 as follows: Figure 9 As shown, based on the arbitrary spectral information "135kVp", forward projection processing is performed on the distribution data D51 to obtain forward projection data P51, forward projection processing is performed on the distribution data D52 to obtain forward projection data P52, and forward projection processing is performed on the distribution data D53 to obtain forward projection data P53.

[0142] Then, image processing function 342 synthesizes the multiple forward projection data from step S214 to obtain the original data corresponding to arbitrary spectral information (step S215). For example, image processing function 342 as follows: Figure 9 As shown, the forward projection data P51, forward projection data P52 and forward projection data P53 are synthesized to obtain the original data R6 corresponding to any spectral information "135kVp".

[0143] Then, the image processing function 342 generates an X-ray CT image corresponding to the arbitrary spectral information based on the raw data obtained in step S215 (step S216). The X-ray CT image corresponding to the arbitrary spectral information is output for display or analysis by the user (step S217).

[0144] Furthermore, similar to the case of the learned model M1, a learned model M2 can be generated based on each of the various conditions, such as the location of the radiograph or patient information. In the generation process of the learned model M2, information such as the location of the radiograph or patient information can also be used as learning data. Alternatively, the second embodiment described above can be combined with this embodiment. That is, the learned model M2 can also be generated based on each X-ray energy (kVp), and in the generation process of the learned model M2, X-ray energy can also be used as learning data.

[0145] (Fourth implementation)

[0146] In the fourth embodiment, an example is described where a learning completion model M3, which accepts the input of distribution data for each substance and outputs distribution data corresponding to arbitrary spectral information, is used to obtain an X-ray CT image (the second CT image) corresponding to spectral information of X-rays different from that at the time of imaging. The differences from the embodiments described above will be explained below; for descriptions of configurations identical to those described above, the same reference numerals will be added and descriptions will be omitted.

[0147] use Figure 11 This describes the learning process performed by the medical information processing device 1 to generate model M3. Figure 11 This is a diagram used to illustrate the learning phase involved in the fourth embodiment.

[0148] First, radiography is performed on any subject to acquire raw data R7. Next, reconstruction processing based on the raw data R7 is performed to generate an X-ray CT image I7. The acquisition function 111 acquires this X-ray CT image I7. The reconstruction processing of the X-ray CT image I7 can be performed either by the acquisition function 111 or in other devices.

[0149] Next, function 111 segments the acquired X-ray CT image I7 for each substance, obtaining the distribution data of the substances in the X-ray CT image I7. For example, in Figure 11 In this process, function 111 segments the X-ray CT image I7 for each substance, obtaining distribution data D71, distribution data D72, and distribution data D73. Here, function 111 can perform segmentation manually or semi-manually, or automatically using methods such as Otsu's binarization method based on CT values.

[0150] exist Figure 11In the diagram, distribution data D71–D73 correspond to the X-ray energy “120 kVp” during imaging. Learning function 112 performs machine learning using the following learning data to generate a completed learning model M3, which consists of a combination of distribution data D71–D73 and distribution data D74–D76, which correspond to the X-ray energy “135 kVp” and are different from distribution data D71–D73.

[0151] exist Figure 11 In the example, model M3 is used to represent neural networks NN4 to NN6. Figure 11 In the illustrated case, neural network NN4 is functionalized to accept input data corresponding to the distribution data of the substance with X-ray energy "120 kVp" and attenuation coefficient μ1, and outputs the distribution data of the substance with X-ray energy "135 kVp". Similarly, neural network NN5 is functionalized to accept input data corresponding to the distribution data of the substance with X-ray energy "120 kVp" and attenuation coefficient μ2, and outputs the distribution data of the substance with X-ray energy "135 kVp". Furthermore, neural network NN6 is functionalized to accept input data corresponding to the distribution data of the substance with X-ray energy "120 kVp" and attenuation coefficient μ3, and output the distribution data of the substance with X-ray energy "135 kVp".

[0152] For example, by segmenting X-ray CT images, distribution data D74–D76 can be obtained. These X-ray CT images are obtained from CT scans corresponding to X-ray energies of “135 kVp”. Figure 11 The X-ray CT images were acquired from the same subject as I7.

[0153] use Figure 12 Explain other methods for obtaining distribution data D74 to D76. Figure 12 This diagram illustrates an example of the method for acquiring learning data according to the fourth embodiment. First, the acquisition function 111 is as follows... Figure 11 As explained in the text, the X-ray CT image I7 was segmented according to each substance to obtain the distribution data D71~D73 corresponding to the X-ray energy "120kVp" at the time of imaging.

[0154] Next, function 111 performs forward projection processing based on spectral information different from that during photography, obtaining forward projection data P71 to P73. For example, in Figure 12In the process, function 111 obtains forward projection data P71 by performing forward projection processing on distribution data D71 based on the spectral information "135kVp" which is different from that at the time of photography, and forward projection processing on distribution data D72 to obtain forward projection data P72, and forward projection processing on distribution data D73 to obtain forward projection data P73.

[0155] Then, function 111 reconstructs the data for each forward projection P71 to P73 to obtain the distribution data D74 to D76 corresponding to the spectral information "135kVp". Specifically, in Figure 12 In this process, function 111 reconstructs distribution data D74 based on forward projection data P71, reconstructs distribution data D75 based on forward projection data P72, and reconstructs distribution data D76 based on forward projection data P73. Furthermore, the reconstruction method is not particularly limited; function 111 can reconstruct distribution data D74–D76 using any method such as the FBP method, successive approximation reconstruction method, successive approximation application reconstruction method, and DLR method.

[0156] Next, use Figure 13 This section describes the utilization phase of model M3 after learning. First, photography is performed on any subject to collect raw data R8. Raw data R8 can be collected from the same subject as raw data R7, or from other subjects. Furthermore, raw data R8 can be obtained from... Figure 2 The X-ray CT device 2 shown can acquire the image, but other X-ray CT devices can also be used. Next, a reconstruction process based on the original data R8 is performed to generate the X-ray CT image I8. The acquisition function 341 acquires the X-ray CT image I8. The reconstruction of the X-ray CT image I8 can be performed by the acquisition function 341 or by other devices.

[0157] Next, the acquisition function 341 segments the acquired X-ray CT image I8 for each substance to obtain the distribution data of the substances in the X-ray CT image I8. For example, the acquisition function 341 segments the X-ray CT image I8 for each substance to obtain the distribution data D81 of the substance corresponding to the attenuation coefficient μ1, the distribution data D82 of the substance corresponding to the attenuation coefficient μ2, and the distribution data D83 of the substance corresponding to the attenuation coefficient μ3.

[0158] Next, image processing function 342 inputs distribution data D81 to D83 corresponding to the X-ray energy "120kVp" during imaging into the learned model M3, thereby obtaining distribution data D91 to D93 corresponding to the X-ray energy "135kVp" different from that during imaging. Specifically, image processing function 342 inputs distribution data D81 into neural network NN4 to obtain distribution data D91. Additionally, image processing function 342 inputs distribution data D82 into neural network NN5 to obtain distribution data D92. Furthermore, image processing function 342 inputs distribution data D83 into neural network NN6 to obtain distribution data D93.

[0159] That is, in the conversion process involved in the fourth embodiment, by applying the learning completion model M3 to each distribution data D81 to D83, distribution data D91 to D93 can be obtained. Distribution data D91 to D93 are an example of the processed distribution data of multiple substances corresponding to the second spectral information.

[0160] Furthermore, the image processing function 342 synthesizes the distribution data D91 to D93 to obtain an X-ray CT image I9 corresponding to an X-ray energy "135kVp" that is different from that during imaging. That is, by synthesizing the processed distribution data of multiple substances, a second CT image corresponding to the second spectral information can be obtained.

[0161] Furthermore, it was also considered to use groups of X-ray CT images with different X-ray energies as training data for machine learning, and to directly convert X-ray CT images corresponding to X-ray energies of "120 kVp" to those corresponding to "135 kVp". However, in this case, the conversion process involves X-ray CT images containing a mixture of various substances, thus complicating the calculations. In contrast, in Figures 11-13 In this example, the learning and utilization phases are performed separately based on the division of each substance. This simplifies the calculations and improves the accuracy of the conversion processing to other X-ray energies (kVp).

[0162] Next, use Figure 14A and Figure 14B Explanation in Figure 11 The medical information processing device 1 performs a series of processes during the learning phase as described in the text. Figure 14A and Figure 14B This is a flowchart illustrating the learning phase processing performed by the medical information processing device 1 according to the fourth embodiment. Furthermore, in Figure 14A and Figure 14B In the middle, explanation and Figure 12The example shown uses different spectral information than that used during photography to perform forward projection and reconstruction processing to obtain a set of learning data.

[0163] First, the acquisition function 111 acquires raw data (step S411), and reconstructs an X-ray CT image based on the raw data (step S412). Alternatively, the acquisition function 111 can replace steps S411 and S412 to acquire an X-ray CT image reconstructed by another device. Next, the acquisition function 111 segments the acquired X-ray CT image by each substance, acquiring distribution data for each substance (step S413). Next, the acquisition function 111 performs forward projection processing on the distribution data acquired in step S413 using spectral information different from that used during imaging (step S414), and then reconstructs distribution data corresponding to the different spectral information used during imaging based on the acquired forward projection data (step S415). Then, the acquisition function 111 establishes a correlation between the distribution data from step S413 and the distribution data from step S415 as a set of learning data and stores it in the memory 12 (step S314).

[0164] Function 111 executes each time new raw data is obtained. Figure 14A The series of steps shown. As a result, the learning data of step S416 is stored in memory 12. Learning function 112 retrieves multiple sets of learning data from memory 12 (step S421), and generates a completed learning model M3 for each combination of X-ray energy (kVp) by performing machine learning using the multiple sets of learning data (step S422).

[0165] Next, use Figure 15 Explanation in Figure 13 The medical information processing device 3 performs a series of processes during the utilization phase as described in the text. Figure 15 This is a flowchart illustrating the processing performed during the utilization phase of the medical information processing device 3 according to the fourth embodiment.

[0166] First, function 341 acquires raw data (step S511), and then reconstructs an X-ray CT image based on the raw data (step S512). For example, in step S512, function 341 acquires raw data based on the raw data. Figure 13 The original data R8 shown is used to reconstruct the X-ray CT image I8. Alternatively, the acquisition function 341 can replace steps S511 and S512 to acquire the X-ray CT image I8 reconstructed by another device. Next, the acquisition function 341 segments the acquired X-ray CT image by each substance to obtain the distribution data of the substances (step S513).

[0167] Next, image processing function 342 uses the learned model M3 to perform transformation processing on the distribution data obtained in step S513 (step S514). For example, image processing function 342 as follows: Figure 13 As shown, neural networks NN4 to NN6 are used to convert the distribution data D81 to D83 corresponding to the X-ray energy "120kVp" into distribution data D91 to D93 corresponding to the X-ray energy "135kVp".

[0168] Then, image processing function 342 synthesizes the multiple distribution data from step S514 to obtain an X-ray CT image corresponding to a different X-ray energy than that used during imaging (step S515). For example, image processing function 342... Figure 13 As shown, the distribution data D91 to D93 were combined to obtain an X-ray CT image I9 corresponding to an X-ray energy "135kVp" that was different from that during the imaging.

[0169] In addition, similar to the case of learning and completing model M1, learning and completing model M3 can also be generated according to each of various conditions such as the location of the photographic object or patient information. In the process of generating learning and completing model M3, information such as the location of the photographic object or patient information can also be used as learning data.

[0170] (Other implementation methods)

[0171] exist Figure 2 The X-ray CT unit 2 and the medical information processing unit 3 are represented separately, but they can also be integrated. For example, the control console included in the X-ray CT unit 2 can function as the medical information processing unit 3 and perform [operations]. Figure 4 The utilization stage is shown.

[0172] In addition, Figure 1 and Figure 2 The document describes a scenario where the learning phase is performed by medical information processing device 1 and the utilization phase is performed by medical information processing device 3. However, both the learning and utilization phases can also be performed by the same device. That is, they can also be performed by either medical information processing device 3 or X-ray CT device 2. Figure 3 The learning stages are shown.

[0173] The term "processor" used in the above description refers to circuits such as CPU, GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), and Programmable Logic Device (SPLD, CPLD, and FPGA). When the processor is, for example, a CPU, the processor implements its function by reading and executing a program stored in a memory circuit. On the other hand, when the processor is, for example, an ASIC, the program is not stored in a memory circuit; instead, its function is directly assembled into the processor's circuitry as logic circuitry. Furthermore, the processors in the embodiments are not limited to being configured as a single circuit; multiple independent circuits can be combined to form a single processor and implement its function. Moreover, multiple constituent elements in each figure can be integrated into a single processor and its function implemented.

[0174] In addition, Figure 1 This section describes how a single memory 12 stores the programs corresponding to each processing function of the processing circuit 11. Additionally, in... Figure 2 The text describes how a single memory 31 stores programs corresponding to the various processing functions of the processing circuit 34. However, the implementation is not limited to this. For example, multiple memories 12 may be distributed, with the processing circuit 11 reading the corresponding program from each individual memory 12. Similarly, multiple memories 31 may be distributed, with the processing circuit 34 reading the corresponding program from each individual memory 31. Alternatively, instead of storing the program in memory 12 or memory 31, the program may be directly loaded into the processor's circuitry. In this case, the processor performs the function by reading and executing the program loaded into the circuitry.

[0175] The constituent elements of the devices involved in the above-described embodiments are functional conceptual elements and do not necessarily need to be physically configured as shown in the figures. That is, the specific manner in which the devices are distributed / integrated is not limited to the figures, and they can be configured, in any unit, functionally or physically distributed / integrated, depending on various loads or usage conditions. Furthermore, all or any part of the processing functions performed by each device can be implemented by a CPU and the program parsed and executed in that CPU, or as hardware implementation based on wiring logic.

[0176] Furthermore, the medical information processing method described in the above embodiments can be implemented by executing a pre-prepared program using a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet. Alternatively, the program can be recorded on a computer-readable non-volatile recording medium such as a hard disk, floppy disk (FD), CD-ROM, MO, or DVD, and executed by a computer from the recording medium.

[0177] According to at least one embodiment described above, it is possible to obtain an X-ray CT image corresponding to spectral information of X-rays that are different from those during imaging.

[0178] Several embodiments have been described above, but these embodiments are given as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope or spirit of the invention, and are included in the scope of the invention as described in the claims and its equivalents.

Claims

1. A medical information processing method, wherein, Includes the following steps: An X-ray CT image and its spectral information at the time of imaging are obtained. The X-ray CT image is segmented for each substance to obtain the distribution data of the substances in the X-ray CT image. Based on the spectral information and the attenuation coefficient of each substance, forward projection processing is performed on the distribution data to obtain multiple forward projection data for each substance. A learning-complete model is generated through machine learning based on the multiple forward projection data and the raw data used in the generation of the X-ray CT images.

2. The medical information processing method as described in claim 1, wherein, The plurality of forward projection data are used as input data, and the original data is used as output data, which are then input into the neural network. The neural network is trained to minimize the error between the sum of the multiple forward projection data and the original data, thereby generating the completed learning model.

3. The medical information processing method as described in claim 2, wherein, It also includes the following steps: The learned model, other X-ray CT images different from the X-ray CT image, and arbitrary spectral information are obtained. The other X-ray CT images are segmented for each substance to obtain the distribution data of the substances in those other X-ray CT images. Based on the arbitrary spectral information and the attenuation coefficient of each substance, forward projection processing is performed on the distribution data to obtain multiple forward projection data for each substance. Multiple forward projection data based on the other X-ray CT images are input into the learned model to obtain the original data corresponding to the arbitrary spectral information.

4. The medical information processing method as described in claim 3, wherein, The arbitrary spectral information is set based on the X-ray energy value input by the user.

5. A medical information processing device, wherein, have: The acquisition unit acquires an X-ray CT image and the spectral information of the X-ray CT image during imaging. It segments the X-ray CT image according to each substance to obtain the distribution data of the substances in the X-ray CT image. Based on the spectral information and the attenuation coefficient of each substance, it performs forward projection processing on the distribution data to obtain multiple forward projection data of each substance. as well as The learning unit generates a completed learning model through machine learning based on the multiple forward projection data and the raw data used in the generation of the X-ray CT images.

6. A medical information processing method, wherein, A first CT image of the subject, obtained by performing a CT scan corresponding to the first spectral information, is then processed using computer segmentation to obtain distribution data of multiple substances. Based on the distribution data of the multiple substances, a conversion process is performed to obtain a second CT image corresponding to the second spectral information. The second CT image is output for display or analysis. In the transformation process, for each distribution data of the multiple substances, a forward projection process based on the second spectral information is performed to obtain multiple forward projection data. A machine learning model is applied to the multiple forward projection data to obtain the original data corresponding to the second spectral information. Based on this original data, the second CT image was reconstructed.

7. The medical information processing method as described in claim 6, wherein, In the computer segmentation process, distribution data of multiple substances are obtained from the first CT image based on a learning completion model utilizing machine learning.

Images