Estimation device, method, and program

Abstract

To accurately predict a locomotorium disease in an estimation device, method, and program.SOLUTION: An estimation device includes at least one processor. The processor functions as a learned neural network for deriving an estimation result of at least one emphasis image that emphasizes a specific composition of a subject from a plain two-dimensional image acquired by executing plain radiography of the subject including a plurality of compositions. The learned neural network is learned by using, as teacher data, a composite two-dimensional image showing the subject derived by compositing three-dimensional CT images of the subject, and an emphasis image for learning that emphasizes a specific composition of the subject derived from the CT image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an estimation device, a method, and a program. [Background technology]

[0002] Conventionally, energy subtraction processing has been known, which uses two radiation images obtained by irradiating a subject with two types of radiation with different energy distributions, taking advantage of the fact that the attenuation of transmitted radiation differs depending on the material that constitutes the subject. The energy subtraction processing is a method of obtaining an image in which a specific structure is emphasized by matching each pixel of the two radiation images obtained as described above, multiplying the pixels by an appropriate weighting coefficient, and then performing subtraction. In addition to bones and soft tissues, the composition of the human body, such as fat and muscle in the soft tissues, is also derived by energy subtraction processing (see Patent Document 1).

[0003] In addition, various methods have been proposed for deriving a radiological image different from the radiological image acquired by photographing a subject, using the radiological image acquired by photographing the subject. For example, Patent Document 2 proposes a method for deriving a bone image from a radiological image of a subject acquired by plain radiography, by using a trained model constructed by training a neural network using a radiological image of the subject acquired by plain radiography and a bone image of the same subject as training data.

[0004] Note that plain radiography is a method of radiography in which a subject is irradiated with radiation once to obtain a single two-dimensional image that is a transmission image of the subject. In the following description, a two-dimensional image obtained by plain radiography will be referred to as a plain two-dimensional image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2018-153605 A [Patent Document 2] U.S. Patent No. 7,545,965 Summary of the Invention [Problem to be solved by the invention]

[0006] However, it is desirable to estimate images that emphasize specific components of bones and the like with higher accuracy.

[0007] The present disclosure has been made in consideration of the above circumstances, and has an object to make it possible to estimate with high accuracy an image in which a specific composition is emphasized. [Means for solving the problem]

[0008] The present disclosure provides an estimation apparatus comprising at least one processor, The processor functions as a trained neural network that derives an estimation result of at least one enhanced image in which a specific composition of the subject is enhanced from a simple two-dimensional image acquired by simply photographing the subject including a plurality of compositions; The trained neural network is trained using as training data a synthetic 2D image representing the subject derived by synthesizing 3D CT images of the subject, and a training enhanced image which enhances specific components of the subject derived from the CT images.

[0009] In addition, in the estimation device according to the present disclosure, the composite two-dimensional image may be derived by deriving the radiation attenuation coefficient for a composition at each position in three-dimensional space, and projecting a CT image in a predetermined direction based on the attenuation coefficient.

[0010] Furthermore, in the estimation device according to the present disclosure, the training emphasized image may be derived by identifying an area of ​​a specific composition in a CT image and projecting the CT image of the area of ​​the specific composition in a predetermined direction.

[0011] Furthermore, in the estimation device according to the present disclosure, the learning emphasized image may be derived by performing weighted subtraction of two composite two-dimensional images that simulate imaging of a subject using radiation with different energy distributions, the composite two-dimensional images being derived by projecting a CT image in a predetermined direction.

[0012] In the estimation device according to the present disclosure, the specific composition may be at least one of a soft part, a bone part, a muscle, and a fat part of the subject.

[0013] The estimation method according to the present disclosure is an estimation method for deriving an estimation result of at least one emphasized image in which a specific composition of a subject is emphasized from a plain radiographic image, using a trained neural network that derives an estimation result of at least one emphasized image in which a specific composition of the subject is emphasized from a plain radiographic image acquired by plain radiographing a subject including a plurality of compositions, the estimation method comprising: The trained neural network is trained using as training data a synthetic 2D image representing the subject derived by synthesizing 3D CT images of the subject, and a training enhanced image which enhances specific components of the subject derived from the CT images.

[0014] The estimation method according to the present disclosure may be provided as a program for causing a computer to execute the method. Effect of the Invention

[0015] According to the present disclosure, it is possible to estimate with high accuracy an image in which a specific composition is emphasized. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic block diagram showing a configuration of a radiation image capturing system to which an estimation device according to a first embodiment of the present disclosure is applied; [Diagram 2] FIG. 1 is a diagram showing a schematic configuration of an estimation device according to a first embodiment. [Diagram 3] FIG. 1 is a diagram showing a functional configuration of an estimation device according to a first embodiment. [Figure 4]FIG. 1 is a diagram showing a schematic configuration of a neural network used in the present embodiment; [Diagram 5] Diagram showing training data [Figure 6] FIG. 1 is a diagram showing a schematic configuration of an information derivation device according to a first embodiment; [Figure 7] FIG. 1 is a diagram showing a functional configuration of an information derivation device according to a first embodiment; [Figure 8] FIG. 1 is a diagram for explaining derivation of a synthetic two-dimensional image. [Figure 9] FIG. 1 is a diagram for explaining derivation of a synthetic two-dimensional image. [Figure 10] Diagram to explain CT value [Figure 11] FIG. 1 is a diagram for explaining derivation of a bone image; [Figure 12] Figure showing bone images [Figure 13] FIG. 1 is a diagram for explaining derivation of a soft tissue image. [Figure 14] Soft tissue images [Figure 15] Diagram to explain neural network learning [Figure 16] Conceptual diagram of the processing performed by a trained neural network [Figure 17] A diagram showing the display screen of the estimation results [Figure 18] 1 is a flowchart of a learning process performed in the first embodiment. [Figure 19] 1 is a flowchart of an estimation process performed in the first embodiment. [Figure 20] FIG. 1 is a diagram for explaining derivation of muscle images. [Figure 21] Diagram showing muscle images [Figure 22] FIG. 1 is a diagram for explaining derivation of a fat image. [Diagram 23] Diagram showing fat image [Figure 24] Another example of training data [Diagram 25] FIG. 13 is a diagram showing a functional configuration of an information derivation device according to a third embodiment. [Figure 26] FIG. 13 is a diagram for explaining derivation of a composite two-dimensional image in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic block diagram showing a configuration of a radiographic image capturing system to which an estimation device according to a first embodiment of the present disclosure is applied. As shown in Fig. 1, the radiographic image capturing system according to the first embodiment includes an imaging device 1, a CT device 7, an image storage system 9, an estimation device 10 according to the first embodiment, and an information derivation device 50. The imaging device 1, the CT (Computed Tomography) device 7, the estimation device 10, and the information derivation device 50 are connected to the image storage system 9 via a network (not shown).

[0018] The imaging device 1 is an imaging device capable of acquiring a simple radiological image G0 of the subject H by irradiating a radiation detector 5 with radiation such as X-rays that is emitted from a radiation source 3 and passes through the subject H. The acquired simple radiological image G0 is input to the estimation device 10. The simple radiological image G0 is, for example, a front image including the crotch area of ​​the subject H.

[0019] The radiation detector 5 is capable of repeatedly recording and reading out radiation images, and may be a so-called direct type radiation detector that generates electric charges by directly receiving radiation, or a so-called indirect type radiation detector that converts radiation into visible light and then converts the visible light into an electric charge signal. As a method for reading out the radiation image signal, it is preferable to use a so-called TFT readout method in which the radiation image signal is read out by turning on and off a TFT (thin film transistor) switch, or a so-called optical readout method in which the radiation image signal is read out by irradiating it with readout light, but this is not limiting and other methods may also be used.

[0020] The CT device 7 performs CT imaging of the subject H to obtain a plurality of tomographic images representing a plurality of tomographic planes in the subject H as a three-dimensional CT image V0. The CT value of each pixel (voxel) in the CT image is a numerical representation of the radiation absorption rate in the components that make up the human body. The CT value will be described later.

[0021] The image storage system 9 is a system that stores image data of radiographic images acquired by the imaging device 1 and image data of CT images acquired by the CT device 7. The image storage system 9 extracts images from the stored radiographic images and CT images in response to requests from the estimation device 10 and the information derivation device 50, and transmits the images to the requesting device. A specific example of the image storage system 9 is a PACS (Picture Archiving and Communication System). In this embodiment, the image storage system 9 stores a large amount of training data for training a neural network, which will be described later.

[0022] Next, an estimation device according to a first embodiment will be described. First, a hardware configuration of the estimation device according to the first embodiment will be described with reference to FIG. 2. As shown in FIG. 2, the estimation device 10 is a computer such as a workstation, a server computer, or a personal computer, and includes a CPU (Central Processing Unit) 11, a non-volatile storage 13, and a memory 16 as a temporary storage area. The estimation device 10 also includes a display 14 such as a liquid crystal display, an input device 15 such as a keyboard and a mouse, and a network I / F (InterFace) 17 connected to a network (not shown). The CPU 11, the storage 13, the display 14, the input device 15, the memory 16, and the network I / F 17 are connected to a bus 18. The CPU 11 is an example of a processor in the present disclosure.

[0023] The storage 13 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 13 as a storage medium stores the estimation program 12A and the learning program 12B installed in the estimation device 10. The CPU 11 reads out the estimation program 12A and the learning program 12B from the storage 13, expands them in the memory 16, and executes the expanded estimation program 12A and the learning program 12B.

[0024] The estimation program 12A and the learning program 12B are stored in a storage device of a server computer connected to a network or in a network storage in an externally accessible state, and are downloaded and installed in the computer constituting the estimation device 10 upon request. Alternatively, they are recorded on a recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and are installed in the computer constituting the estimation device 10 from the recording medium.

[0025] Next, a functional configuration of the estimation device according to the first embodiment will be described. FIG. 3 is a diagram showing the functional configuration of the estimation device according to the first embodiment. As shown in FIG. 3, the estimation device 10 includes an image acquisition unit 21, an information acquisition unit 22, an estimation unit 23, a learning unit 24, and a display control unit 25. The CPU 11 executes an estimation program 12A to function as the image acquisition unit 21, the information acquisition unit 22, the estimation unit 23, and the display control unit 25. The CPU 11 executes a learning program 12B to function as the learning unit 24.

[0026] The image acquiring unit 21 causes the imaging device 1 to perform simple imaging of the subject H, thereby acquiring from the radiation detector 5 a simple radiation image G0 which is, for example, a front image of the crotch area of ​​the subject H. When acquiring the simple radiation image G0, imaging conditions are set, such as the imaging dose, radiation quality, tube voltage, SID (Source Image Receptor Distance) which is the distance between the radiation source 3 and the surface of the radiation detector 5, SOD (Source Object Distance) which is the distance between the radiation source 3 and the surface of the subject H, and the presence or absence of an anti-scatter grid.

[0027] The imaging conditions may be set by the operator through input from the input device 15. The set imaging conditions are stored in the storage 13. The plain radiographic image G0 and the imaging conditions are also transmitted to the image storage system 9 and stored therein.

[0028] In this embodiment, the simple radiographic image G0 may be acquired by a program separate from the estimation program 12A and stored in the storage 13. In this case, the image acquisition unit 21 acquires the simple radiographic image G0 stored in the storage 13 by reading it out from the storage 13 for processing.

[0029] The information acquisition unit 22 acquires teacher data for training a neural network, which will be described later, from the image storage system 9 via the network I / F 17.

[0030] The estimation unit 23 derives an estimation result of a bone image in which the bones contained in the subject H are emphasized and a soft tissue image in which the soft tissue is emphasized from the simple radiographic image G0. For this purpose, the estimation unit 23 derives the estimation results of the bone image and the soft tissue image using a trained neural network 23A that outputs a bone image and a soft tissue image when the simple radiographic image G0 is input. In this embodiment, the subject from which the estimation results of the bone image and the soft tissue image are derived is an image near the hip joint of the subject H, but is not limited to this. In addition, the high-grain image and the soft tissue image derived by the estimation unit 23 are an example of an emphasized image.

[0031] The learning unit 24 constructs a trained neural network 23A by machine learning the neural network using the teacher data. Examples of the neural network include a simple perceptron, a multilayer perceptron, a deep neural network, a convolutional neural network, a deep belief network, a recurrent neural network, and a probabilistic neural network. In this embodiment, a convolutional neural network is used as the neural network.

[0032] Fig. 4 is a diagram showing a neural network used in this embodiment. As shown in Fig. 4, the neural network 30 includes an input layer 31, an intermediate layer 32, and an output layer 33. The intermediate layer 32 includes, for example, a plurality of convolution layers 35, a plurality of pooling layers 36, and a fully connected layer 37. In the neural network 30, the fully connected layer 37 exists in front of the output layer 33. In the neural network 30, the convolution layers 35 and the pooling layers 36 are alternately arranged between the input layer 31 and the fully connected layer 37.

[0033] The configuration of the neural network 30 is not limited to the example shown in Fig. 4. For example, the neural network 30 may include one convolution layer 35 and one pooling layer 36 between the input layer 31 and the fully connected layer 37.

[0034] Fig. 5 is a diagram showing an example of training data used for training a neural network. As shown in Fig. 5, training data 40 is composed of training data 41 and correct answer data 42. In this embodiment, the data input to trained neural network 23A to obtain an estimation result of bone density is a plain radiographic image G0, while training data 41 includes a composite two-dimensional image C0 representing subject H derived by combining with a CT image V0.

[0035] The correct answer data 42 is a bone image Gb and a soft tissue image Gs near the target bone (i.e., the femur) of the subject from which the learning data 41 was obtained. The bone image Gb and the soft tissue image Gs, which are the correct answer data 42, are derived from the CT image V0 by an information derivation device 50. The information derivation device 50 will be described below. The bone image Gb and the soft tissue image Gs derived from the CT image V0 are an example of a learning emphasized image.

[0036] 6 is a schematic block diagram showing the configuration of the information derivation device according to the first embodiment. As shown in FIG. 6, the information derivation device 50 according to the first embodiment is a computer such as a workstation, a server computer, or a personal computer, and includes a CPU 51, a non-volatile storage 53, and a memory 56 as a temporary storage area. The information derivation device 50 also includes a display 54 such as a liquid crystal display, an input device 55 consisting of a pointing device such as a keyboard and a mouse, and a network I / F 57 connected to a network (not shown). The CPU 51, the storage 53, the display 54, the input device 55, the memory 56, and the network I / F 57 are connected to a bus 58.

[0037] The storage 53 is realized by an HDD, an SSD, a flash memory, or the like, similar to the storage 13. The storage 53 as a storage medium stores an information derivation program 52. The CPU 51 reads out the information derivation program 52 from the storage 53, expands it in the memory 56, and executes the expanded information derivation program 52.

[0038] Next, a functional configuration of the information derivation device according to the first embodiment will be described. FIG. 7 is a diagram showing the functional configuration of the information derivation device according to the first embodiment. As shown in FIG. 7, the information derivation device 50 according to the first embodiment includes an image acquisition unit 61, a synthesis unit 62, and an emphasized image derivation unit 63. Then, the CPU 51 executes the information derivation program 52, so that the CPU 51 functions as the image acquisition unit 61, the synthesis unit 62, and the emphasized image derivation unit 63.

[0039] The image acquisition unit 61 acquires a CT image V0 for deriving the learning data 41 from the image storage system 9. Note that the image acquisition unit 61 may acquire the CT image V0 by causing the CT device 7 to capture an image of the subject H, similar to the image acquisition unit 21 of the estimation device 10.

[0040] The synthesis unit 62 synthesizes the CT images V0 to derive a synthetic two-dimensional image C0 representing the subject H. FIG. 8 is a diagram for explaining the derivation of the synthetic two-dimensional image C0. Note that in FIG. 8, the three-dimensional CT image V0 is shown in two dimensions for the sake of explanation. As shown in FIG. 8, the subject H is included in the three-dimensional space represented by the CT image V0. The subject H is composed of multiple components, such as bones, fat, muscles, and internal organs.

[0041] Here, the CT value V0(x, y, z) of each pixel of the CT image V0 can be expressed by the following formula (1) using the attenuation coefficient μi of the composition at that pixel and the attenuation coefficient μw of water. (x, y, z) are coordinates that represent the pixel position of the CT image V0. In the following explanation, the attenuation coefficient means the radiation source attenuation coefficient unless otherwise specified. The attenuation coefficient represents the degree (proportion) of attenuation of radiation due to absorption or scattering, etc. The attenuation coefficient differs depending on the specific composition (density, etc.) and thickness (mass) of the structure through which the radiation passes. V0(x,y,z)=(μi-μw) / μw×1000 (1)

[0042] The attenuation coefficient μw of water is known. Therefore, by solving equation (1) for μi, the attenuation coefficient μi of each composition can be calculated as shown in equation (2) below. μi=V0(x,y,z)×μw / 1000+μw (2)

[0043] As shown in FIG. 8, the synthesis unit 62 virtually irradiates the subject H with radiation of exposure dose I0, and derives a synthetic two-dimensional image C0 by virtually detecting the radiation transmitted through the subject H by a radiation detector (not shown) installed on a virtual plane 64. The exposure dose I0 and radiation energy of the virtual radiation are set according to predetermined imaging conditions. At this time, the reaching dose I1(x,y) for each pixel of the synthetic two-dimensional image C0 passes through one or more compositions in the subject H. For this reason, the reaching dose I1(x,y) can be derived by the following formula (3) using the attenuation coefficient μi of one or more compositions through which the radiation of exposure dose I0 passes. The reaching dose I1(x,y) becomes the pixel value of each pixel of the synthetic two-dimensional image C0. I1(x,y)=I0×exp(-∫μi·dt) (3)

[0044] If the irradiating radiation source is assumed to be a surface light source, the attenuation coefficient μi used in equation (3) may be derived from equation (2) using the CT values ​​of the pixels arranged in the vertical direction as shown in Fig. 8. If the irradiating radiation source is assumed to be a surface light source, as shown in Fig. 9, pixels on the path of the radiation that reaches each pixel may be identified based on the geometrical positional relationship between the point light source and each position on a virtual plane 64, and the attenuation coefficient derived from the CT value of the identified pixel using equation (2) may be used.

[0045] The enhanced image derivation unit 63 uses the CT image V0 to derive a bone image Gb in which the bones of the subject are enhanced, and a soft tissue image Gs in which the soft tissue is enhanced. Here, the CT value will be explained. FIG. 10 is a diagram for explaining the CT value. The CT value is a numerical representation of the X-ray absorption rate in the human body. Specifically, as shown in FIG. 10, the CT value is determined according to the composition that constitutes the human body, such that water has a CT value of 0 and air has a CT value of -1000 (unit: HU).

[0046] The enhanced image derivation unit 63 first identifies a bone region in the CT image V0 based on the CT value of the CT image V0. Specifically, a region consisting of pixels with a CT value of 100 to 1000 is identified as the bone region by threshold processing. Note that instead of threshold processing, the bone region may be identified by using a trained neural network that has been trained to detect the bone region from the CT image V0. Alternatively, the CT image V0 may be displayed on the display 54, and the bone region may be identified by manually specifying the bone region in the displayed CT image V0.

[0047] 11, the enhanced image derivation unit 63 derives a bone image Gb by projecting the CT value of the bone region Hb in the subject H included in the CT image V0 onto a virtual plane 64 in the same manner as in deriving the composite two-dimensional image C0. The bone image Gb is shown in FIG.

[0048] The enhanced image derivation unit 63 also identifies a soft tissue region in the CT image V0 based on the CT value of the CT image V0. Specifically, a region consisting of pixels with a CT value of -100 to 70 is identified as the soft tissue region by threshold processing. Instead of threshold processing, the soft tissue region may be identified using a trained neural network that has been trained to detect a soft tissue region from the CT image V0. The CT image V0 may also be displayed on the display 54, and the soft tissue region may be identified by manually specifying the soft tissue region in the displayed CT image V0.

[0049] 13, the enhanced image derivation unit 63 derives a soft tissue image Gs by projecting the CT value of the soft tissue region Hs in the subject H included in the CT image V0 onto a virtual plane 64 in the same manner as when deriving the composite two-dimensional image C0. The soft tissue image Gs is shown in FIG.

[0050] The bone images Gb and soft tissue images Gs used as the correct answer data 42 are derived at the same time as the learning data 41 is acquired, and are transmitted to the image storage system 9. In the image storage system 9, the learning data 41 and the correct answer data 42 are associated with each other and stored as the teacher data 40. Note that, in order to improve the robustness of learning, teacher data 40 including images obtained by performing at least one of the following on the same image: enlargement / reduction, contrast change, movement, in-plane rotation, inversion, and noise addition, may be additionally created and stored as the learning data 41.

[0051] Returning to the estimation device 10, the learning unit 24 learns the neural network using a large number of teacher data 40. FIG. 15 is a diagram for explaining the learning of the neural network 30. When learning the neural network 30, the learning unit 24 inputs learning data 41, i.e., a composite two-dimensional image C0, to the input layer 31 of the neural network 30. Then, the learning unit 24 causes the output layer 33 of the neural network 30 to output a bone image and a soft tissue image as output data 47. Then, the learning unit 24 derives the difference between the output data 47 and the correct answer data 42 as a loss L0. Note that the loss is derived between the bone image of the output data 47 and the bone image of the correct answer data 42, and between the soft tissue image of the output data and the soft tissue image of the correct answer data 42, but the loss is referred to as L0.

[0052] The learning unit 24 learns the neural network 30 based on the loss L0. Specifically, the learning unit 24 adjusts the kernel coefficients in the convolution layer 35, the weights of the connections between the layers, and the weights of the connections in the fully connected layer 37 (hereinafter referred to as parameters 48) so as to reduce the loss L0. For example, the backpropagation method can be used as a method for adjusting the parameters 48. The learning unit 24 repeats the adjustment of the parameters 48 until the loss L0 becomes equal to or less than a predetermined threshold value. As a result, when a simple radiographic image G0 is input, the parameters 48 are adjusted so as to output a bone image Gb and a soft tissue image Gs for the input simple radiographic image G0, and the trained neural network 23A is constructed. The constructed trained neural network 23A is stored in the storage 13.

[0053] Fig. 16 is a conceptual diagram of the process performed by the trained neural network 23A. As shown in Fig. 16, when a simple radiographic image G0 of a patient is input to the trained neural network 23A constructed as described above, the trained neural network 23A outputs a bone image Gb and a soft tissue image Gs for the input simple radiographic image G0.

[0054] The display control unit 25 displays the estimation results of the bone image Gb and the soft tissue image Gs estimated by the estimation unit 23 on the display 14. FIG. 17 is a diagram showing a display screen of the estimation results. As shown in FIG. 17, a display screen 70 has a first image display area 71 and a second image display area 72. A simple radiographic image G0 of the subject H is displayed in the first image display area 71. Furthermore, the bone image Gb and the soft tissue image Gs estimated by the estimation unit 23 are displayed in the second image display area 72.

[0055] Next, the process performed in the first embodiment will be described. FIG. 18 is a flowchart showing the learning process performed in the first embodiment. First, the information acquisition unit 22 acquires the teacher data 40 from the image storage system 9 (step ST1), and the learning unit 24 inputs the learning data 41 included in the teacher data 40 to the neural network 30 to output the bone image Gb and the soft tissue image Gs, and learns the neural network 30 using the loss L0 based on the difference from the correct answer data 42 (step ST2), and returns to step ST1. Then, the learning unit 24 repeats the processes of steps ST1 and ST2 until the loss L0 becomes a predetermined threshold value, and ends the learning process. Note that the learning unit 24 may end the learning process by repeating the learning a predetermined number of times. In this way, the learning unit 24 constructs the trained neural network 23A.

[0056] Next, the estimation process in the first embodiment will be described. FIG. 19 is a flowchart showing the estimation process in the first embodiment. It is assumed that the simple radiographic image G0 is acquired by imaging and stored in the storage 13. When an instruction to start the process is input from the input device 15, the image acquisition unit 21 acquires the simple radiographic image G0 from the storage 13 (step ST11). Next, the estimation unit 23 derives an estimation result of the bone image Gb and the soft tissue image Gs from the simple radiographic image G0 (step ST12). Then, the display control unit 25 displays the estimation result of the bone image Gb and the soft tissue image Gs derived by the estimation unit 23 on the display 14 together with the simple radiographic image G0 (step ST13), and the process ends.

[0057] In this manner, in this embodiment, the trained neural network 23A constructed by learning using the composite two-dimensional image C0 derived from the CT image V0 and the bone image Gb and soft tissue image Gs derived from the CT image V0 as teacher data is used to derive the estimation results of the bone image Gb and the soft tissue image Gs for the simple radiographic image G0. Here, in this embodiment, the composite two-dimensional image C0 derived from the CT image V0 and the bone image Gb and the soft tissue image Gs derived from the CT image V0 are used for learning the neural network. Therefore, compared with the case where one radiographic image and information related to the bone image Gb and the soft tissue image Gs derived from the radiographic image are used as teacher data, the trained neural network 23A is capable of deriving the estimation results of the bone image Gb and the soft tissue image Gs from the simple radiographic image G0 with higher accuracy. Therefore, according to this embodiment, the estimation results of the bone image Gb and the soft tissue image Gs can be derived with higher accuracy.

[0058] In the first embodiment, the estimation results of the bone image Gb and the soft tissue image Gs are derived, but the present invention is not limited to this. The estimation results of either the bone image Gb or the soft tissue image Gs may be derived. In this case, the trained neural network 23A may be constructed by performing training using teacher data in which the correct answer data is either the bone image Gb or the soft tissue image Gs.

[0059] In the first embodiment, the estimation results of the bone image Gb and the soft tissue image are derived from the simple radiographic image G0, but the present invention is not limited to this. For example, the estimation results of the muscle image and the fat image may be derived. This will be described below as the second embodiment.

[0060] The configurations of the estimation device and the information derivation device in the second embodiment are the same as those of the estimation device 10 and the information derivation device 50 in the first embodiment, and only the processes performed are different, so detailed explanations will be omitted here. In the second embodiment, the enhanced image derivation unit 63 of the information derivation device 50 derives a muscle image Gm and a fat image Gf instead of deriving a bone image Gb and a soft tissue image Gs as the correct answer data 42.

[0061] In the second embodiment, the enhanced image derivation unit 63 first identifies a muscle region in the CT image V0 based on the CT value of the CT image V0. Specifically, a region consisting of pixels with a CT value of 60 to 70 is identified as the muscle region by threshold processing. Note that instead of threshold processing, the muscle region may be identified using a trained neural network that has been trained to detect the muscle region from the CT image V0. Alternatively, the CT image V0 may be displayed on the display 54, and the muscle region may be identified by manually specifying the muscle region in the displayed CT image V0.

[0062] Then, the enhanced image derivation unit 63 derives a muscle image Gm by projecting the CT value of a muscle region Hm in the subject H included in the CT image V0 onto a virtual plane 64 in the same manner as in deriving the composite two-dimensional image C0, as shown in FIG. 20. The muscle image Gm is shown in FIG.

[0063] The enhanced image derivation unit 63 also identifies a fatty region in the CT image V0 based on the CT value of the CT image V0. Specifically, a region consisting of pixels with a CT value of -100 to -10 is identified as a fatty region by threshold processing. Note that instead of threshold processing, a trained neural network that has been trained to detect a fatty region from the CT image V0 may be used to identify the fatty region. Alternatively, the CT image V0 may be displayed on the display 54, and a fatty region may be identified by manually specifying the fatty region in the displayed CT image V0.

[0064] Then, the enhanced image derivation unit 63 derives a fat image Gf by projecting the CT value of the fat region Hf in the subject H included in the CT image V0 onto the virtual plane 64 in the same manner as in deriving the composite two-dimensional image C0, as shown in Fig. 22. The fat image Gf is shown in Fig. 23.

[0065] In the second embodiment, the muscle image Gm and fat image Gf derived by the information derivation device 50 are used as correct answer data of the teacher data. Fig. 24 is a diagram showing the teacher data derived in the second embodiment. As shown in Fig. 24, the teacher data 40A is composed of learning data 41 including a composite two-dimensional image C0 and correct answer data 42A including a muscle image Gm and a fat image Gf.

[0066] By training a neural network using the training data 40A shown in FIG. 24, it is possible to construct a trained neural network 23A that outputs a muscle image Gm and a fat image Gf as estimation results when a plain radiation image G0 is input.

[0067] In the second embodiment, the estimation results of the muscle image Gm and the fat image Gf are derived, but the present invention is not limited to this. The estimation results of either the muscle image Gm or the fat image Gf may be derived. In this case, the trained neural network 23A may be constructed by training using teacher data in which the correct answer data is either the muscle image Gm or the fat image Gf.

[0068] Next, a third embodiment of the present disclosure will be described. FIG. 25 is a diagram showing a functional configuration of an information derivation device according to the third embodiment. In FIG. 25, the same configuration as in FIG. 7 is given the same reference numerals, and detailed description will be omitted. In the above first and second embodiments, learning emphasis images (i.e., bone image Gb, soft tissue image Gs, muscle image Gm, and fat image Gf) serving as the correct answer data 42 are derived by projecting a specific composition in the subject H in the CT image V0. In the third embodiment, two composite two-dimensional images obtained by simulating a low-energy radiation image and a high-energy radiation image are derived from the CT image V0, and the learning emphasis image is derived by weighting and subtracting the two derived composite two-dimensional images.

[0069] 25, the information derivation device 50A according to the third embodiment further includes a synthesis unit 62A and an emphasized image derivation unit 63A in addition to the components of the information derivation device 50 according to the first embodiment. In the following description, a bone image Gb and a soft tissue image Gs are derived as learning emphasized images.

[0070] In the third embodiment, as shown in Fig. 26, the synthesis unit 62A virtually irradiates subject H with two types of radiation doses IL0 and IH0 with different energy distributions, and derives two synthetic two-dimensional images CL0 and CH0 by virtually detecting radiation with transmitted doses IL1 and IH1 that have passed through subject H using a radiation detector installed on a virtual plane 64. The synthetic two-dimensional image CL0 corresponds to a radiographic image of subject H created using low-energy radiation including so-called soft rays. The synthetic two-dimensional image CH0 corresponds to a radiographic image of subject H created using high-energy radiation from which soft rays have been removed.

[0071] The enhanced image derivation unit 63A identifies the bone region and the soft tissue region in the composite two-dimensional image CL0, CH0. The bone region and the soft tissue region have clearly different pixel values ​​in the composite two-dimensional image CL0, CH0. Therefore, the enhanced image derivation unit 63A identifies the bone region and the soft tissue region in the composite two-dimensional image CL0, CH0 by threshold processing. Note that instead of the threshold processing, the bone region and the soft tissue region may be identified by using a trained neural network that has been trained to detect the bone region and the soft tissue region in the composite two-dimensional image CL0, CH0. Also, the composite two-dimensional image CL0, CH0 may be displayed on the display 54, and the bone region and the soft tissue region may be identified by accepting the designation of the bone region and the soft tissue region by a manual operation in the displayed composite two-dimensional image CL0, CH0.

[0072] In the third embodiment, the enhanced image derivation unit 63A performs weighted subtraction on two composite two-dimensional images to derive a bone image Gb and a soft tissue image Gs. For this purpose, the enhanced image derivation unit 63A derives, for the soft tissue region in the two composite two-dimensional images CL0 and CH0, a ratio CLs(x,y) / CHs(x,y) between pixel values ​​CLs(x,y) of the composite two-dimensional image CL0 corresponding to a low-energy image and pixel values ​​CHs(x,y) of the composite two-dimensional image CH0 corresponding to a high-energy image, as a weighting coefficient α when performing weighted subtraction to derive a bone image Gb. The ratio CLs(x,y) / CHs(x,y) represents a ratio μls / μhs of attenuation coefficient μls for low-energy radiation to attenuation coefficient μhs for high-energy radiation for soft tissue.

[0073] Furthermore, for the bone regions in the two composite 2D images CL0 and CH0, the enhanced image derivation unit 63A derives the ratio CLb(x,y) / CHb(x,y) between the pixel value CLb(x,y) of the composite 2D image CL0 corresponding to the low-energy image and the pixel value CHb(x,y) of the composite 2D image CH0 corresponding to the high-energy image as the weighting coefficient β when performing weighted subtraction to derive the soft tissue image Gs. Note that the ratio CLb(x,y) / CHb(x,y) represents the ratio μlb / μhb of the attenuation coefficient μlb for low-energy radiation to the attenuation coefficient μhb for high-energy radiation for the bones.

[0074] In the third embodiment, the enhanced image derivation unit 63A uses the derived weighting coefficients α and β to perform weighted subtraction on the composite two-dimensional images CL0 and CH0 according to the following equations (4) and (5), thereby deriving a bone image Gb and a soft tissue image Gs. Gb(x,y)=α·CH0(x,y)-CL0(x,y) (4) Gs(x, y)=CH0(x, y)-β×CH0(x, y) (5)

[0075] In each of the above embodiments, the trained neural network 23A is constructed by training the neural network in the estimation device 10, but the present invention is not limited to this. The trained neural network 23A constructed in a device other than the estimation device 10 may be used in the estimation unit 23 of the estimation device 10 in this embodiment.

[0076] Furthermore, in each of the above embodiments, a process of estimating an enhanced image is performed using a radiological image acquired in a system that uses a radiation detector 5 to capture an image of the subject H. However, the technology of the present disclosure can of course also be applied to a case in which a radiological image is acquired using a storage phosphor sheet instead of a radiation detector.

[0077] Moreover, the radiation in the above embodiment is not particularly limited, and in addition to X-rays, α-rays, γ-rays, etc. can be used.

[0078] In the above embodiment, for example, the image acquisition unit 21, information acquisition unit 22, estimation unit 23, learning unit 24, and display control unit 25 of the estimation device 10, and the image acquisition unit 61, synthesis unit 62, and enhanced image derivation unit 63 of the information derivation device 50, etc. may have a hardware structure of a processing unit that executes various processes, such as various processors shown below. The above various processors include, as described above, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, etc., which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0079] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0080] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System On Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0081] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0082] The following are appended claims to this disclosure. (Additional note 1) at least one processor; The processor, The present invention functions as a trained neural network that derives an estimation result of at least one enhanced image in which a specific composition of a subject is enhanced from a simple two-dimensional image acquired by simply photographing the subject including a plurality of compositions, the trained neural network is trained using, as training data, a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image in which a specific composition of the subject is enhanced, derived from the CT images. (Additional note 2) 2. The estimation device according to claim 1, wherein the composite two-dimensional image is derived by deriving a radiation attenuation coefficient for a composition at each position in three-dimensional space and projecting the CT image in a predetermined direction based on the attenuation coefficient. (Additional note 3) 3. The estimation device according to claim 1 or 2, wherein the training weighted image is derived by identifying an area of ​​a specific composition in the CT image and projecting the CT image of the area of ​​the specific composition in a predetermined direction. (Additional note 4) The estimation device according to appended claim 1 or 2, wherein the learning emphasized image is derived by performing weighted subtraction on two composite two-dimensional images obtained by simulating imaging of the subject with radiation having different energy distributions, the composite two-dimensional images being derived by projecting the CT image in a predetermined direction. (Additional note 5) 5. The estimation device according to claim 1, wherein the specific composition is at least one of a soft part, a bone part, a muscle, and a fat part of the subject. (Additional note 6) An estimation method for deriving an estimation result of at least one emphasized image in which a specific composition of a subject is emphasized from a plain radiographic image obtained by plain radiographing the subject including a plurality of compositions, using a trained neural network that derives an estimation result of at least one emphasized image in which a specific composition of the subject is emphasized from the plain radiographic image, the method comprising: the trained neural network is trained using, as training data, a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image that enhances a specific composition of the subject derived from the CT images. (Additional note 7) An estimation program that causes a computer to execute a procedure of deriving an estimation result of at least one emphasized image in which a specific composition of a subject is emphasized from a plain radiographic image obtained by plain radiographing the subject including a plurality of compositions, using a trained neural network that derives an estimation result of at least one emphasized image in which a specific composition of the subject is emphasized from the plain radiographic image, the estimation program comprising: the trained neural network is an estimation program that is trained using, as training data, a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image that enhances a specific composition of the subject derived from the CT images. [Explanation of symbols]

[0083] 1. Imaging device 3 Radiation source 5. Radiation detectors 7 CT device 9. Image Storage System 10 Estimation device 11, 51 CPU 12 Estimation Processing Program 12B Study Program 13, 53 Storage 14,54 Display 15, 55 Input Devices 16,56 memory 17, 57 Network I / F 18, 58 Bus 21 Image acquisition unit 22 Information Acquisition Department 23 Estimation part 23A Trained Neural Networks 24 Learning Department 25 Display control section 30 Neural Networks 31 Input Layer 32 Middle Class 33 Output layer 35 Convolutional Layer 36 Pooling Layer 37 Fully connected layer 40, 40A Teacher Data 41 Training Data 42, 42A Correct data 47 Output Data 48 Parameters 50,50A Information derivation device 52 Information derivation program 61 Image acquisition unit 62, 62A synthesis section 63, 63A Enhanced image derivation unit 64 plane 70 display screen 71 First image display area 72 Second image display area C0, CL0, CH0 composite 2D image G0 Plain radiographic image Gb bone image Gf Fat Images Gm muscle images Gs Soft tissue image H Subject Hb bone area Hf fat area Hm muscle area Hs Soft region I0, IL0, IH0 Exposure dose I1, IL1, IH1 achieved dose V0 CT image

Claims

1. At least one processor; The processor, The present invention functions as a trained neural network that derives an estimation result of at least one enhanced image in which at least one of muscle and fat compositions of a subject is enhanced from a plain radiographic image obtained by plain radiographing a subject including a plurality of compositions, the trained neural network is trained using as training data a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image derived from the CT images and which enhances the composition of the subject that is emphasized in the estimation result of the enhanced image output by the trained neural network.

2. The estimation device according to claim 1 , wherein the synthetic two-dimensional image is derived by deriving a radiation attenuation coefficient for a composition at each position in three-dimensional space, and projecting the CT image in a predetermined direction based on the attenuation coefficient.

3. 3. The estimation device according to claim 1, wherein the training emphasized image is derived by identifying an area in the CT image that is composed of at least one of muscle and fat, and projecting the CT image of the area in a predetermined direction.

4. 3. The estimation device according to claim 1, wherein the learning emphasized image is derived by performing weighted subtraction on two composite two-dimensional images obtained by simulating imaging of the subject with radiation having different energy distributions, the composite two-dimensional images being derived by projecting the CT image in a predetermined direction.

5. An estimation method for deriving an estimation result of at least one emphasized image in which at least one of muscle and fat compositions of a subject is emphasized from a plain radiographic image obtained by simple radiography of a subject including a plurality of compositions, using a trained neural network to derive an estimation result of at least one emphasized image in which a specific composition of the subject is emphasized from the plain radiographic image, the method comprising: the trained neural network is trained using as training data a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image derived from the CT images, which enhances the composition of the subject that is enhanced in the estimation result of the enhanced image output by the trained neural network.

6. An estimation program for causing a computer to execute a procedure for deriving an estimation result of at least one emphasized image in which a specific composition of a subject is emphasized from a plain radiographic image obtained by simple radiography of a subject including a plurality of compositions, using a trained neural network for deriving an estimation result of at least one emphasized image in which at least one of a muscle composition and a fat composition of the subject is emphasized from the plain radiographic image, the estimation program comprising: the trained neural network is trained using as training data a synthetic two-dimensional image representing the subject derived by synthesizing three-dimensional CT images of the subject, and a training enhanced image derived from the CT images and which enhances the composition of the subject that is emphasized in the estimation result of the enhanced image output by the trained neural network.

Images