IMAGE PROCESSING APPARATUS, LEARNING APPARATUS, RADIOLOGICAL IMAGE CAPTURE SYSTEM, IMAGE PROCESSING METHOD, LEARNING METHOD, IMAGE PROCESSING PROGRAM, AND LEARNING PROGRAM

Abstract

To provide an image processing device, a learning device, a radiographic imaging system, an image processing method, a learning method, an image processing program and a learning program which can generate a high-quality tomographic image with a learned model learned by learning data according to a projection image by tomosynthesis imaging.SOLUTION: An image processing device generates a tomographic image group from a plurality of projection images, inputs the tomographic image group to a tomographic image estimation model that is a learned model generated by machine-learning a machine learning model using learning data constituted by a set of correct answer data being three-dimensional data expressing a three-dimensional structure and a virtual tomographic image group generated from a plurality of virtual projection images projected with the three-dimensional structure by pseudo-projection of the radiation from a plurality of virtual irradiation positions to the three-dimensional structure using the three-dimensional data and outputs an estimation tomographic image group with the tomographic image group as input, and acquires the estimation tomographic image group output from the tomographic image estimation model.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, a learning device, a radiation image capturing system, an image processing method, a learning method, an image processing program, and a learning program. [Background technology]

[0002] A so-called tomosynthesis imaging is known in which radiation is irradiated onto a subject from a plurality of irradiation positions each having a different irradiation angle, and a plurality of projection images of the subject at different irradiation positions are captured. A technology for generating a tomographic image from a plurality of projection images obtained by the tomosynthesis imaging is known.

[0003] When the number of projection images for generating a tomographic image is insufficient, artifacts may occur in the generated tomographic image. Therefore, a technology that can obtain a high-quality tomographic image even when the number of projection images is insufficient is desired. For example, Patent Document 1 describes a technology that generates missing data using a trained neural network, which is a learned model, when a part of the projection data is damaged or missing, and uses the missing data for reconstruction. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special table number 2020-506742 Summary of the Invention [Problem to be solved by the invention]

[0005] In the above conventional technology, a sufficient number of projection images without missing or corrupted images are required for training the neural network. The above conventional technology is based on computed tomography (CT), which can obtain a sufficient number of projection images.

[0006] However, radiological imaging devices that perform tomosynthesis imaging have a narrower range of radiation irradiation angles than CT, making it difficult to prepare sufficient projection images for training a trained model.

[0007] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image processing device, a learning device, a radiological image capturing system, an image processing method, a learning method, an image processing program, and a learning program that are capable of generating high-quality tomographic images using a trained model trained using learning data corresponding to projection images obtained by tomosynthesis imaging. [Means for solving the problem]

[0008] In order to achieve the above object, an image processing device of a first aspect of the present disclosure is an image processing device that processes a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, and includes at least one processor, wherein the processor acquires a plurality of projection images, generates a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images, and uses the three-dimensional data to pseudo-project radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, thereby generating a trained model by machine learning a machine learning model using training data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected, and the trained model inputs the generated group of tomographic images into a tomographic image estimation model that inputs the group of tomographic images and outputs a group of estimated tomographic images, and acquires the group of estimated tomographic images output from the tomographic image estimation model.

[0009] An image processing device according to a second aspect of the present disclosure is the image processing device according to the first aspect, wherein the three-dimensional data that is the correct data is image data representing a plurality of correct tomographic images each corresponding to a different tomographic plane.

[0010] An image processing device according to a third aspect of the present disclosure is the image processing device according to the first or second aspect, in which the subject is a breast.

[0011] In order to achieve the above object, a learning device of a fourth aspect of the present disclosure includes at least one processor, which uses three-dimensional data representing a three-dimensional structure to generate a plurality of virtual projection images onto which the three-dimensional structure is projected by pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, generates a group of virtual tomographic images from the plurality of virtual projection images, and trains a machine learning model using learning data consisting of a pair of ground truth data, which is three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that takes the group of tomographic images as input and outputs a group of estimated tomographic images.

[0012] A learning device according to a fifth aspect of the present disclosure is the learning device according to the fourth aspect, which generates a plurality of virtual projection images by simulating attenuation of radiation according to an absorption coefficient.

[0013] A learning device of a sixth aspect of the present disclosure is a learning device of the fourth or fifth aspect, in which the three-dimensional data that is the correct data is image data representing a plurality of correct tomographic images each corresponding to a different tomographic plane.

[0014] A seventh aspect of the present disclosure relates to a learning device according to any one of the fourth to sixth aspects, further comprising: Each of the tomographic images A plurality of virtual projection images are generated by adding noise components according to the radiation dose assumed to reach the radiation detector that generates the projection images used to generate the tomographic images.

[0015] A learning device of an eighth aspect of the present disclosure is a learning device of any one of the fourth to seventh aspects, in which the processor generates a group of virtual tomographic images from a plurality of virtual projection images using the FBP method or the iterative reconstruction method.

[0016] A learning device of a ninth aspect of the present disclosure is a learning device of any one of the fourth to eighth aspects, wherein the multiple virtual irradiation positions are positions that simulate radiation irradiation positions in tomosynthesis imaging.

[0017] A learning device of a tenth aspect of the present disclosure is a learning device of any one of the fourth to ninth aspects, in which the three-dimensional structure is a structure representing a breast, and the group of cross-sectional images is a group of cross-sectional images generated from projection images taken of a breast as a subject.

[0018] In addition, in order to achieve the above object, a radiographic imaging system of an eleventh aspect of the present disclosure includes a radiation source that generates radiation, a radiographic imaging device that performs tomosynthesis imaging in which radiation is irradiated toward a subject from each of a plurality of irradiation positions having different irradiation angles and a projection image of the subject is captured for each irradiation position, an image processing device of the present disclosure, and a learning device of the present disclosure.

[0019] In addition, in order to achieve the above-mentioned object, an image processing method of a twelfth aspect of the present disclosure is an image processing method for processing a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, which acquires a plurality of projection images, generates a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images, and generates a trained model by machine learning using learning data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected by using the three-dimensional data to pseudo-project radiation from a plurality of virtual irradiation positions, and the image processing method is an image processing method in which a computer executes a process of inputting the generated group of tomographic images into a tomographic image estimation model that takes the group of tomographic images as input and outputs a group of estimated tomographic images, and acquiring the group of estimated tomographic images output from the tomographic image estimation model.

[0020] In addition, in order to achieve the above-mentioned object, the learning method of a thirteenth aspect of the present disclosure is a learning method in which a computer executes a process of generating a tomography image estimation model that uses three-dimensional data representing a three-dimensional structure, generates a plurality of virtual projection images onto which the three-dimensional structure is projected by pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, generates a group of virtual tomography images from the plurality of virtual projection images, and trains a machine learning model using learning data consisting of a pair of ground truth data, which is three-dimensional data, and the group of virtual tomography images, thereby generating a tomography image estimation model that takes the group of tomography images as input and outputs a group of estimated tomography images.

[0021] In addition, in order to achieve the above-mentioned object, an image processing program of a fourteenth aspect of the present disclosure is an image processing program that processes a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, and obtains a plurality of projection images, generates a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images, and uses the three-dimensional data to pseudo-project radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, thereby generating a trained model by machine learning a machine learning model using training data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected, and causes a computer to execute a process of inputting the generated group of tomographic images into a tomographic image estimation model that takes the group of tomographic images as input and outputs a group of estimated tomographic images, and obtaining the group of estimated tomographic images output from the tomographic image estimation model.

[0022] In addition, in order to achieve the above-mentioned object, the learning program of a fifteenth aspect of the present disclosure causes a computer to execute a process of generating a plurality of virtual projection images onto which the three-dimensional structure is projected by using three-dimensional data representing a three-dimensional structure and pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, generating a group of virtual tomographic images from the plurality of virtual projection images, and training a machine learning model using learning data consisting of a pair of ground truth data, which is three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that takes the group of tomographic images as input and outputs a group of estimated tomographic images. Effect of the Invention

[0023] According to the present disclosure, high-quality tomographic images can be generated using a trained model trained using learning data corresponding to projection images obtained by tomosynthesis imaging. [Brief description of the drawings]

[0024] [Figure 1] 1 is a configuration diagram illustrating an example of the overall configuration of a radiation image capturing system according to an embodiment; [Diagram 2] FIG. 1 is a diagram for explaining an example of tomosynthesis imaging. [Diagram 3] 1 is a block diagram showing an example of the configuration of a mammography apparatus, a console, and an assistance device according to an embodiment. [Figure 4] FIG. 13 is a diagram for explaining an operation phase of the tomographic image estimation model. [Diagram 5] FIG. 13 is a diagram for explaining a learning phase of a tomographic image estimation model. [Figure 6] FIG. 2 is a functional block diagram illustrating an example of a function of the support device according to the embodiment. [Figure 7] 11 is a diagram for explaining an example of a reprojection method in a virtual projection image generating unit of the support device. FIG. [Figure 8] FIG. 11 is a diagram for explaining an example of learning of a tomographic image estimation model using a plurality of learning data. [Figure 9]11 is a flowchart illustrating an example of a flow of a learning process by the support device of the embodiment. [Figure 10] FIG. 2 is a functional block diagram illustrating an example of functions of a console according to an embodiment. [Figure 11] 10 is a flowchart illustrating an example of a flow of image processing by the console of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment.

[0026] First, an example of the overall configuration of the radiation image capturing system of this embodiment will be described. Fig. 1 shows a configuration diagram showing an example of the overall configuration of the radiation image capturing system 1 of this embodiment. As shown in Fig. 1, the radiation image capturing system 1 of this embodiment includes a mammography device 10, a console 12, and a support device 14.

[0027] First, the mammography device 10 of this embodiment will be described. Fig. 1 shows a side view illustrating an example of the appearance of the mammography device 10 of this embodiment. Fig. 1 shows an example of the appearance of the mammography device 10 when viewed from the left side of a subject.

[0028] The mammography device 10 of this embodiment operates under the control of a console 12, and is a device that takes a radiographic image of a subject's breast by irradiating the breast with radiation R (e.g., X-rays) as a subject's subject. Note that the mammography device 10 may be a device that takes an image of the subject's breast not only when the subject is standing (standing position) but also when the subject is sitting in a chair (including a wheelchair) or the like (seated position).

[0029] In addition, the mammography apparatus 10 of this embodiment has the function of performing normal imaging, in which imaging is performed with the radiation source 29 positioned at an irradiation position along the normal direction of the detection surface 20A of the radiation detector 20, and so-called tomosynthesis imaging, in which imaging is performed by moving the radiation source 29 to each of a number of irradiation positions.

[0030] The radiation detector 20 detects radiation R that has passed through the breast, which is the subject. In detail, the radiation detector 20 detects radiation R that has entered the subject's breast and the imaging table 24 and reached the detection surface 20A of the radiation detector 20, generates a radiographic image based on the detected radiation R, and outputs image data representing the generated radiographic image. Hereinafter, the series of operations of irradiating radiation R from the radiation source 29 and generating a radiographic image by the radiation detector 20 may be referred to as "imaging". The detection surface 20A of the radiation detector 20 of this embodiment has i pixels (pixels 21 in FIG. 7) corresponding to the radiographic image generated by the radiation detector 20. i (see i=1, 2, . . . )) are arranged in a matrix. The type of radiation detector 20 in this embodiment is not particularly limited, and may be, for example, an indirect conversion type radiation detector that converts radiation R into light and then converts the converted light into an electric charge, or a direct conversion type radiation detector that directly converts radiation R into an electric charge.

[0031] 1, the radiation detector 20 is disposed inside an imaging stand 24. In the mammography apparatus 10 of this embodiment, when imaging is performed, the breast of the subject is positioned on the imaging surface 24A of the imaging stand 24 by the user.

[0032] The compression plate 38 used to compress the breast when imaging is performed is attached to a compression unit 36 ​​provided on the imaging table 24. In detail, the compression unit 36 ​​is provided with a compression plate drive section (not shown) that moves the compression plate 38 in a direction toward or away from the imaging table 24 (hereinafter referred to as the "up and down direction"). A support section 39 of the compression plate 38 is detachably attached to the compression plate drive section and is moved in the up and down direction by the compression plate drive section to compress the breast of the subject between the compression plate 38 and the imaging table 24. The compression plate 38 of this embodiment is an example of a compression member of the present disclosure.

[0033] As shown in Fig. 1, the mammography apparatus 10 of this embodiment includes an imaging table 24, an arm unit 33, a base 34, and an axis unit 35. The arm unit 33 is held by the base 34 so as to be movable in the vertical direction (Z-axis direction). The axis unit 35 enables the arm unit 33 to rotate relative to the base 34. The axis unit 35 is fixed to the base 34, and the axis unit 35 and the arm unit 33 rotate together.

[0034] Gears are provided on the shaft 35 and the compression unit 36 ​​of the imaging table 24, and by switching between an engaged state and a non-engaged state of these gears, it is possible to switch between a state in which the compression unit 36 ​​of the imaging table 24 and the shaft 35 are connected and rotate together, and a state in which the shaft 35 is separated from the imaging table 24 and rotates freely. Note that the switching between transmission and non-transmission of power to the shaft 35 is not limited to the above gears, and various mechanical elements can be used.

[0035] The arm unit 33 and the imaging stand 24 are separately rotatable relative to the base 34, with the shaft unit 35 as a rotation axis. In this embodiment, the base 34, the arm unit 33, and the compression unit 36 ​​of the imaging stand 24 are each provided with an engagement unit (not shown), and by switching the state of this engagement unit, each of the arm unit 33 and the compression unit 36 ​​of the imaging stand 24 is connected to the base 34. One or both of the arm unit 33 and the imaging stand 24 connected to the shaft unit 35 rotate together around the shaft unit 35.

[0036] When performing tomosynthesis imaging in the mammography apparatus 10, the radiation source 29 of the radiation irradiation unit 28 is moved sequentially to each of a plurality of irradiation positions having different irradiation angles by the rotation of the arm unit 33. The radiation source 29 has a radiation tube (not shown) that generates radiation R, and the radiation tube is moved to each of the plurality of irradiation positions in accordance with the movement of the radiation source 29. FIG. 2 shows a diagram for explaining an example of tomosynthesis imaging. Note that the compression plate 38 is not shown in FIG. 2. In this embodiment, as shown in FIG. 2, the radiation source 29 is moved sequentially to each of the irradiation positions 19 having irradiation angles that differ in increments of a predetermined angle β. t (t=1, 2, . . . , the maximum value is 7 in FIG. 2), in other words, the radiation detector 20 is moved to a position where the irradiation angle of the radiation R with respect to the detection surface 20A of the radiation detector 20 is different. t In response to an instruction from the console 12, radiation R is irradiated from the radiation source 29 toward the subject U, and a radiation image is captured by the radiation detector 20. t 19 to each of the irradiation positions 19 t In the example of Fig. 2, when tomosynthesis imaging is performed to capture radiation images at each irradiation position 19, seven radiation images are obtained. In the following, when describing the radiation images captured at each irradiation position 19 in tomosynthesis imaging in order to distinguish them from other radiation images, they will be referred to as "projection images." In addition, when referring to radiation images collectively regardless of the type, such as projection images and tomographic images described later, they will be simply referred to as "radiation images." In addition, in the following, when describing the radiation images collectively, regardless of the type, such as projection images and tomographic images described later, they will be simply referred to as "radiation images." t Projection images taken at 19 irradiation positions t For the corresponding images, the symbols representing each image are indicated with the irradiation position 19. t The symbol "t" is added to indicate this.

[0037] As shown in Fig. 2, the irradiation angle of radiation R refers to the angle α between a normal CL to the detection surface 20A of the radiation detector 20 and a radiation axis RC. The radiation axis RC refers to an axis connecting the focal point of the radiation source 29 at each irradiation position 19 and a preset position such as the center of the detection surface 20A. Hereinafter, the detection surface 20A of the radiation detector 20 is assumed to be a surface that is approximately parallel to the imaging surface 24A. Hereinafter, the predetermined range within which the irradiation angle is changed in tomosynthesis imaging as shown in Fig. 2 will be referred to as an "irradiation angle range."

[0038] On the other hand, when performing normal imaging in the mammography device 10, the radiation source 29 of the radiation irradiation unit 28 is positioned at the irradiation position 19 where the irradiation angle α is 0 degrees. t (Irradiation position along the normal direction 19 t In Fig. 2, the irradiation position is 19 4 In response to an instruction from the console 12, radiation R is emitted from the radiation source 29, and a radiation image is captured by the radiation detector 20.

[0039] Fig. 3 is a block diagram showing an example of the configuration of the mammography apparatus 10, console 12, and support device 14 of this embodiment. As shown in Fig. 3, the mammography apparatus 10 of this embodiment further includes a control unit 40, a memory unit 42, an I / F (Interface) unit 44, an operation unit 46, and a radiation source movement unit 47. The control unit 40, the memory unit 42, the I / F unit 44, the operation unit 46, and the radiation source movement unit 47 are connected via a bus 49 such as a system bus or a control bus so that they can exchange various information with each other.

[0040] The control unit 40 controls the overall operation of the mammography apparatus 10 in response to control by the console 12. The control unit 40 includes a CPU (Central Processing Unit) 40A, a ROM (Read Only Memory) 40B, and a RAM (Random Access Memory) 40C. The ROM 40B stores various programs in advance, including an imaging program 41 for controlling radiographic imaging, which is executed by the CPU 40A. The RAM 40C temporarily stores various data.

[0041] The storage unit 42 stores image data of a radiation image captured by the radiation detector 20 and various other information. Specific examples of the storage unit 42 include a hard disk drive (HDD) and a solid state drive (SSD). The I / F unit 44 communicates various information with the console 12 by wireless communication or wired communication. Image data of a radiation image captured by the radiation detector 20 in the mammography apparatus 10 is transmitted to the console 12 via the I / F unit 44 by wireless communication or wired communication.

[0042] In this embodiment, the control unit 40 , the storage unit 42 , and the I / F unit 44 are each provided inside the imaging stand 24 .

[0043] The operation unit 46 is provided as a plurality of switches on, for example, the imaging table 24 of the mammography apparatus 10. The operation unit 46 may be provided as a touch panel switch, or as a foot switch that is operated by a user such as a doctor or technician with his / her foot.

[0044] When performing tomosynthesis imaging as described above, the radiation source moving unit 47 moves the radiation source 29 to the irradiation position 19 under the control of the control unit 40. t Specifically, the radiation source moving unit 47 rotates the arm unit 33 relative to the imaging table 24 to move the radiation source to the irradiation position 19. t The radiation source 29 is moved to each of the above positions. The radiation source moving unit 47 in this embodiment is provided inside the arm unit 33.

[0045] On the other hand, the support device 14 of this embodiment has a function of supporting image processing in the console 12.

[0046] The support device 14 of this embodiment is, for example, a server computer. As shown in Fig. 3, the support device 14 includes a control unit 60, a storage unit 62, an I / F unit 64, an operation unit 66, and a display unit 68. The control unit 60, the storage unit 62, the I / F unit 64, the operation unit 66, and the display unit 68 are connected to each other via a bus 69 such as a system bus or a control bus so as to be able to transmit and receive various information to and from each other.

[0047] The control unit 60 of this embodiment controls the overall operation of the support device 14. The control unit 60 includes a CPU 60A, a ROM 60B, and a RAM 60C. The ROM 60B stores various programs including a learning program 61 executed by the CPU 60A in advance. The RAM 60C temporarily stores various data. The support device 14 of this embodiment is an example of a learning device of the present disclosure, and the CPU 60A of this embodiment is an example of a processor in the learning device of the present disclosure. The learning program 61 of this embodiment is an example of a learning program of the present disclosure.

[0048] The storage unit 62 stores various information including a tomographic image estimation model 63, the details of which will be described later. Specific examples of the storage unit 62 include an HDD and an SSD.

[0049] The operation unit 66 is used by the user to input instructions and various information related to the generation of the tomographic image estimation model 63. The operation unit 66 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 68 displays various information. The operation unit 66 and the display unit 68 may be integrated into a touch panel display.

[0050] The I / F unit 64 communicates various information between the console 12 and a PACS (Picture Archiving and Communication Systems) via wireless or wired communication. In the radiation image capturing system 1 of this embodiment, the console 12 receives image data of a radiation image captured by the mammography apparatus 10 from the mammography apparatus 10 via the I / F unit 54 via wireless or wired communication.

[0051] A description will be given of a function of the support device 14 of this embodiment to support image processing in the console 12. The support device 14 of this embodiment has a function to support image processing by the console 12. Specifically, the support device 14 has a function to generate a tomographic image estimation model 63 used by the console 12 to generate a tomographic image in image processing.

[0052] The tomographic image estimation model 63 is, for example, an estimation algorithm for a tomographic image using deep learning. For example, a tomographic image estimation model configured with R-CNN (Regional CNN), which is a type of Convolutional Neural Network (CNN), or U-net, which is a type of Fully Convolution Network (FCN), can be used as the tomographic image estimation model 63. As shown in FIG. 4, the tomographic image estimation model 63 receives as input a group of tomographic images 96 generated from a plurality of projection images 90 obtained in tomosynthesis imaging, and estimates the projection images 90 at the irradiation position 19. t The method outputs a group of estimated tomographic images 92 obtained by estimating a plurality of tomographic images generated from a plurality of projection images 90 using the above method.

[0053] As shown in FIG. 5, the tomographic image estimation model 63 is a trained model generated by training a machine learning model using training data 100 consisting of a set of ground truth data 102 and a group of virtual tomographic images 98.

[0054] With reference to FIG. 5, an example of a learning phase in which the tomographic image estimation model 63 is trained by machine learning will be described. The tomographic image estimation model 63 is trained by training data 100. The training data 100 is composed of a set of a virtual tomographic image group 98 generated from a virtual projection image 91 and correct answer data 102. The correct answer data 102 is three-dimensional data representing a three-dimensional structure 110 of the breast, and is image data representing a plurality of correct answer tomographic images each corresponding to a different tomographic plane. Note that the slice thickness of the correct answer tomographic image which is the correct answer data 102, in other words, the height of each tomographic plane of the correct answer tomographic image, can be set arbitrarily.

[0055] The answer data 102 is preferably a sinogram obtained with a wider irradiation angle range than the irradiation angle range in tomosynthesis imaging, and more preferably a complete sinogram obtained by imaging the subject. As an example, in this embodiment, CT images, which are multiple tomographic images of the subject obtained by CT (Computed Tomography), are used as the answer data 102. The virtual projection images 91 are multiple projection images obtained virtually by projecting the three-dimensional structure 110 of the breast by pseudo-projecting radiation R from multiple virtual irradiation positions onto the three-dimensional structure 110.

[0056] In the learning phase, a virtual tomographic image group 98 of the learning data 100 is input to the tomographic image estimation model 63. The tomographic image estimation model 63 outputs an estimated tomographic image group 92 corresponding to the virtual tomographic image group 98. A loss calculation is performed using a loss function based on the estimated tomographic image group 92 and the ground truth data 102. Then, update settings of various coefficients of the tomographic image estimation model 63 are performed according to the result of the loss calculation, and the tomographic image estimation model 63 is updated according to the update settings.

[0057] In the learning phase, a series of processes are repeatedly performed, including input of the virtual tomographic image group 98 of the learning data 100 to the tomographic image estimation model 63, output of the estimated tomographic image group 92 from the tomographic image estimation model 63, loss calculation based on the estimated tomographic image group 92 and ground truth data 102, update setting, and updating of the tomographic image estimation model 63.

[0058] Fig. 6 shows a functional block diagram of an example of a configuration related to a function of generating a tomographic image estimation model 63 in the support device 14 of this embodiment. As shown in Fig. 6, the support device 14 includes a three-dimensional data acquisition unit 80, a virtual projection image generation unit 82, a virtual tomographic image group generation unit 83, and a tomographic image estimation model generation unit 84. As an example, in the support device 14 of this embodiment, the CPU 60A of the control unit 60 executes the learning program 61 stored in the ROM 60B, so that the CPU 60A functions as the three-dimensional data acquisition unit 80, the virtual projection image generation unit 82, the virtual tomographic image group generation unit 83, and the tomographic image estimation model generation unit 84.

[0059] The three-dimensional data acquisition unit 80 has a function of acquiring three-dimensional data to be used as the correct answer data 102. As described above, in this embodiment, a CT image is used as the correct answer data 102, so the three-dimensional data acquisition unit 80 acquires a plurality of CT images representing a plurality of cross sections of the breast. Specifically, the three-dimensional data acquisition unit 80 of this embodiment acquires image data representing a plurality of CT images obtained by CT imaging of a three-dimensional structure 110 of the breast as three-dimensional data. The source from which the three-dimensional data is acquired is not particularly limited. For example, the three-dimensional data acquisition unit 80 may acquire a CT image of a breast as an imaging target as three-dimensional data from a PACS. Also, for example, the three-dimensional data acquisition unit 80 may acquire a CT image of a breast as an imaging target as three-dimensional data from an imaging device (not shown) of a CT image outside the radiation image imaging system 1. The three-dimensional data acquisition unit 80 outputs image data representing the acquired plurality of CT images to the virtual projection image generation unit 82.

[0060] The virtual projection image generation unit 82 has the function of generating multiple virtual projection images 91 onto which the three-dimensional structure 110 is projected, by pseudo-projecting radiation R from multiple virtual irradiation positions onto the three-dimensional structure 110 using the three-dimensional data acquired by the three-dimensional data acquisition unit 80.

[0061] As described above, the three-dimensional data in this embodiment is a CT image, so the virtual projection image generating unit 82 generates a plurality of virtual projection images 91 onto which the three-dimensional structure 110 is projected by pseudo-projecting radiation R from a plurality of virtual irradiation positions onto the three-dimensional structure 110 from the CT image. The plurality of virtual irradiation positions used to generate the virtual projection images 91 are the irradiation positions 19 of radiation in tomosynthesis imaging. t In this embodiment, the virtual irradiation position is set as the irradiation position 19. t It is considered the same as above.

[0062] The pixel values ​​of the CT image correspond to an absorption coefficient according to the energy of the radiation used in the imaging. The absorption coefficient of the radiation in the CT image imaging and the absorption coefficient of the radiation R in the tomosynthesis imaging are generally different. Therefore, as an example, the virtual projection image generating unit 82 of this embodiment corrects the pixel values ​​of the CT image to an absorption coefficient according to the energy of the radiation R in the tomosynthesis imaging and reprojects them, thereby generating a plurality of virtual irradiation positions 19. t , and a plurality of virtual projection images 91 are generated by reprojecting from the image.

[0063] An example of a reprojection method in the virtual projection image generating unit 82 will be described with reference to Fig. 7. As shown in Fig. 7, a plurality of voxels 94 are projected onto a three-dimensional structure 110. j (j=1, 2, . . . , J) are virtually set. Multiple voxels 94 j is set according to, for example, the slice thickness of the CT image. t The radiation detector 20 detects radiation from a radiation source 29 located at i Path X of radiation R being irradiated to t i The route is shown as X t i and each voxel 94 of the three-dimensional structure 110 j Voxel 94 where and intersect j The intersection length for each t ij And voxel 94 j The absorption coefficient for each 1 ,μ2 ,μ J ), then the irradiation position is 19 t When radiation R is irradiated from the i-th pixel 21 of the radiation detector 20, i The number of photons detected in t i is expressed by the following equation (1), which is based on the attenuation of radiation according to the absorption coefficient.

number

[0064] The virtual projection image generating unit 82 generates a virtual projection position 19 t Each pixel 21 of the radiation detector 20 i The number of photons in t i is derived by simulation using the above formula (1), and the projection image 90 t Generate.

[0065] It is preferable that the virtual projection image generating unit 82 generates the virtual projection image 91 in consideration of noise corresponding to the radiation R actually irradiated in the tomosynthesis imaging.

[0066] For example, since it is known that the generation of radiation R follows a Poisson distribution, the virtual projection image generation unit 82 may generate a virtual projection image 91 that takes noise into account by adding a noise component corresponding to the dose of radiation R assumed to reach the radiation detector 20.

[0067] In addition, it is preferable that the virtual projection image generating unit 82 generates the virtual projection image 91 in consideration of scattered rays generated in the tomosynthesis imaging. tA virtual projection image 91 taking scattered rays into consideration may be generated by adding a scattered ray component caused by radiation R irradiated from and transmitted through the subject to the virtual projection image 91 obtained by the above formula (1). The scattered ray component can be derived, for example, by a simulation based on a convolution kernel that represents the thickness of the three-dimensional structure 110 in the direction in which the radiation R penetrates, i.e., a scattering characteristic according to the body thickness of the subject, a Monte Carlo simulation, or the like.

[0068] Furthermore, it is preferable that the virtual projection image generating unit 82 generates the virtual projection image 91 by reflecting the imaging conditions in the tomosynthesis imaging. For example, it is preferable to reflect the energy distribution in the tomosynthesis imaging. The energy distribution of the irradiated radiation R is determined by the anode and filter (both not shown) of the radiation source 29 used in the tomosynthesis imaging. Therefore, it is preferable that the virtual projection image generating unit 82 derives the amount of transmitted radiation from the energy distribution determined according to the imaging conditions and the radiation energy dependency of the absorption coefficient.

[0069] The virtual projection image generating unit 82 outputs image data representing the generated virtual projection images 91 to the virtual tomographic image group generating unit 83 .

[0070] The virtual tomographic image group generating unit 83 extracts a plurality of virtual irradiation positions 19 from the plurality of virtual projection images 91 generated by the virtual projection image generating unit 82. tThe virtual tomographic image group generating unit 83 has a function of generating a virtual tomographic image group 98 using the above-mentioned method. The method by which the virtual tomographic image group generating unit 83 generates the virtual tomographic image group 98 is not particularly limited, and a known method can be used. For example, the reconstruction may be performed by a back projection method such as the FBP (Filter Back Projection) method or the iterative reconstruction method, and known techniques can be applied. When the FBP method is used, the time required for processing and calculation can be shortened. On the other hand, when the iterative reconstruction method is used, a virtual tomographic image group 98 with less noise and fewer artifacts is obtained. In the learning phase, the processing and calculation may take longer than in the operation phase. Therefore, it is preferable to perform reconstruction using the iterative reconstruction method, and the image quality of the virtual tomographic image group 98 can be improved. The slice thickness of the virtual tomographic image group 98 generated by the virtual tomographic image group generating unit 83, in other words, the height of each tomographic plane of the virtual tomographic image group 98, is not particularly limited and can be any, but it is preferable to make it the same as the slice thickness of the correct tomographic image that is the correct data 102.

[0071] The virtual tomographic image group generating unit 83 outputs image data representing the generated virtual tomographic image group 98 to the tomographic image estimation model generating unit 84.

[0072] The tomographic image estimation model generation unit 84 has the function of generating a tomographic image estimation model 63 that receives the tomographic image group 96 as input and outputs the estimated tomographic image group 92 by training a machine learning model using learning data 100 consisting of a set of ground truth data 102 and a virtual tomographic image group 98.

[0073] First, the tomographic image estimation model generating unit 84 prepares learning data 100. As described above, the absorption coefficient of radiation in CT image capture is different from the absorption coefficient of radiation R in tomosynthesis capture. Therefore, the CT image and the radiation image captured by the mammography device 10 look different. Specifically, even if they are images showing the same tomographic plane of the breast, the CT image and the tomographic image generated from the projection image obtained by tomosynthesis capture by the mammography device 10 look different because the absorption coefficients in the capture are different.

[0074] Therefore, in this embodiment, the CT image itself, which is the three-dimensional data acquired by the three-dimensional data acquisition unit 80, is not used as the correct answer data 102, but an image obtained by correcting the CT image according to the absorption coefficient of the radiographic image is used as the correct answer data 102. Therefore, the tomographic image estimation model generation unit 84 corrects the pixel values ​​of the CT image, which is the three-dimensional data acquired by the three-dimensional data acquisition unit 80, to an absorption coefficient according to the energy of the radiation R in tomosynthesis imaging to generate the correct answer data 102. The tomographic image estimation model generation unit 84 pairs the virtual tomographic image group 98 generated by the virtual tomographic image group generation unit 83 with the correct answer data 102 to generate learning data 100.

[0075] 8, a plurality of pieces of learning data 100 are used for learning the tomographic image estimation model 63. Therefore, in the support device 14 of this embodiment, a plurality of pieces of learning data 100 are prepared by the three-dimensional data acquisition unit 80, the virtual projection image generation unit 82, the virtual tomographic image group generation unit 83, and the tomographic image estimation model generation unit 84. The supervised answer data 102 constituting the learning data 100 is three-dimensional data assuming a breast to be the subject of tomosynthesis imaging as a three-dimensional structure 110. Therefore, in this embodiment, the learning data 100 is constituted by the supervised answer data 102 in which breasts in various states are regarded as the three-dimensional structure 110. For example, as shown in FIG. 8, multiple pieces of learning data 100 are used for machine learning, including learning data 100 consisting of a set of correct answer data 102 in which a breast in a so-called normal state, i.e., in which no tumor or the like is present, is shown as a three-dimensional structure 110, and a group of virtual tomographic images 98, and learning data 100 consisting of a set of correct answer data 102 in which a breast in a state in which a tumor is present is shown as a three-dimensional structure 110, and a group of virtual tomographic images 98.

[0076] Furthermore, the tomographic image estimation model generation unit 84 generates the tomographic image estimation model 63 by machine learning the machine learning model using the learning data 100. As described above with reference to Fig. 6, the tomographic image estimation model generation unit 84 generates the tomographic image estimation model 63 by repeatedly performing a series of processes including inputting the virtual tomographic image group 98 of the learning data 100 to the tomographic image estimation model 63, outputting the estimated tomographic image group 92 from the tomographic image estimation model 63, loss calculation based on the estimated tomographic image group 92 and the correct answer data 102, update setting, and updating the tomographic image estimation model 63. As a loss function in the loss calculation, for example, the total value of differences in pixel values ​​of corresponding pixels between tomographic images having the same height of the tomographic plane in the estimated tomographic image group 92 and the correct answer data 102 which is the correct answer tomographic image may be applied.

[0077] The slice thickness of the estimated tomographic image group 92 output from the tomographic image estimation model 63, in other words, the height of the tomographic plane of each estimated tomographic image, can be set arbitrarily. The slice thickness of the estimated tomographic image group 92 is preferably set to be the same as the slice thickness of the correct tomographic image, which is the correct data 102.

[0078] The tomographic image estimation model generating unit 84 stores the generated tomographic image estimation model 63 in the storage unit 62 .

[0079] Next, the operation of the support device 14 of this embodiment in the learning phase will be described with reference to Fig. 9. The learning process shown in Fig. 9 is carried out by the CPU 60A executing the learning program 61 stored in the ROM 60B.

[0080] In step S100 of FIG. 9, the three-dimensional data acquisition unit 80 acquires image data of a CT image as three-dimensional data to be used as the supervised data 102, as described above.

[0081] In the next step S102, the virtual projection image generating unit 82 uses the three-dimensional data acquired in the above step S100 to generate a plurality of virtual projection images 91. As described above, the virtual projection image generating unit 82 uses the CT image acquired in the above step S100 and applies the above formula (1) to generate a plurality of virtual irradiation positions 19 for the three-dimensional structure 110. t By performing pseudo-projection of rays R from the target object, a plurality of virtual projection images 91 onto which a three-dimensional structure 110 is projected are generated.

[0082] In the next step S104, the virtual tomographic image group generating unit 83 generates a virtual tomographic image group 98 from the plurality of virtual projection images 91 generated in the above step S102. As described above, the virtual tomographic image group generating unit 83 generates a plurality of virtual projection images 91 generated in the above step S102 at a plurality of virtual irradiation positions 19. t A virtual tomographic image group 98 is generated from the image by a back projection method such as the FBP method or the iterative reconstruction method.

[0083] In the next step S106, the tomographic image estimation model generation unit 84 prepares a plurality of learning data 100. As described above, the tomographic image estimation model generation unit 84 prepares a plurality of learning data 100 each including a pair of the corrective data 102 obtained by correcting the CT image acquired in the above step S100 and the virtual tomographic image group 98 generated in the above step S104.

[0084] In the next step S108, the tomographic image estimation model generation unit 84 trains the tomographic image estimation model 63 using the training data 100 prepared in the above step S106. As described above, the tomographic image estimation model generation unit 84 trains the tomographic image estimation model 63 by repeatedly performing a series of processes including inputting the virtual tomographic image group 98 to the tomographic image estimation model 63, outputting the estimated tomographic image group 92, calculating losses based on the estimated tomographic image group 92 and the ground truth data 102, setting updates, and updating the tomographic image estimation model 63. The tomographic image estimation model generation unit 84 stores the trained tomographic image estimation model 63 in the storage unit 62. When the process of step S108 ends, the learning process shown in FIG. 9 ends.

[0085] The tomographic image estimation model 63 generated in the learning phase in the support device 14 as described above is used in the operation phase in image processing executed in the console 12.

[0086] On the other hand, as shown in Figure 3, the console 12 of this embodiment has the function of controlling the mammography apparatus 10 using shooting orders and various information obtained from a RIS (Radiology Information System) or the like via a wireless communication LAN (Local Area Network) or the like, and instructions given by a user via an operation unit 56 or the like.

[0087] The console 12 of this embodiment is, for example, a server computer. As shown in Fig. 3, the console 12 includes a control unit 50, a storage unit 52, an I / F unit 54, an operation unit 56, and a display unit 58. The control unit 50, the storage unit 52, the I / F unit 54, the operation unit 56, and the display unit 58 are connected to each other via a bus 59 such as a system bus or a control bus so as to be able to transmit and receive various information to and from each other.

[0088] The control unit 50 of this embodiment controls the overall operation of the console 12. The control unit 50 includes a CPU 50A, a ROM 50B, and a RAM 50C. The ROM 50B stores various programs including an image generation program 51 executed by the CPU 50A in advance. The RAM 50C temporarily stores various data. The console 12 of this embodiment is an example of an image processing device of the present disclosure, and the CPU 50A of this embodiment is an example of a processor in the image processing device of the present disclosure. The image generation program 51 of this embodiment is an example of a learning program of the present disclosure.

[0089] The storage unit 52 stores image data of radiographic images captured by the mammography apparatus 10, various other information, etc. Specific examples of the storage unit 52 include an HDD and an SSD.

[0090] The operation unit 56 is used by the user to input instructions regarding radiographic image capture, including instructions for irradiating radiation R, and various types of information. The operation unit 56 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 58 displays various types of information. The operation unit 56 and the display unit 58 may be integrated into a touch panel display.

[0091] The I / F unit 54 communicates various types of information between the mammography apparatus 10, the support device 14, the RIS, and the PACS via wireless or wired communication. In the radiation image capturing system 1 of this embodiment, image data of a radiation image captured by the mammography apparatus 10 is received by the console 12 from the mammography apparatus 10 via the I / F unit 54 via wireless or wired communication.

[0092] The console 12 of this embodiment has a function of generating a tomographic image from a plurality of projection images 90 using a tomographic image estimation model 63. Fig. 10 shows a functional block diagram of an example of a configuration related to a function of generating a tomographic image from a plurality of projection images 90 using the tomographic image estimation model 63 in the console 12 of this embodiment. As shown in Fig. 10, the console 12 includes a projection image acquisition unit 70, a tomographic image group generation unit 72, an estimated tomographic image group acquisition unit 73, and a display control unit 74. As an example, in the console 12 of this embodiment, the CPU 50A of the control unit 50 executes an image generation program 51 stored in the ROM 50B, so that the CPU 50A functions as the projection image acquisition unit 70, the tomographic image group generation unit 72, the estimated tomographic image group acquisition unit 73, and the display control unit 74.

[0093] The projection image acquisition unit 70 has a function of acquiring a plurality of projection images 90. Specifically, the projection image acquisition unit 70 of this embodiment acquires image data representing the plurality of projection images 90 acquired by tomosynthesis imaging in the mammography apparatus 10. The projection image acquisition unit 70 outputs the image data representing the acquired plurality of projection images 90 to the tomographic image group generation unit 72.

[0094] The tomographic image group generating unit 72 extracts the tomographic images from the plurality of projection images 90 acquired by the projection image acquiring unit 70. t The tomographic image group generating unit 72 has a function of generating a tomographic image group 96 representing a plurality of tomographic planes of the subject based on the above. The method by which the tomographic image group generating unit 72 generates the tomographic image group 96 is not particularly limited, and a known method can be used. For example, reconstruction may be performed by a back projection method such as the FBP method or the iterative reconstruction method, and known techniques can be applied. The method by which the tomographic image group generating unit 72 generates the tomographic image group 96 may be the same as or different from the method by which the virtual tomographic image group generating unit 83 of the support device 14 generates the virtual tomographic image group 98, but is preferably the same method. The tomographic image group generating unit 72 outputs image data representing the generated tomographic image group 96 to the estimated tomographic image group acquiring unit 73.

[0095] The estimated tomographic image group acquisition unit 73 has a function of acquiring an estimated tomographic image group 92 output from the tomographic image estimation model 63 .

[0096] As shown in FIG. 4 described above, the estimated tomographic image group acquisition unit 73 inputs the tomographic image group 96 generated by the tomographic image group generation unit 72 to the tomographic image estimation model 63. Specifically, the tomographic image group generation unit 72 of this embodiment outputs image data representing the tomographic image group 96 to the support device 14 via the I / F unit 54. In the support device 14, the tomographic image group 96 input from the console 12 is input to the tomographic image estimation model 63. As shown in FIG. 4, the tomographic image estimation model 63 outputs the estimated tomographic image group 92 in accordance with the input tomographic image group 96. The estimated tomographic image group acquisition unit 73 acquires image data representing the estimated tomographic image group 92 output from the tomographic image estimation model 63 of the support device 14 via the I / F unit 64 and the I / F unit 54.

[0097] The estimated tomographic image group acquisition unit 73 outputs image data representing the acquired estimated tomographic image group 92 to the display control unit 74.

[0098] The display control unit 74 has a function of displaying the estimated tomographic image group 92 acquired by the estimated tomographic image group acquisition unit 73 on the display unit 58. Note that the display destination of the estimated tomographic image group 92 is not limited to the display unit 58. For example, the display destination may be an image reading device or the like external to the radiation image capturing system 1.

[0099] Next, the operation of the console 12 of this embodiment in the operation phase will be described with reference to Fig. 11. As an example, when the mammography apparatus 10 of this embodiment finishes tomosynthesis imaging, it outputs image data of the captured multiple projection images 90 to the console 12. The console 12 stores the image data of the multiple projection images 90 input from the mammography apparatus 10 in the memory unit 52. The console 12 executes the image processing shown in Fig. 11 to generate a tomographic image using the multiple projection images obtained by tomosynthesis imaging and display the image on the display unit 58 or the like. The image processing shown in Fig. 11 is executed by the CPU 50A executing the image generation program 51 stored in the ROM 50B.

[0100] In step S200 of FIG. 11, the projection image acquisition unit 70 acquires the respective captured irradiation positions 19 t As described above, the projection image acquisition unit 70 of the present embodiment acquires image data of the projection images 90 from the storage unit 52.

[0101] In the next step S202, the tomographic image group generating unit 72 generates a tomographic image group 96 from the plurality of projection images 90 acquired in the above step S200. As described above, the tomographic image group generating unit 72 of this embodiment generates a tomographic image group 96 from the plurality of projection images 90 acquired by the projection image acquiring unit 70. t A group of tomographic images 96 representing a plurality of tomographic planes of the subject is generated based on the above.

[0102] In the next step S204, the estimated tomographic image group acquisition unit 73 inputs the tomographic image group 96 generated in the above step S202 to the tomographic image estimation model 63 of the support device 14. As described above, the estimated tomographic image group acquisition unit 73 outputs image data representing the tomographic image group 96 generated in the above step S202 to the support device 14, whereby a plurality of projection images 90 are input to the tomographic image estimation model 63 of the support device 14. As described above, the estimated tomographic image group 92 is output from the tomographic image estimation model 63 to which the tomographic image group 96 has been input.

[0103] Therefore, in the next step S206, the estimated tomographic image group acquisition unit 73 acquires the estimated tomographic image group 92 output from the tomographic image estimation model 63 of the support device 14 as described above.

[0104] In the next step S208, the display control unit 74 causes the tomographic images, which are the estimated tomographic image group 92 acquired in the above step S206, to be displayed on the display unit 58. When the process of step S208 ends, the image processing shown in FIG.

[0105] As described above, the console 12 of each of the above embodiments has the irradiation positions 19 with different irradiation angles. t The image processing device executes an operation phase for processing a plurality of projection images 90 obtained by sequentially irradiating a subject with radiation R from each of the projection images 90. The console 12 includes a CPU 50A. The CPU 50A acquires the plurality of projection images 90, and generates a group of tomographic images 96 representing a plurality of tomographic planes of the subject from the plurality of projection images 90. The CPU 50A inputs the generated group of tomographic images 96 to a tomographic image estimation model 63, and acquires a group of estimated tomographic images 92 output from the tomographic image estimation model 63.

[0106] The support device 14 is a learning device that executes a learning phase for performing machine learning of the tomographic image estimation model 63. The support device 14 includes a CPU 60A. The CPU 60A uses three-dimensional data representing a three-dimensional structure 110, and calculates a plurality of virtual irradiation positions 19 for the three-dimensional structure 110. tBy pseudo-projecting a ray R from the target object, a plurality of virtual projection images 91 onto which a three-dimensional structure 110 is projected are generated. The CPU 60A generates a virtual tomographic image group 98 from the plurality of virtual projection images 91. The CPU 60A trains a machine learning model using learning data 100 constituted by a set of ground truth data 102, which is three-dimensional data, and the virtual tomographic image group 98, thereby generating a tomographic image estimation model 63 that receives the tomographic image group 96 as an input and outputs an estimated tomographic image group 92.

[0107] The irradiation angle range of a tomographic image obtained by reconstructing the projection image 90 obtained by the mammography device 10 using a back projection method such as the FBP (Filter Back Projection) method or the iterative reconstruction method is limited. Therefore, for example, the image quality of the tomographic image may be lower than that of a CT image generated by a so-called perfect sinogram in which the restriction of the irradiation angle range is relaxed. In contrast, the console 12 of the above embodiment generates a tomographic image from the projection image 90 obtained by tomosynthesis imaging using a tomographic image estimation model 63 machine-learned using learning data 100 consisting of a set of correct answer data 102, which is three-dimensional data, and a virtual tomographic image group 98 generated from a plurality of virtual projection images 91 obtained by performing pseudo projection based on the correct answer data 102. Therefore, according to the above embodiment, a high-quality tomographic image can be generated by the tomographic image estimation model 63 trained using the learning data 100 corresponding to the projection image 90 obtained by tomosynthesis imaging. That is, according to the above embodiment, a high-quality tomographic image can be generated by the tomographic image estimation model 63 trained using the training data 100 corresponding to a relatively small number of projection images 90.

[0108] In the above embodiment, a breast is used as an example of the subject of the present disclosure, but the subject is not limited to the breast. For example, the subject may be a chest or an abdomen, and the radiation image capturing device may be a radiation image capturing device other than a mammography device. When the subject is a breast, a relatively large amount of noise is superimposed on the projection image 90 obtained by the mammography device 10. Therefore, as in the above embodiment, a virtual projection image 91 is used instead of the projection image 90 actually obtained by the mammography device 10, and a virtual tomography image group 98 generated from the virtual projection image 91 is used as the learning data 100, thereby generating a tomography image estimation model 63 that outputs a high-quality estimated tomography image group 92 with higher accuracy.

[0109] In the above embodiment, the CT image is used as the three-dimensional data used for the correct answer data 102. However, the three-dimensional data used as the correct answer data 102 is not limited to the CT image, and may be any three-dimensional data that represents the three-dimensional structure 110 of the subject. For example, the three-dimensional data may be obtained by simulating the three-dimensional structure 110 of the subject using a digital phantom. The digital phantom is a numerical construct that represents an object that mimics the clinical anatomical structure of the subject, and is configured by arranging a structure having a pixel value corresponding to the absorption coefficient of radiation R in a three-dimensional space. As the three-dimensional data using the digital phantom, for example, image data that represents an image having a pixel value corresponding to the amount of radiation that reaches the radiation detector 20, similar to a CT image, can be applied. In addition, the multiple correct answer data 102 used as the learning data 100 may include multiple types of three-dimensional data, and may include, for example, three-dimensional data using a CT image and three-dimensional data using a digital phantom.

[0110] In the above embodiment, since a CT image looks different from a general radiological image, an embodiment has been described in which an image obtained by correcting a CT image according to an absorption coefficient of the radiological image is used as the corrective data 102, but the corrective data 102 is not limited to this embodiment. For example, the corrective data 102 may be the CT image itself, or may be the three-dimensional data expressing the three-dimensional structure 110 itself.

[0111] In the above embodiment, the support device 14 generates one type of tomographic image estimation model 63, but the present invention is not limited to this embodiment, and multiple types of tomographic image estimation models 63 may be generated. For example, since the appearance of an image varies greatly depending on the amount of mammary glands, the support device 14 may generate the tomographic image estimation model 63 for each mammary gland content rate or mammary gland category. In this case, the support device 14 prepares multiple types of three-dimensional structures 110 with different mammary gland content rates or mammary gland categories, and performs machine learning using learning data 100 consisting of a set of the correct answer data 102 and a virtual tomographic image group 98 generated from the virtual projection image 91 for each type of three-dimensional structure 110, thereby generating multiple types of tomographic image estimation models 63 according to the mammary gland content rate or mammary gland category.

[0112] For example, a series of irradiation positions 19 t In this case, for example, the support device 14 may generate a tomographic image estimation model 63 for each of the virtual irradiation positions 19 with an irradiation angle range of −30 degrees to +30 degrees and with the irradiation angle in 5 degree increments. t A tomographic image estimation model 63 is generated using learning data 100 consisting of a set of a virtual tomographic image group 98 generated from a virtual projection image 91 obtained by the above and correct answer data 102. In addition, for example, the support device 14 generates virtual irradiation positions 19 with an irradiation angle range of -30 degrees to +30 degrees and irradiation angles in 3 degree increments. t A tomographic image estimation model 63 is generated using learning data 100 consisting of a set of a virtual tomographic image group 98 generated from a virtual projection image 91 obtained by the above and correct answer data 102. In addition, for example, the support device 14 generates virtual irradiation positions 19 with an irradiation angle range of -15 degrees to +15 degrees and irradiation angles in 5 degree increments. tA tomographic image estimation model 63 is generated using learning data 100 consisting of a set of a virtual tomographic image group 98 generated from a virtual projection image 91 obtained by the above and correct answer data 102. In this way, the support device 14 generates a tomographic image estimation model 63 by using a set of a virtual tomographic image group 98 generated from a virtual projection image 91 obtained by the above method. t A plurality of types of tomographic image estimation models 63 may be generated according to the above.

[0113] Also, for example, the support device 14 may generate a tomographic image estimation model 63 for the right breast and a tomographic image estimation model 63 for the left breast. In this case, the support device 14 can generate the tomographic image estimation model 63 for the right breast by performing machine learning only using the learning data 100 configured as a set of the correct answer data 102 corresponding to the three-dimensional structure 110 corresponding to the right breast and the virtual tomographic image group 98 generated from the virtual projection image 91. The support device 14 can generate the tomographic image estimation model 63 for the left breast by performing machine learning only using the learning data 100 configured as a set of the correct answer data 102 corresponding to the three-dimensional structure 110 corresponding to the left breast and the virtual tomographic image group 98 generated from the virtual projection image 91.

[0114] In the above embodiment, the support device 14 is an example of a learning device that executes the learning phase, and the console 12 is an example of an image processing device that executes the operation phase. However, the devices that operate as the learning device and the image processing device are not limited to the support device 14 and the console 12. For example, one device may have the functions of the learning device and the image processing device. As a specific example, the console 12 may have the functions of the learning device that executes the learning phase and the image processing device that executes the operation phase. In addition, some or all of the functions of the projection image acquisition unit 70, the tomographic image group generation unit 72, the estimated tomographic image group acquisition unit 73, and the display control unit 74 of the console 12 may be provided by a device other than the console 12, such as the mammography device 10, the support device 14, or an external device. In addition, some or all of the functions of the 3D data acquisition unit 80, the virtual projection image generation unit 82, the virtual tomographic image group generation unit 83, and the tomographic image estimation model generation unit 84 possessed by the support device 14 may be possessed by a device other than the support device 14, such as the mammography device 10, the console 12, or an external device.

[0115] In the above embodiment, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the projection image acquisition unit 70, the tomographic image group generation unit 72, the estimated tomographic image group acquisition unit 73, and the display control unit 74, and the hardware structure of the processing unit that executes various processes, such as the three-dimensional data acquisition unit 80, the virtual projection image generation unit 82, the virtual tomographic image group generation unit 83, and the tomographic image estimation model generation unit 84. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0116] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0117] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as represented by computers such as client and server. Second, there is a form in which a processor is used to realize the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as represented by a system on chip (SoC). In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0118] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0119] In the above embodiments, the photographing program 41 is pre-stored (installed) in the ROM 40B, the image generating program 51 is pre-stored (installed) in the ROM 50B, and the learning program 61 is pre-stored (installed) in the ROM 60B. However, the present invention is not limited to this. Each of the photographing program 41, the image generating program 51, and the learning program 61 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Each of the photographing program 41, the image generating program 51, and the learning program 61 may be downloaded from an external device via a network. [Explanation of symbols]

[0120] 1 Radiography system 10 Mammography equipment 12 Console 14 Support equipment 19 1 ~19 7 , 19 t Irradiation position 20 Radiation detector, 20A detection surface twenty one i Pixels 24 Imaging Table, 24A Imaging Surface 28 Radiation Department 29 Radiation source 33 Arm section 34 Foundation 35 Shaft 36 Compression Unit 38 Compression Plate 39 Support part 40A, 50A, 60A CPU, 40B, 50B, 60B ROM, 40C, 50C, 60C RAM 41 Shooting Program 42, 52, 62 Storage section 44, 54, 64 I / F section 46, 56, 66 Operation section 47 Source moving part 49, 59, 69 Bus 51 Image Generation Program 58, 68 Display section 61 Study Programs 63 Tomographic image estimation model 70 Projection image acquisition unit 72 Tomographic image group generation unit 73 Estimated tomographic image group acquisition unit 74 Display control section 80 3D Data Acquisition Section 82 Virtual projection image generation unit 83 Virtual tomographic image group generation unit 84 Tomographic image estimation model generation unit 90, 90 t Projected Image 91, 91 t Virtual Projection Image 92 Estimated tomographic images 94 1 , 94 j Voxel 96 Tomographic images 98 Virtual tomographic images 100 training data 102 Correct Data 110 3D structure CL normal R radiation, RC radiation axis U Subject w t ij Cross Length X t i route α, β angles

Claims

1. 1. An image processing device that processes a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, comprising: At least one processor; The processor, acquiring the plurality of projection images; generating a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images; A trained model is generated by machine learning a machine learning model using training data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected by using the three-dimensional data and performing pseudo-projection of radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, the trained model being generated by machine learning a group of tomographic images into a tomographic image estimation model which receives a group of tomographic images as input and outputs a group of estimated tomographic images, A group of estimated tomographic images output from the tomographic image estimation model is obtained. Image processing device.

2. The three-dimensional data, which is the correct data, is image data representing a plurality of correct tomographic images each corresponding to a different tomographic plane. The image processing device according to claim 1 .

3. The subject is a breast.

3. The image processing device according to claim 1 or 2.

4. At least one processor; The processor, using three-dimensional data representing a three-dimensional structure, pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure to generate a plurality of virtual projection images onto which the three-dimensional structure is projected; generating a group of virtual tomographic images from the plurality of virtual projection images; A machine learning model is trained using learning data consisting of a set of the ground truth data, which is the three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that receives the group of tomographic images as input and outputs a group of estimated tomographic images. Learning device.

5. The processor, and generating the plurality of virtual projection images by simulating the attenuation of radiation according to an absorption coefficient. The learning device according to claim 4.

6. The three-dimensional data, which is the correct data, is image data representing a plurality of correct tomographic images each corresponding to a different tomographic plane. The learning device according to claim 4 or 5.

7. The processor, A projection image used for generating each tomographic image of the group of tomographic images is generated. The plurality of virtual projection images are generated by adding a noise component corresponding to the dose of radiation assumed to reach the radiation detector. The learning device according to any one of claims 4 to 6.

8. The processor, The group of virtual tomographic images is generated from the plurality of virtual projection images using the FBP method or the iterative reconstruction method. The learning device according to any one of claims 4 to 7.

9. The plurality of virtual irradiation positions are positions simulating irradiation positions of radiation in tomosynthesis imaging. The learning device according to any one of claims 4 to 8.

10. the three-dimensional structure being a structure representative of a breast; The tomographic image group is a tomographic image group generated from projection images captured of a breast as a subject. The learning device according to any one of claims 4 to 9.

11. A radiation source that generates radiation; a radiation image capturing apparatus that performs tomosynthesis imaging by irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles and capturing a projection image of the subject for each of the irradiation positions; a processor uses three-dimensional data representing a three-dimensional structure, and pseudo-projects radiation from a plurality of virtual irradiation positions onto the three-dimensional structure to generate a plurality of virtual projection images onto which the three-dimensional structure is projected, generates a group of virtual tomographic images from the plurality of virtual projection images, and trains a machine learning model using learning data formed of a pair of ground truth data, which is the three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that takes a group of tomographic images as input and outputs a group of estimated tomographic images; an image processing device that processes a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, the image processing device including a processor that acquires the plurality of projection images, generates a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images, inputs the group of tomographic images into the tomographic image estimation model learned by the learning device, and acquires a group of estimated tomographic images output from the tomographic image estimation model; A radiation imaging system comprising:

12. 1. An image processing method for processing a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, comprising: acquiring the plurality of projection images; generating a group of tomographic images representing a plurality of tomographic planes of the subject based on the plurality of irradiation positions from the plurality of projection images; A trained model is generated by machine learning a machine learning model using training data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected by using the three-dimensional data and performing pseudo-projection of radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, the trained model being generated by machine learning a group of tomographic images into a tomographic image estimation model which receives a group of tomographic images as input and outputs a group of estimated tomographic images, A group of estimated tomographic images output from the tomographic image estimation model is obtained. An image processing method in which the processing is performed by a computer.

13. using three-dimensional data representing a three-dimensional structure, pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure to generate a plurality of virtual projection images onto which the three-dimensional structure is projected; generating a group of virtual tomographic images from the plurality of virtual projection images using the plurality of virtual irradiation positions; A machine learning model is trained using learning data consisting of a set of the ground truth data, which is the three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that receives the group of tomographic images as input and outputs a group of estimated tomographic images. A learning method in which a computer performs a process.

14. An image processing program for processing a plurality of projection images obtained by sequentially irradiating a subject with radiation from each of a plurality of irradiation positions having different irradiation angles, the program comprising: acquiring the plurality of projection images; generating a group of tomographic images representing a plurality of tomographic planes of the subject from the plurality of projection images; A trained model is generated by machine learning a machine learning model using training data consisting of a set of ground truth data, which is three-dimensional data representing a three-dimensional structure, and a group of virtual tomographic images generated from a plurality of virtual projection images onto which the three-dimensional structure is projected by using the three-dimensional data and performing pseudo-projection of radiation from a plurality of virtual irradiation positions onto the three-dimensional structure, the trained model being generated by machine learning a group of tomographic images into a tomographic image estimation model which receives a group of tomographic images as input and outputs a group of estimated tomographic images, A group of estimated tomographic images output from the tomographic image estimation model is obtained. An image processing program that causes a computer to carry out the processing.

15. using three-dimensional data representing a three-dimensional structure, pseudo-projecting radiation from a plurality of virtual irradiation positions onto the three-dimensional structure to generate a plurality of virtual projection images onto which the three-dimensional structure is projected; generating a group of virtual tomographic images from the plurality of virtual projection images; A machine learning model is trained using learning data consisting of a set of the ground truth data, which is the three-dimensional data, and the group of virtual tomographic images, thereby generating a tomographic image estimation model that receives the group of tomographic images as input and outputs a group of estimated tomographic images. A learning program that allows a computer to carry out processing.

Images